3j7^SSKrSSKJr SSKJrJrJr SSKrSSK r SSK J r SSK J r Jr SSKJrJrJrJr SSKJrJrJr SS KJr SS KJr \(aSS KJr "S S \SS9r\"SS9\"SS\555rS/rg)N) accumulate) TYPE_CHECKINGOptionalUnion) BatchFeature) ImageInputis_valid_image)MultiModalDataProcessingKwargsProcessorMixinUnpack) AddedToken BatchEncoding TextInput)auto_docstring)requires)PreTokenizedInputc0\rSrSrSS0SSSS.SS0S .rS rg ) ColModernVBertProcessorKwargs(paddinglongestTchannels_first)return_row_col_info data_formatdo_convert_rgbreturn_tensorspt) text_kwargs images_kwargs common_kwargsN)__name__ __module__ __qualname____firstlineno__ _defaults__static_attributes__r#v/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/models/colmodernvbert/processing_colmodernvbert.pyrr(s/ y $(+" +D1 Ir*rF)total)torch)backendsc N^\rSrSr\rS"S\S\S-S\S-4U4Sjjjr\ S#S\ \ \ -\ \ \ -S\ \ S \ \ \ S 4S\S-S \\S \4 S jj5rS$S\ S-S\ \ S \ \ \ S 4S \\4S jjrS$S\ S-S\ \ S \ \ \ S 4S \\4U4SjjjrS\S\S \4SjrS\ S\ \S \ \ \4SjrS%SjrS%S\ S-S \\S \4SjjrS\ \ \ -S \\S \4SjrS&S\ S\ S4S\ S\ S4S\S\SS\ S\4S S4 S jjrS!rU=r$)'ColModernVBertProcessor6N image_seq_lenvisual_prompt_prefix query_prefixc >Sn[SSSS9RUl[SSSS9RUl[SSSS9RUlSUlX@lURUR5UlURUR5Ul URUR 5Ul [S 5VV s/sH3n[S 5H oRS US -S U S -S 35PM" M5 sn nUl [R"S5UlSURURUR/0n UR!U 5 URUR5Ul["T U]H"X4SU0UD6 U=(d SURS3UlU=(d SUlURUlgs sn nf)aJ image_seq_len (`int`, *optional*, defaults to 64): The length of the image sequence i.e. the number of tokens per image in the input. visual_prompt_prefix (`str`, *optional*): A string that gets tokenized and prepended to the image tokens. query_prefix (`str`, *optional*): A prefix to be used for the query. NzFT) normalizedspecialzzz z*(\n?\n?|\n?)+additional_special_tokens chat_templatez<|begin_of_text|>User:z0Describe the image. Assistant:)rcontentfake_image_token image_tokenend_of_utterance_tokenglobal_image_tagr2convert_tokens_to_idsimage_token_idfake_image_token_idglobal_image_token_idrange row_col_idsrecompile%_regex_to_remove_extra_special_tokensadd_special_tokenssuper__init__r3r4query_augmentation_token) selfimage_processor tokenizerr>r2r3r4kwargsij tokens_to_add __class__s r+rP ColModernVBertProcessor.__init__;s$ *+FSXbf g o o%iE4PXX&01ERWae&f&n&n# .*'==d>N>NO#,#B#B4CXCX#Y %.%D%DTEZEZ%["SXYZS[ S[aejklem`a + +eAE7%Awa,H Iem IS[  68ZZ@m5n2 (%%  ++*  $$]3'==d>N>NO [=[TZ[$8% $T%5%5$66g h !).B(,(C(C%1 s :GimagestextrrUreturnc UR"SXS.UD6upUR"SXS.UD6 UR"[4SURR 0UD6nUbUO UR nUSRSS5nUSRSS5nUSRSS5n0=pUGbUR"U40US D6upU RS S5 U RS S5 UbURX+S 9up,UR"U40USD6n U(aXS '/n [U 5Hgup/nUHInUSunnU RUU5nU RUUS- 5nURUU- S-5 MK U RU5 Mi U(aURU SU 5U S'URX*S/S9 OUbUR"SSU0USD6n [0U EU EUS9$)z image_seq_len (`int`, *optional*): The length of the image sequence. If not provided, the default value of self.image_seq_len is used. image_seq_len should be equal to int(((image_size // patch_size) ** 2) / (scale_factor**2)) )r[r\tokenizer_init_kwargsNr return_text_replacement_offsetsFreturn_mm_token_type_idsrr!rowscols)images_replacementstext_replacement_offsetsnew_spanr: input_idsmm_token_type_idsimage) modalitiesr\)data tensor_typer#)prepare_inputs_layoutvalidate_inputs _merge_kwargsrrT init_kwargsr2pop_process_imagesget_text_with_replacements enumerate char_to_tokenappendcreate_mm_token_type_ids_check_special_mm_tokensr)rRr[r\r2rU output_kwargsr`rar image_inputs text_inputsrdrebatch_image_seq_lengthsbatch_idtext_replacement_offsetimage_seq_lensrkstartend start_id_pos end_id_poss r+__call__ ColModernVBertProcessor.__call__psW11UUfU  @F@@** ) "&.."<"<  *7)B HZHZ *7 *F*J*JKlns*t'#0#?#C#CD^`e#f &}599:JDQ%''  040D0DV0n}]lOm0n -L   VT *   VT *151P1P2Q2.#nnTR]=5QR 2>V :;*,'9BC[9\5H%'N 7%)*%5 s'2'@'@5'Q %0%>%>xq%Q &--j<.G!.KL !8 ,22>B:],7;7T7T#K02I8K 34--dWI-V  ..SdSmM6RSK!@K!@Ff%%!($|#FT5M22~fQi7P7P#UY'ZUY6 T5E5E(FUY$'Z-.C$zBR7S2T,T)"'s+;'>!-8Mb8Q8S1T0U!U | ".|%XF|([$s $D.*D3c >[TU]"X40UD6 UcUc [S5eUbUVs/sHoDRUR5PM nnUbMUVs/sHn[ U5PM nnXW:wa,[SURSUSURSUS3 5egUc6[ U5(a%[S[U5SURS35egggs snfs snf) Nz+You must provide either `text` or `images`.zThe total number of zP tokens in the prompts should be the same as the number of images passed. Found  z tokens and z images per sample.zFound z. tokens in the text but no images were passed.)rOrn ValueErrorrrBranysum) rRr[r\rUrrsublistn_images_in_imagesrYs r+rn'ColModernVBertProcessor.validate_inputss 77 T R!BH%I&wc'l&"%I#9$.t/?/?.@A""2!31T5E5E4FlSeRffy{: C(8$9$9 S!1231T5E5E4FFtu%: R%Is $CC!rz image_idxc USVVs/sH o3HoDPM M snnUnUSVVs/sH ofHowPM M snnUnUS:XaJUS:XaDURUR-URUR--UR-$Sn [ U5HUn [ U5H>n U URSU S-SU S-S3-URUR--- n M@ U S - n MW U S UR3UR-URUR--UR-- n U $s snnfs snnf) Nrbrcrr?r9r:r;r< )rArDrBr2rI) rRrzrrow_listrow image_rowscol_listcol image_colstext_split_imagesn_hn_ws r+replace_image_token+ColModernVBertProcessor.replace_image_tokens*6v*>S*>h(3c(c*>ST]^ *6v*>S*>h(3c(c*>ST]^ ?zQ(()**+-%%&$*<*<<=**+- !# Z( ,C%001!#'%ay:;!--.$2D2DDE%- "T)!) T**+,**+-%%&$*<*<<=**+-   % $5TSs EErgr|c/n[U5HupE[R"X5n[R"U5n[R"X`R :H5SnSn UH6n U [ U5:a O&Xn X-n SX{U &[R"X5n M8 URUR55 M U$)Nrr:) rtnparray zeros_likewhererGr searchsortedrvtolist) rRrgr|rhrV seq_lengths array_idsmm_token_typesimage_start_positionsrWseq_lenrrs r+rw0ColModernVBertProcessor.create_mm_token_type_ids s'(?@NA.I]]95N$&HHY:R:R-R$STU$V !A&122-0o,-S)OO$9? '  $ $^%:%:%< =A! r*c 0nUb[RRS05nURU5 UVs/sH!nURR "/UQUP76PM# nnUR S-nUR S-n/n /n UH4upn X-S-nU RX~U --5 U RU 5 M6 URXS.5 [S0UD6$s snf)a{ Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. r!rr:)num_image_tokensnum_image_patchesr#) rr(getupdaterSget_number_of_image_patchesr2rvr )rR image_sizesrU vision_datar! image_sizenum_image_row_colsbase_image_length col_lengthrr num_patchesnum_rowsnum_cols row_lengths r+_get_num_multimodal_tokens2ColModernVBertProcessor._get_num_multimodal_tokenss   "9CCGGY[\M   (#.""-J$$@@\*\m\"- " !% 2 2Q 6 ++a/J!  " 3E/ x'2Q6  ''(9(=R(ST!((54F   4Dm n, ,,#"s(Cc UR"[4SURR0UD6nUSR SS5nUSLn[ U5(aU/nOw[ U[5(a[ US5(aON[ U[5(a.[ US[5(a[ USS5(d [S5eUVs/sHofRS5PM nnURUR/[U5-UUSUSS 9nU(a.US RUS S:HS 5nURS U05 U$s snf)au Prepare for the model one or several image(s). Handles input validation, RGB conversion, and prepends the `visual_prompt_prefix` to each image. Optionally computes labels from `token_type_ids` when a `suffix` is provided in `text_kwargs`. Args: images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `list[PIL.Image.Image]`, `list[np.ndarray]`, `list[torch.Tensor]`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. In case of a NumPy array/PyTorch tensor, each image should be of shape (C, H, W), where C is a number of channels, H and W are image height and width. return_tensors (`str` or [`~utils.TensorType`], *optional*): If set, will return tensors of a particular framework. Acceptable values are: - `'pt'`: Return PyTorch `torch.Tensor` objects. - `'np'`: Return NumPy `np.ndarray` objects. Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. r_r suffixNrzAimages must be an image, list of images or list of list of imagesRGBr!)r\r[r!r rgtoken_type_idsilabels)rorrTrprqr rrrconvertrr3r masked_fillr) rRr[rUryrreturn_token_type_idsri batch_docrs r+process_images&ColModernVBertProcessor.process_imagesCsj<** ) "&.."<"<  }-11(DA &d 2 & ! !XF  % %.*C*C VT**z&)T/J/J~^def^ghi^jOkOk`a a5;;F5--&F;MM++,s6{:'8%m4 "  !{+77 BR8SWX8XZ^_F   h/ 022R7FSW!WRV"3"3e";f"DRV !Wmm"'%m4$ "Xs*Cquery_embeddingsz torch.Tensorpassage_embeddings batch_size output_dtypez torch.dtype output_devicez torch.devicec [U5S:Xa [S5e[U5S:Xa [S5eUSRUSR:wa [S5eUSRUSR:wa [S5eUcUSRn/n[ S[U5U5GHn/n[ R RRRXXs-SSS9n [ S[U5U5H}n [ R RRRX*X-SSS9n UR[ R"SX5RS S 9SRS S 95 M UR[ R"US S 9RU5RU55 GM [ R"USS 9$) a Compute the late-interaction/MaxSim score (ColBERT-like) for the given multi-vector query embeddings (`qs`) and passage embeddings (`ps`). For ColQwen2, a passage is the image of a document page. Because the embedding tensors are multi-vector and can thus have different shapes, they should be fed as: (1) a list of tensors, where the i-th tensor is of shape (sequence_length_i, embedding_dim) (2) a single tensor of shape (n_passages, max_sequence_length, embedding_dim) -> usually obtained by padding the list of tensors. Args: query_embeddings (`Union[torch.Tensor, list[torch.Tensor]`): Query embeddings. passage_embeddings (`Union[torch.Tensor, list[torch.Tensor]`): Passage embeddings. batch_size (`int`, *optional*, defaults to 128): Batch size for computing scores. output_dtype (`torch.dtype`, *optional*, defaults to `torch.float32`): The dtype of the output tensor. If `None`, the dtype of the input embeddings is used. output_device (`torch.device` or `str`, *optional*, defaults to "cpu"): The device of the output tensor. Returns: `torch.Tensor`: A tensor of shape `(n_queries, n_passages)` containing the scores. The score tensor is saved on the "cpu" device. rzNo queries providedzNo passages providedz/Queries and passages must be on the same devicez-Queries and passages must have the same dtypeT) batch_first padding_valuez bnd,csd->bcnsr)dimrr:)rrdevicedtyperIr-nnutilsrnn pad_sequencerveinsummaxrcatto) rRrrrrrscoresrV batch_scores batch_queriesrWbatch_passagess r+score_retrieval'ColModernVBertProcessor.score_retrievals@  A %23 3 ! "a '34 4 A  % %);A)>)E)E ENO O A  $ $(:1(=(C(C CLM M  +A.44L%'q#./:\]"A"##LL-PTTYZT[\]^bbghbi C MM%))La8;;LILL][ \=yyQ''r*) rMrCrArGrDrHr2rBrFrQr4rJr3)NN@NN)NNN)NN)N)Ncpu)r$r%r&r'rvalid_processor_kwargsintrrPrr rrrrrrrmr rndictrrwrrrrrrr) __classcell__)rYs@r+r0r06s; +/#'3D  3D "Dj 3DDj3D3DjJNbf$( >^T*--T*5E0FF>^I2DOTJ]E^^_>^Tz >^ 67 >^  >^>^D%)bf T! I2DOTJ]E^^_ 67 H%)bfT!I2DOTJ]E^^_)* 2%%%%:!$!QUVYQZ!_cdhildm_n!*#-N%)@T!@67@  @D7$y/)7677  7z0449 >(^0D DE>(".$~2F"FG>( >( }- >( ^S01 >( >(>(r*r0) rK itertoolsrtypingrrrnumpyrr-feature_extraction_utilsr image_utilsr r processing_utilsr r r rtokenization_utils_baserrrrrutils.import_utilsrrrr0__all__r#r*r+rs~* 11 45XXKK#*< $4E  :D(nD(D(N % %r*