3j_ESSKJr SSKJrJr SSKrSSKJs Jr SSK J r SSK J r SSKJrJrJr SSKJr SSKJr SS KJr SS KJrJr SS KJrJr "S S 5rg)) annotations)AnyOptionalN)LongestMaxSize)AugmentationSequential) KORNIA_CHECKKORNIA_CHECK_IS_TENSORKORNIA_CHECK_SHAPE) normalize)Boxes) Keypoints)Sam SamConfig)PromptsSegmentationResultsc^\rSrSrSrSSU4SjjjrSSSjjr\R"5SSSjj5r SSjr SSjr SS jr SS .SS jjr SSS jjr\R"5SS S jj5rS!SjrSSSSSSS.S"SjjrSrU=r$)#VisualPrompter#aAllow the user to run multiple query with multiple prompts for a model. At the moment, we just support the SAM model. The model is loaded based on the given config. For default the images are transformed to have their long side with size of the `image_encoder.img_size`. This Prompter class ensure to transform the images and the prompts before prediction. Also, the image is passed automatically for the method `preprocess_image`, which is responsible for F.normalize the image and F.pad it to have the right size for the SAM model :math:`(\text{image_encoder.img_size}, \text{image_encoder.img_size})`. For default the image is normalized by the mean and standard deviation of the SAM dataset values. Args: config: A model config to generate the model. Now just the SAM model is supported. device: The desired device to use the model. dtype: The desired dtype to use the model. Example: >>> # prompter = VisualPrompter() # Will load the vit h for default >>> # You can load a custom SAM type for modifying the config >>> prompter = VisualPrompter(SamConfig('vit_b')) >>> image = torch.rand(3, 25, 30) >>> prompter.set_image(image) >>> boxes = Boxes( ... torch.tensor( ... [[[[0, 0], [0, 10], [10, 0], [10, 10]]]], ... device=prompter.device, ... dtype=torch.float32 ... ), ... mode='xyxy' ... ) >>> prediction = prompter.predict(boxes=boxes) >>> prediction.logits.shape torch.Size([1, 3, 256, 256]) Nc6>[TU]5 Uc [SSS9n[U[5(a[R "U5Ul[UR RRSS94n[R"/SQX#S9S- Ul [R"/S QX#S9S- Ul O[eUR RX#S9Ul[!US S06UlX lX0lSUlSUlSUlUR/5 g) Nvit_hT) model_type pretrainedg?)p)g33333^@gR]@gRY@)devicedtype)g(\2M@g(\L@gL@ same_on_batch)super__init__r isinstancer from_configmodelr image_encoderimg_sizetorchtensor pixel_mean pixel_stdNotImplementedErrortor transformsrr_original_image_size_input_image_size_input_encoder_size reset_image)selfconfigrrr+ __class__s X/home/wildlama/miniconda3/lib/python3.13/site-packages/kornia/contrib/visual_prompter.pyrVisualPrompter.__init__Gs  >'dCF fi ( (0DJ()A)A)J)JcRTJ 6vSVYY O 4VQTWW N& %ZZ]]&]> 0*QDQ  <@!9=;?  c:[U[R5(a,[U[R5(a [XU5nOr[UR[R5(aI[UR [R5(a [XRUR 5nUR RRnXARS- nXARS- n[R"USUSU45nU$)aNormalize and F.pad a torch.Tensor. For F.normalize the tensor: will prioritize the `mean` and `std` passed as argument, if None will use the default Sam Dataset values. For F.pad the tensor: Will F.pad the torch.Tensor into the right and bottom to match with the size of `self.model.image_encoder.img_size` Args: x: The image to be preprocessed mean: Mean for each channel. std: Standard deviations for each channel. Returns: The image preprocessed (normalized if has mean and str available and padded to encoder size) r) r r%Tensorkornia_normalizer'r(r"r#r$shapeFpad)r0xmeanstdencoder_im_sizepad_hpad_ws r3preprocess_imageVisualPrompter.preprocess_imagegs* dELL ) )jell.K.K #.A  6 6:dnnV[VbVb;c;c OOT^^DA**22;;''"+-''"+- EE!a5) *r5c<URS:Xa [U/SQ5 URS5nO[U/SQ5 UR5 URSURS4UlUR US/S9nUR RUlURSURS4Ul URXU5nURSURS4Ul URRU5UlS Ulg ) aSet the embeddings from the given image with `image_decoder` of the model. Prepare the given image with the selected transforms and the preprocess method. Args: image: RGB image. Normally images with range of [0-1], the model preprocess F.normalize the pixel values with the mean and std defined in its initialization. Expected to be into a float32 dtype. Shape :math:`(3, H, W)`. mean: mean value of dataset for normalization. std: standard deviation of dataset for normalization. )3HWr)BrHrIrJr7r8input data_keysTN)ndimr unsqueezer/r;r,r+_params _tfs_paramsr-rDr.r"r#image_embeddings is_image_set)r0imager?r@s r3 set_imageVisualPrompter.set_images ::? uo 6OOA&E u&: ; %*[[_ekk"o$F!';??22"'++b/5;;r?!C%%e37$)KKOU[[_#E $ 8 8 ? r5c"[UR/SQ5 [URSS/5 [URSURS:HS5 [ U[ R 5(a[R"U5nU$)z:Validate the keypoints shape and ensure to be a Keypoints.)KN2rYrZrz8The keypoints and labels should have the same batch size) r datarr;r r%r9r from_tensor)r0 keypointslabelss r3_valid_keypointsVisualPrompter._valid_keypointssj9>>?;6;;c 3Y__Q'6<<?:G >RS>R(#-->RSSSsA#cz/n/n[U[[R45(aR[U[R5(a3UR X5nUR S5 UR U5 [U[ [R45(a3URU5 UR S5 UR U5 [U[R5(aURU5 UR"USU06nSU;aO[US[5(a7USR5n[U5(a[U5(aX4n OSn SU;a=[US[ 5(a%USRSS9n [U 5(aU n OSn [W W US9$)zMValidate and preprocess the given prompts to be aligned with the input image.r^ bbox_xyxyrNNrerfpointsrirp) r r r%r9r`appendr rkrqr|rhr r) r0r^keypoints_labelsrirprN to_transformr\ kpts_tensorr_bboxbboxs r3preprocess_prompts!VisualPrompter.preprocess_promptss ?A i)U\\!: ; ; K[]b]i]i@j@j--iJI   [ )    * eeU\\2 3 3   e $   [ )    & eU\\ * *   e $&& J J $ :d;.?#K#K{+557K%k227MN^7_7_%8F $ :d;.?#G#G%//V/>r5cf[URS5 URXX45nURR UR UR URS9upAURRURURRR5UU US9up[X5n U(ao[UR[5(aP[UR[5(a1U R!URURUR"5 U $)aPredict masks for the given image based on the input prompts. Args: keypoints: Point prompts to the model. Each point is in (X,Y) in pixels. Shape :math:`(K, N, 2)`. Where `N` is the number of points and `K` the number of prompts. keypoints_labels: Labels for the point prompts. 1 indicates a foreground point and 0 indicates a background point. Shape :math:`(K, N)`. Where `N` is the number of points, and `K` the number of prompts. boxes: A box prompt to the model. If a torch.Tensor, should be in a xyxy mode. Shape :math:`(K, 4)` masks: A low resolution mask input to the model, typically coming from a previous prediction iteration. Has shape :math:`(K, 1, H, W)`, where for SAM, H=W=256. multimask_output: If true, the model will return three masks. For ambiguous input prompts (such as a single click), this will often produce better masks than a single prediction. If only a single mask is needed, the model's predicted quality score can be used to select the best mask. For non-ambiguous prompts, such as multiple input prompts, multimask_output=False can give better results. output_original_size: If true, the logits of `SegmentationResults` will be post-process to match the original input image size. Returns: A prediction with the logits and scores (IoU of each predicted mask) zKAn image must be set with `self.set_image(...)` before `predict` be called!r)rSimage_pesparse_prompt_embeddingsdense_prompt_embeddingsmultimask_output)rrTrr"prompt_encoderrrirp mask_decoderrS get_dense_perr r-rvr,original_res_logitsr.) r0r^rrirproutput_original_sizerxsparse_embeddingsdense_embeddingslogitsscoresresultss r3predictVisualPrompter.predictsB T&&(uv)))uT/3jj.G.G>>gmm/H/ + 00!22ZZ..;;=%6$4- 1 &f5 411599444e<<  ' '(>(>@Y@Y[_[s[s tr5c~SUlSUlSUlSUl[ US5(aU?SUlSUlg)a|Clear cached image state and prompt-transform metadata. This method invalidates previously computed image embeddings and resets all size/transform bookkeeping so a new call to :meth:`set_image` starts from a clean state. In practice, this resets: - transformed-image parameters, - original/input/encoder spatial sizes, - cached image embeddings, - ``is_image_set`` status flag. NrSF)rRr,r-r.hasattrrSrT)r0s r3r/VisualPrompter.reset_imageCsH $(!!%#' 4+ , ,% $!r5Finductor fullgraphdynamicbackendrgoptionsdisablec r[R"URRUUUUUUS9URl[R"URRUUUUUUS9URl[R"URR UUUUUUS9URlg)uApply `torch.compile(...)`/dynamo API into the VisualPrompter API. .. note:: For more information about the dynamo API check the official docs https://pytorch.org/docs/stable/generated/torch.compile.html Args: fullgraph: Whether it is ok to break model into several subgraphs dynamic: Use dynamic shape tracing backend: backend to be used mode: Can be either “default”, “reduce-overhead” or “max-autotune” options: A dictionary of options to pass to the backend. disable: Turn torch.compile() into a no-op for testing Example: >>> # prompter = VisualPrompter() >>> # prompter.compile() # You should have torch >= 2.0.0 installed >>> # Use the prompter methods ... rN)r%compiler"r#rr)r0rrrrgrrs r3rVisualPrompter.compile[sN$)== JJ $ $$   >#(-- JJ # ##  %*MM JJ % %%  !r5) r.r-r,rRrrrSrTr"r'r(r+)NNN)r1zOptional[SamConfig]rzOptional[torch.device]rzOptional[torch.dtype]returnNone)NN)r> torch.Tensorr?Optional[torch.Tensor]r@rrr)rUrr?rr@rrr)r^zKeypoints | torch.Tensorr_rrr )rizBoxes | torch.Tensorrr )rprrr)rxz torch.Tensor | Boxes | KeypointsrNzOptional[list[str]]rz+dict[str, torch.Tensor | Boxes | Keypoints])NNNN) r^"Optional[Keypoints | torch.Tensor]rrriOptional[Boxes | torch.Tensor]rprrr)NNNNTT)r^rrrrirrprrboolrrrr)rr)rrrrrstrrgz Optional[str]rzOptional[dict[Any, Any]]rrrr)__name__ __module__ __qualname____firstlineno____doc__rrDr%no_gradrVr`rkrqr|rrr/r__static_attributes__ __classcell__)r2s@r3rr#s!J'+)-'+ #'%   Bcg%;I_ B ]]_fj"!!"!)?"!Mc"! "!"!H " \` T8 TEX T 4 T9=3704(, (?5(?1(?. (? & (?  (?T ]]_9=3704(,!%%)<5<1<. < & <  <#< <<|"6 !",0W W  W  W  W *W W  W W r5r) __future__rtypingrrr%torch.nn.functionalnn functionalr<kornia.augmentationr%kornia.augmentation.container.augmentrkornia.core.checkrr r kornia.enhancer r:kornia.geometry.boxesr kornia.geometry.keypointsr kornia.models.samrrkornia.models.structuresrrrr5r3rs=$# .HVV8'/,AO O r5