+j2ddlZdedefdZd\ZZeedz ZdZdZd Zd Z d Z d Z d Z dededefdZ dejdejfdZdejdededejfdZdejdededejfdZdZdZdZd(dejdedejfdZd(dedejfdZdejdejfdZd(d edejfd!Zd"ed#edejfd$Zd(defd%Zdejdejfd&Zdejdejfd'ZdS))Nnreturncd|zdz SN)rs \/home/wildlama/comfy/ComfyUI/.venv/lib/python3.11/site-packages/comfy_kitchen/float_utils.py_n_onesr s Fa<)rg@g?g|@g?g@g?xmultiplec||zdz |z|zS)z#Round up x to the nearest multiple.rr)rrs r roundupr s \A ( *h 66r cz|tjtjS)N)totorch float8_e4m3fnfloat32)rs r _float8_roundr%s' 44# $ $ ' ' 6 66r ebitsmbitsc |jtjksJd|z|zdksJt|dz }t||z}d||zz}tt|z dz }dt||z zt|dzd|zz z}dd|z z}t |z t|z zdz} | tz} tj| tjtj } |tj}|dz} || z }|tj}||k} tj tj | ||k}tj tj | |}|| z}|tj}|| z}| tj}|tj}|t|z z dz}|t z tz|z}||z }||z }|t|z z }| tj}tj||tj}tj|||}tj|||}| tt"z|z |z z }| tj}||z}||z}| tjS)aUConvert FP32 numbers to sub-byte floating point numbers with the given number of exponent and mantissa bits. Input: torch.Tensor of dtype torch.float Output: torch.Tensor of dtype torch.uint8, where the bit encoding is stored in the least significant bits. e.g. fp4: bits 0-3 empty and bits 4-7 in fp4_e2m1 encoding fp6: bits 0-1 empty and bits 2-7 in fp6_e2m3 or fp6_e3m2 encoding Note: there are no special values (NaN, inf) support in this code. Values outside the representable range of Floatx after rounding are clamped to the maximum Floatx magnitude (sign is preserved). Background 1: last answer in https://stackoverflow.com/questions/8981913/how-to-perform-round-to-even-with-floating-point-numbers # noqa: E501 Background 2: Computer Organization and Design, RISC-V edition, Chapter 3.5 rr )dtypel)rrfloatr MBITS_F32 F32_EXP_BIAStensorint32viewr logical_and logical_not logical_orruint8 full_likewhere EBITS_F32)rrrexp_biasmax_int sign_mask magic_adder max_normal min_normal denorm_expdenorm_mask_intdenorm_mask_floatsign saturate_mask denormal_mask normal_mask denormal_xnormal_xmant_odd val_to_addsign_lps r _f32_to_floatx_unpackedr=)s" 7ek ! ! ! ! u9u  ! ! ! !uqy!!Heem$$Geem$I)e+a/00Kwu~~01WUQY5G5G1e85TUJq8|$J  u    !I-O _EKHHHMMem\\ u{A z>D DA u{AOM%e&7 &F&FJWWM#E$4]M$R$RSSK&&J--J/!Ju{++J vvek""HY./14Hl*y8KGJ H HI-.H{{5;''H 7%+666A M:q11A K1--Ay9,u4u<=Gjj%%G  !G G A 44  r c|jtjksJd|z|zdksJd||zz}t|dz }t|}||z}||z }|dk}tj|dk||z dk} ||z } | |z t z} | tjtz} ||ztj} | t|z z} | | z}d||<d|z t z}|dkr||z tz|| <nxt|D]R}td|zd|dzzD]6}||z }|d|zz |tz|z z} ||z tz} | | z| | |k<7Stj | | |}|tjt|z tz|z z}||z}| tj S)aConvert sub-byte floating point numbers with the given number of exponent and mantissa bits to FP32. Input: torch.Tensor of dtype uint8, where the bit encoding is stored in the least significant bits. e.g. fp4: bits 0-3 empty and bits 4-7 in fp4_e2m1 encoding fp6: bits 0-1 empty and bits 2-7 in fp6_e2m3 or fp6_e3m2 encoding Output: torch.Tensor of dtype fp32 with the dequantized value rr r)rrr'r r$r rr"rranger)r*r#r)rrrr-r+ mantissa_maskr<x_pos zero_maskr6 exp_biased_lpexp_biased_f32mantissa_lp_int32 mantissa_f32resultdenormal_exp_biasedi mantissa_cmp left_shiftsign_f32s r _floatx_unpacked_to_f32rMs? 7ek ! ! ! ! u9u  ! ! ! !eem$Iuqy!!HENNM)mG KE  I %uqyUe^4IKKMUNM"X- N>NOOO14a4 A% 33Q37==i>N>NOOOr c|sJ|j}|dz tj}|dztj}|r9tj||gdt|}n8tj||gdt|}|S)z?Get the original weight from the normalized float weight formatr\rO)dim) is_contiguousr]rrr'stackr#rT)rYrZr]hilounpackeds r unpack_uint4rhs  # # % %%% %  E /  ek * *B v  ! !%+ . .BF;BxR00055gennEE;BxR00055gennEE Or packedcR|dz|dz ztjS)abExchange the high and low nibbles of each byte in a uint8 tensor. This converts between hi_first=True and hi_first=False FP4 packing without re-quantizing: ``0xAB`` becomes ``0xBA``. Args: packed: A uint8 tensor of nibble-packed FP4 values. Returns: A new uint8 tensor with the nibble order reversed in every byte. r\)rrr')ris r swap_nibblesrks'q[Vq[ ) - -ek : ::r flattenc|j\}}t|d}t|d}|dz}|dz}|}||f||fkr.tj||f|j|j}||d|d|f<||d|ddddd} | d dd d ddd d d } |r| S| ||S) aY Rearrange a large matrix by breaking it into blocks and applying the rearrangement pattern. See: https://docs.nvidia.com/cuda/cublas/index.html#d-block-scaling-factors-layout Args: input_matrix: Input tensor of shape (H, W) Returns: Rearranged tensor of shape (32*ceil_div(H,128), 16*ceil_div(W,4)) r\)devicerNrrrrO ) r]rXrzerosrorr#permutereshape transposerl) input_matrixrlrowscols n_row_blocks n_col_blocks padded_rows padded_colspaddedblocks rearrangeds r to_blockedr$s5#JD$D#&&LD!$$L$K"K F d| [111 + &&$    ,uuete|[[sL! < < D DQ1a P PFAr1--771==EEb"bQQJ$!!###   k; 7 77r num_rowsnum_colsct|d}t|d}|dz}|dz}|ddd}|dddddd}|||ddd} | ||dd} | dddd } | ||} | d |d |fS) aO Reverse the cuBLAS tiled layout back to normal (H, W) layout. Args: blocked_matrix: Swizzled tensor from cuBLAS layout (padded_rows, padded_cols) num_rows: Desired output rows (unpadded) num_cols: Desired output cols (unpadded) Returns: Unswizzled tensor of shape (num_rows, num_cols) rnr\rOrqrrrrrrpN)rXrurvrt) blocked_matrixrrrzr{r|r}step1step2step3step4step5 unblockeds r from_blockedrIsHc**LHa((L$K"K  " "2r2 . .E MM"b!Q ' ' 1 1!Q 7 7E MM, aQ ? ?E MM, c1 = =E MM!Q1 % %E k;77I YhY  ) **r cJt||}t|dd}|S)N)rZrr)rhrM)rVrZa_u8a_f32s r fp4_x2_to_f32rcs* H - - -D #D!Q / /E Lr c|jtjks Jd|tj}|t z dz}|t t z}|dt dz zk}||tjz}tj|dd}|tj S)NzInput must be float32rr) rrrr#r"rr rclampr')rx_int biased_expmantissaround_ups r f32_to_e8m0rhs 7em # # #%< # # # FF5;  E9$,Jwy)))HA)a-01Hhkk%+666JZC00J == % %%r c(|jtjks Jd|tj}|t z}tj|dktj||}|tj S)NzInput must be uint8r) rrr'rr"rr) zeros_liker#r)rrrGs r e8m0_to_f32rxss 7ek ! ! !#8 ! ! !ek""J 9 $F [q%*:6*B*BF K KF ;;u} % %%r )T)rintr r*rr F4_E2M1_MAX F4_E2M1_EPS F8_E4M3_MAX F8_E4M3_EPS F8_E5M2_MAX F8_E5M2_EPS E8M0_BIASrTensorrr=rMrRrTrXboolr_rhrkrrrrrrr r rs ss 9wy1}%%         7s7c7c7777 7U\7el7777mu|mCmm mmmm`T$u|T$CT$T$ T$T$T$T$n''' &&&PP5<P4P5<PPPP  t u|     ; ;%, ; ; ; ;"8"8d"8el"8"8"8"8J+3+#+%,++++4t &5< &EL & & & & &5<&EL&&&&&&r