Reconstruction of Smooth 3D Color Functions from Keypoints: Application to Lossy Compression and Examplar-Based Generation of Color LUTs D. Tschumperlé C. Porquet A. Mahboubi Normandie Univ, UNICAEN, ENSICAEN, CNRS GREYC, F-14000 Caen, France June 2020 Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 1 / 32
Context G REYC’s M agic for I mage C omputing https://gmic.eu Application context: Provide various colorimetric transformations available in our open-source framework for image processing. Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 2 / 32
What is a CLUT ( Color Look-Up Table ) ? (b) Original color image (a) CLUT F : RGB → RGB , visualized in 3 D (c) Image after transformation F Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 3 / 32
Examples of of CLUT -based transformations Original image “60’s” “Color Negative” “Orange Tone” “Ilford Delta 3200” “Backlight Filter” “Bleach Bypass” “Late Sunset” Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 4 / 32
Standard ways of storing a CLUT a) a CLUT is a 3 D b) Storage as a .cube c) Storage as a .png dense color volume In both cases, lossless compression, but restricted to small sizes: .cube file: ASCII zipped format ( CLUT 64 3 ≈ 1 Mo) 1 .png file: 2 D image ( CLUT 64 3 ≈ 64 to 100 Ko) 2 ⇒ Issue: Promote a large-scale distribution of CLUT s ( 500+ ), by limiting the number of parameters as much as possible. Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 5 / 32
Our approach: CLUT compression Compression: Let F : RGB → RGB be a 3 D CLUT . We generate K , a smaller representation based on the storage of a set of color keypoints. − → − → Storage in Determination of 3 D Original CLUT F compressed color keypoints K form Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 6 / 32
Our approach: CLUT decompression Decompression: A 3 D interpolation based on anisotropic diffusion PDEs is applied to K in order to generate a reconstructed CLUT � F visually close to F . → Stockage in Decompressed No visible perceptual differences compressed CLUT � F between the two transformations form Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 7 / 32
Reconstruction principles (1/3) Let F : RGB → RGB be a 3 D CLUT . It is assumed that its sparse representation is known. K = { K k ∈ RGB × RGB | k = 1 . . . N } i.e the color keypoints N , located in the RGB cube. The k th keypoint of K is defined by vecteur K k = ( X k , C k ) = ( x k , y k , z k , R k , G k , B k ) , where X k = ( x k , y k , z k ) is the 3 D keypoint position in the RGB cube, and C k = ( R k , G k , B k ) its associated color. Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 8 / 32
Reconstruction principles (2/3) We propagate/average the colors C k of the keypoints in the whole RGB domain throgh an anisotropic diffusion process Let d K : RGB → R + , the distance function to the set of keypoints K : ∀ X ∈ RGB , d K ( X ) = inf k ∈ 0 ... N � X − X k � CLUT F is reconstructed by solving the following anisotropic diffusion PDE : ∂ 2 F ∂ F ∀ X ∈ RGB , ∂ t ( X ) = m ( X ) ∂η 2 ( X ) � 0 ∇ d K ( X ) if ∃ k , X = X k η = m ( X ) = where and 1 otherwise �∇ d K ( X ) � Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 9 / 32
Reconstruction principles (3/3) (b) Initial state F t=0 ( Voronoï 3 D smoothed ) (a) Set K of known keypoints (d) State at convergence ( PDE Solution) (c) Diffusion orientations η Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 10 / 32
Spatial discretization (1/2) d K is not derivable on its local maxima. We propose the following numerical scheme for discretization: maxabs ( ∂ for x d K , ∂ back d K ) x maxabs ( ∂ for y d K , ∂ back ∇ d K ( X ) = d K ) y maxabs ( ∂ for z d K , ∂ back d K ) z where � a if | a | > | b | maxabs ( a , b ) = b otherwise and ∂ for x d K = d K ( x + 1 , y , z ) − d K ( x , y , z ) ∂ back d K = d K ( x , y , z ) − d K ( x − 1 , y , z ) x (Discrete forward / backward first derivative approximations). Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 11 / 32
Spatial discretization (2/2) (a) Keypoints and (b) Estimation of η using (c) Estimation of η using forward scheme ∂ for d K backward scheme ∂ back d K distance function d K (d) Estimation of η using centred (e) Estimation of η using scheme 1 2 ( ∂ for d K + ∂ back d K ) proposed scheme Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 12 / 32
Temporal discretization For the sake of algorithmic efficiency, we discrete the PDE by a semi-implicit scheme: � � F t + dt − F t F t ( X + η ) + F t ( X − η ) − 2 F t + dt = m ( X ) ( X ) dt By choosing dt appropriately, we simplify the scheme by: F t + dt ( X ) = F t if m ( X ) = 0 ( X ) � � F t + dt ( X ) = 1 F t ( X + η ) + F t otherwise 2 ( X − η ) where F t ( X + η ) and F t ( X − η ) are accurately estimated using tricubic spatial interpolation. Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 13 / 32
Multi-scale resolution Speed-up of convergence through multi-scale resolution . → Reduction of the number of required iterations per scale ( ≈ 20). → Total algorithmic complexity for a size r 3 : O ( log 2 ( r ) r 3 ) . → Reconstruction of a 64 3 CLUT in less than 1s. Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 14 / 32
Isotropic/Anisotropic comparison (a) (b) (a) Color Keypoints K (b) Isotropic reconstruction, in O ( log 2 ( r ) r 3 ) (c) (c) Anisotropic reconstruction, in O ( log 2 ( r ) r 3 ) Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 15 / 32
Comparison with RBFs (Radial Basis Functions) (a) (b) (a) Color Keypoints K (b) Anisotropic reconstruction, in O ( log 2 ( r ) r 3 ) (c) (c) RBFs reconstruction, in O ( N 3 + N r 3 ) . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 16 / 32
Compression: Generation of keypoints Let F be the CLUT to be compressed. The compression: Generates a set K of N keypoints representing F . � F N the reconstructed CLUT from K must be close enough to F . Two quality criteria for reconstruction: ∆ max = 8, maximum reconstruction error allowed, ∆ moy = 2, average reconstruction error for the entire CLUT − → − → Storage in Determination of 3 D Original CLUT F compressed color keypoints K form Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 17 / 32
Compression: Generation of keypoints Initialization of K = { ( X k , F ( X k ) | k = 1 . . . 8 } (the 8 vertices of the 1 cube). − → ⇒ Calculation of the L 2 reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 18 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 19 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 20 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 21 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 22 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 23 / 32
Compression: Generation of keypoints Iterative addition into K of the keypoints with maximum 2 reconstruction error, while E max > ∆ max or E moy > ∆ moy . − → ⇒ Calculation of the reconstruction error: E N ( X ) = � F ( X ) − � F N ( X ) � . Tschumperlé, Porquet, Mahboubi Compression and Examplar-Based Generation of CLUTs 24 / 32
Recommend
More recommend
