cfa interpolation detection
play

CFA Interpolation Detection Leszek Swirski October 15, 2009 - PowerPoint PPT Presentation

Outline CFA Introduction Interpolation Detection CFA Pattern Synthesis CFA Interpolation Detection Leszek Swirski October 15, 2009 Leszek Swirski CFA Interpolation Detection Outline CFA Introduction Interpolation Detection CFA


  1. Outline CFA Introduction Interpolation Detection CFA Pattern Synthesis CFA Interpolation Detection Leszek ´ Swirski October 15, 2009 Leszek ´ Swirski CFA Interpolation Detection

  2. Outline CFA Introduction Interpolation Detection CFA Pattern Synthesis CFA Introduction Example Interpolation Interpolation Detection Methods Examples Identifying Forged Regions CFA Pattern Synthesis Reasoning Methods Leszek ´ Swirski CFA Interpolation Detection

  3. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis CFA? ▶ “Colour Filter Array” ▶ Photosensors have no wavelength specificity ▶ So filter RGB onto array of photosensors ▶ e.g. Bayer filter Leszek ´ Swirski CFA Interpolation Detection

  4. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Target Image Leszek ´ Swirski CFA Interpolation Detection

  5. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Filters Leszek ´ Swirski CFA Interpolation Detection

  6. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Sensor Data Leszek ´ Swirski CFA Interpolation Detection

  7. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Coloured Sensor Data Leszek ´ Swirski CFA Interpolation Detection

  8. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Coloured Sensor Data (Detail) Leszek ´ Swirski CFA Interpolation Detection

  9. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Why interpolate? ▶ Each pixel is only R, G or B ▶ Want full colour, full size image ▶ So guess interpolate! Leszek ´ Swirski CFA Interpolation Detection

  10. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

  11. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

  12. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

  13. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) and B R G ⎡ ⎤ ⎡ ⎤ 1 2 1 0 1 0 R ′ / B ′ = R / B ∗ 1 G ′ = G ∗ 1 2 4 2 1 4 1 ⎣ ⎦ ⎣ ⎦ 4 4 1 2 1 0 1 0 Leszek ´ Swirski CFA Interpolation Detection

  14. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) Leszek ´ Swirski CFA Interpolation Detection

  15. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) Leszek ´ Swirski CFA Interpolation Detection

  16. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Seperate luminance ( G ) and chrominance ( R and B ). Interpolate G bilinearly ⎡ ⎤ 0 1 0 G ′ = G ∗ 1 1 4 1 ⎣ ⎦ 4 0 1 0 ij = R ij For R (and similarly for B ), interpolate the ratio R ′′ ij , and G ′ pointwise multiply by G ⎛ ⎡ ⎤ ⎞ 1 2 1 ⎝ R ′′ ∗ 1 R ′ ij = G ij × 2 4 2 ⎣ ⎦ ⎠ 4 1 2 1 ij Leszek ´ Swirski CFA Interpolation Detection

  17. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Leszek ´ Swirski CFA Interpolation Detection

  18. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Leszek ´ Swirski CFA Interpolation Detection

  19. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter ▶ Bilinearly filter R , G , B to get R ′′ , G ′′ , B ′′ . ▶ Calculate pairwise differences ( R ′′ − G ′′ , R ′′ − B ′′ , G ′′ − B ′′ ) ▶ Median filter these to get M rg , M rb , M gb ▶ Each resulting pixel is CFA pixel image plus/minus appropriate median. e.g. (1,0) is a green pixel in CFA, so R ′ 1 , 0 = G 1 , 0 + ( M rg ) 1 , 0 G ′ 1 , 0 = G 1 , 0 B ′ 1 , 0 = G 0 , 0 − ( M gb ) 1 , 0 Leszek ´ Swirski CFA Interpolation Detection

  20. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter Leszek ´ Swirski CFA Interpolation Detection

  21. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter Leszek ´ Swirski CFA Interpolation Detection

  22. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based ▶ Want to preserve edges, so ‘adaptively’ interpolate G ▶ Approximate horizontal and vertical second derivatives of R and B and take absolutes, e.g. � � � � ∂ 2 R R i , j − 2 + R i , j +2 � � � � ≈ − R i , j � � � � ∂ x 2 2 � � � � i , j ▶ Compare these H and V . If H i , j < V i , j , (i,j) is a horizontal edge, so interpolate horizontally. ⎧ G i , j − 1 + G i , j +1 H i , j < V i , j  2 ⎨ G i − 1 , j + G i +1 , j G ′ i , j = H i , j > V i , j 2  G i , j − 1 + G i , j +1 + G i − 1 , j + G i +1 , j ⎩ H i , j = V i , j 4 Leszek ´ Swirski CFA Interpolation Detection

  23. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based Leszek ´ Swirski CFA Interpolation Detection

  24. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based Leszek ´ Swirski CFA Interpolation Detection

  25. Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis And more... ▶ Adaptive Colour Plane Has adaptive interpolation for G using first order derivative of luminance and second order derivative of chrominance, and uses adaptive interpolation again for R and B . ▶ Threshold-Based Variable Number of Gradients Eight gradient samples taken from a 5 × 5 neighbourhood of each pixel, averages are calculated for each gradient, gradients of values less than a (dynamic!) threshold are averaged, and averages are added to/subtracted from the CFA values. (Though both are similar in principle to the gradient-based) Leszek ´ Swirski CFA Interpolation Detection

  26. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis Why detect? ▶ Image/Camera verification ▶ Identification of forged regions ▶ Recognition of PRCG (PhotoRealistic Computer Generated images) Leszek ´ Swirski CFA Interpolation Detection

  27. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis General idea ▶ Interpolation creates correlation ▶ Most CFA interpolation is regular and approximately linear (especially for G ) ▶ If can determine some regular correlation, image is interpolated Leszek ´ Swirski CFA Interpolation Detection

  28. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis EM Algorithm ▶ ‘Expectation-Maximisation’ ▶ Simulataneously estimate parameters of correlation (i.e. what interpolation is used) and which points are correlated to their neighbours ▶ Two-step iterative algorithm ▶ Creates a separable parameter space Leszek ´ Swirski CFA Interpolation Detection

  29. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis A statistical Achilles’ Heel ▶ Interpolation creates correlation ▶ Correlation decreases variance ▶ Variance can be measured! ▶ Periodic low variance is indicative of interpolation Leszek ´ Swirski CFA Interpolation Detection

  30. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis A statistical Achilles’ Heel 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 Leszek ´ Swirski CFA Interpolation Detection

  31. Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis Gallagher and Chen ▶ First, high-pass filter: ⎡ ⎤ 0 1 0 h = 1 4 1 ⎣ ⎦ 0 1 0 ▶ Estimate variance using mean of absolutes along anti-diagonals ∑ ∣ ( h ∗ i ) x , y ∣ x + y = d m ( d ) = N d Leszek ´ Swirski CFA Interpolation Detection

Recommend


More recommend