A Model for I mage Splicing Tian-Tsong Ng, Shih-Fu Chang Department - PowerPoint PPT Presentation

A Model for I mage Splicing Tian-Tsong Ng, Shih-Fu Chang Department of Electrical Engineering Columbia University, New York, USA

Outline Review � Problem and Motivation � Our Approach � Definition: Bicoherence � Why Bicoherence good for splicing detection? Previous Hypothesis � Bicoherence Features � Magnitude feature � Phase feature � Proposed Image Splicing Model � Bipolar Perturbation Hypothesis � Bicoherence of bipolar signal � Bipolar perturbation effect on magnitude feature � Bipolar perturbation effect on phase feature �

Problem & Motivation: How much can we trust digital images? General problem: Image Forgery � Detection Image Forgery: Images with manipulated � or fake content (In)Famous examples: � March 2003: A Iraq war news photograph � on LA Times front page was found to be a photomontage Feb 2004: A photomontage showing John � Kerry and Jane Fonda together was circulated on the Internet Adobe Photoshop: 5 million registered � users

Definitions: Photomontage and Spliced Image Specific problem: Image Splicing Detection � Photomontage: A paste-up produced by sticking together � photographic images, possibly followed by post-processing (e.g. edge softening and adding noise). Spliced Image (see figures): Splicing of image fragments � without post-processing. A simplest form of photomontage. spliced Why interested in detecting image splicing? � Image splicing is a basic and essential operation in the � creation of photomontage Therefore, a comprehensive solution for photomontage � detection includes detection of post-processing operations and intelligent techniques for detecting internal scene inconsistencies spliced

Image Forgery Detection Approaches Active approach: Passive and blind approach : Fragile/Semi Fragile Digital Without any prior information (e.g. � � Watermarking: Inserting digital digital watermark or authentication watermark at the source side signature), verifying whether an and verifying the mark integrity image is authentic or fake. at the detection side. Advantages: No need for watermark � Authentication Signature: embedding or signature generation � Extracting image features for at the source side generating authentication signature at the source side and verifying the image integrity by signature comparison at the receiver side. Effective when there is � � A secure trustworthy camera � A secure digital watermarking algorithm � A widely accepted watermarking standard

What are the qualities of authentic images? � Image Authenticity � Natural-imaging Quality � Entailed by natural imaging process with real imaging devices, e.g. camera and scanner � Effects from optical low-pass, sensor noise, lens distortion, demosicking, nonlinear transformation. � Natural-scene Quality Computer Graphics � Entailed by physical light transport in 3D real- world scene with real-world objects � Results are real-looking texture, right shadow, right perspective and shading, etc. � Examples: � Computer graphics and photomontages lack in both qualities. photomontage

Definition: Bicoherence Bicoherence = normalized bispectrum (3 rd order moment � spectra) Definition (Bicoherence) The bicoherence of a signal x(t) � with its Fourier transform being X( ω ) is given by: Numerator = Bispectrum ω ω ω + ω * [ ( ) ( ) ( )] E X X X ω ω = = ω ω Φ ω ω ( ( , ) 1 2 1 2 ( , ) ( , ) j b X b b e 1 2 1 2 1 2 2 2 ω ω ω + ω [ ( ) ( ) ] [ ( ) ] E X X E X 1 2 1 2 X X Phase Magnitude Normalized by the Cauchy-Schwartz Inequality upper bound Notations: ⋅ = Cauchy-Schwartz Inequality magnitude 2 Κ = < ∞ Φ ⋅ = Hilbert space, { : is a random variable satisfying [ ] } ( ) x x E x phase 2 2 ∗ ≤ ∈ Κ [ ] [ ] [ ] ( , ) E xy E x E y x y

Why BIC is Good for Splicing Detection? Hypothesis I [Farid99] � Quadratic Phase Coupling (QPC) � A phenomena where quadratic related frequencies ω ω ω + ω , and has the same quadratic relationship 1 2 1 2 φ φ φ + φ , and 1 2 1 2 Phases are quadratic coupled (not independent)! If ( ω 1 , ω 2 , ω 1 + ω 2 ) are quadratic phase If ( ω 1 , ω 2 , ω 1 + ω 2 ) have statistically coupled, independent phase, Φ ω ω 1) [ ( , )] would be 0 ω ω b 1) ( , ) would be 0 due b 1 2 1 2 * Φ ω ω ω + ω [ ( ) ( ) ( )] to statistical averaging X X X 1 2 1 2 = Φ ω + Φ ω −Φ ω + ω [ ( )] [ ( )] [ ( )] Φ ω ω 2) [ ( , )] would be random X X X b 1 2 1 2 1 2 = φ + φ − φ + φ = ( ) 0 1 2 1 2 * ω ω ω + ω [ ( ) ( ) ( )] ω ω E X X X 2) ( , ) would be close to unity b ω ω = 1 2 1 2 ( , ) X b 1 2 1 2 2 2 ω ω ω + ω [ ( ) ( ) ] [ ( ) ] E X X E X (imagine X now becomes positive RV) 1 2 1 2 X X

Hypothesis I (cont.) Quadratic linear Operation = + 2 ( ) ( ) ( ) y t x t x t = ω + φ + ω + φ ( ) 1 cos(2 2 ) 1 cos(2 2 ) = ω + φ + ω + φ y t t t ( ) cos( ) cos( ) x t t t 1 1 2 2 2 2 1 1 2 2 + ω + ω + φ + φ cos(( ) ( )) t 1 2 1 2 + ω − ω + φ − φ + cos(( ) ( )) t 1 2 1 2 ω + φ + ω + φ + cos( ) cos( ) 1 t t 1 1 2 2 Argument [Farid99]: Quadratic-linear operation gives rise to QPC and a nonlinear function, in Taylor expansion, contains quadratic-linear term. As splicing is a nonlinear operation, hence bicoherence is good at detecting splicing. Problems: 1. No detailed analysis was given. 2. The quadratic-linear operation here is a point-wise operation, it is not clear how splicing can be related to a point-wise operation?

Outline Review � Problem and Motivation � Our Approach � Definition: Bicoherence � Why Bicoherence good for splicing? Quadratic Phase Coupling � Hypothesis Bicoherence Features � Magnitude feature � Phase feature � Proposed Image Splicing Model � Bipolar Perturbation Hypothesis � Bicoherence of bipolar signal � Bipolar perturbation effect on magnitude feature � Bipolar perturbation effect on phase feature �

Columbia I mage Splicing Detection Evaluation Dataset 933 authentic and 912 spliced image blocks (128x128 pixels) � Extracted from � Berkeley’s CalPhotos images (contributed by photographers) which � we assume to be authentic Splicing is done by cut-and-paste of arbitrary-shaped objects � and also vertical/horizontal strip. Authentic � Samples Textured Smooth Textured Textured Smooth Smooth Smooth Textured Spliced � Download URL: http://www.ee.columbia.edu/dvmm/newDownloads.htm

Definition: Phase Histogram Strong phase � Phase histogram (normalized) concentration at ± 90 ° 1 ∑ Ψ = Φ ω ω ∈ Ψ = − ( ) 1 { [ ( , )] }, ,..., p b i N N 1 2 i 2 i Ω M where = 1 { } 1 otherwise 0 true π π 2 2 m m Ω = ω ω ω = 1 ω = 2 = − { , | , ; , 0,..., 1} m m M 1 2 1 2 1 2 M M − π + π (2 1) (2 1) i i Ψ = φ ≤ φ ≤ { | } i + + (2 1) (2 1) N N � Symmetry Property: For real- valued signal, bicoherence phase histogram is symmetrical, i.e., Ψ = Ψ ( ) ( ) p p − i i

Bicoherence Features � Definition: Phase feature ∑ = Ψ Ψ Ψ ( )log ( ) where ( ) is phase histogram f p p p P i i i i � Definition: Magnitude feature 1 ∑ = ω ω ( , ) f b 1 2 M 2 ω ω ∈Ω ( , ) M 1 2

Additional Results on Bicoherence Features [ISCAS’04] 0.9 0.85 62% 72% 0.8 Plain BIC 0.75 Prediction Residue Plain BIC + Prediction Residue 0.7 Plain BIC + Edge 0.65 Prediction Residue + Edge 0.6 Plain BIC + Prediction Residue + Edge 0.55 0.5 Accuracy Mean Average Percision Average Recall

Outline Review � Problem and Motivation � Our Approach � Definition: Bicoherence � Why Bicoherence good for splicing? Quadratic Phase Coupling � Hypothesis Bicoherence Features � Magnitude feature � Phase feature � Proposed Image Splicing Model � Bipolar Perturbation Hypothesis � Bicoherence of bipolar signal � Bipolar perturbation effect on phase feature � Bipolar perturbation effect on magnitude feature �

Hypothesis II: Bipolar perturbation model � Original image signal is relatively smooth due to the low-pass anti-aliasing operation in camera or scanner. � Spliced image signal can have arbitrary discontinuity ∆ ( ) d x k 2 ( ) s x p x o = + ( ) ( ) s x d x difference ( ) − s x ( ) ( ) s x s x p k 1 Definition (Bipolar signal): = δ − + δ − −∆ ⇔ ω = − ω + − + ∆ ω ( ) ( ) ( ) ( ) exp( ) exp( ( ) ) d x k x x k x x D k jx k j x 1 2 1 2 o o o o Fourier transform δ ⋅ < ∆ > where ( ) is a delta function, 0, 0 k k 1 2

Bicoherence of Bipolar Signal � Results: Bicoherence phase of bipolar signal is concentrated at ± 90°: [ ] ∗ 3 ω ω ω + ω = ∆ ω + ∆ ω − ∆ ω + ω ( ) ( ) ( ) 2 sin( ) sin( ) sin( ( )) D D D k j 1 2 1 2 1 1 1 2 Resulting in ± 90 ° phase bias � When there is phase coherency, bicoherence magnitude is close to unity