Lecture 3: Signal processing Andrew Owens
•PS1 due next Tuesday •Updated holiday office hours: •Questions?
Today • A few more filters • Fourier analysis
<latexit sha1_base64="6JG0rPkXeHIWG47l8jIU5ZTrDE8=">ACOXicfVDLSgMxFM3UV62vqks3wSIQpmpgm6Eohu7q2Af0GnLnThmYyQ5IRy9DfcuNfuBPcuFDErT9g+kC0FQ8ETs49yb3eBFnStv2k5VaWFxaXkmvZtbWNza3sts7VRXGktAKCXko6x4oypmgFc0p/VIUg8Tmte/3JUr91SqVgobvQgos0AuoL5jIA2UjtbdgV4HFoFXMLn2PUlkMSNQGoGvFUoDb8v+K5VGOKjfy0DY2lnc3beHgPE2dKcmiKcjv76HZCEgdUaMJBqYZjR7qZjEYSTocZN1Y0AtKHLm0YKiCgqpmMNx/iA6N0sB9Kc4TGY/VnRwKBUoPAM84AdE/N1kbiX7VGrP2zZsJEFGsqyOQhP+ZYh3gUI+4wSYnmA0OASGb+ikPTDTahJ0xITizK8+TaiHvHOft65Nc8WIaRxrtoX10iBx0ioroCpVRBRF0j57RK3qzHqwX6936mFhT1rRnF/2C9fkF1ous/w=</latexit> Laplacian filter • Used to detect object boundaries and other salient image structures • The Laplacian operator is defined as the sum of the second order derivatives of a function: r 2 I = ∂ 2 I ∂ x 2 + ∂ 2 I ∂ y 2 • Sensitive to noise. Blur first!
<latexit sha1_base64="nHOK6xLOIulxV7TgkJhAuH5SE0=">ACJHicbVDLSgMxFM3UV62vUZdugkVoKS0zVAQoejGZQXbCu1QMmDc1khiQjlKEf48ZfcePCBy7c+C1mpoNo64WQk3PO5eYeN2RUKsv6NHJLyura/n1wsbm1vaOubvXlkEkMGnhgAXizkWSMpJS1HFyF0oCPJdRjru+CrRO/dESBrwWzUJieOjIacexUhpqm+el7wSr9plWIUalJM7YbJ3xdbMBfyx1FOpkl9s2jVrLTgIrAzUARZNfvmW28Q4MgnXGpOzaVqicGAlFMSPTQi+SJER4jIakqyFHPpFOnC45hUeaGUAvEPpwBVP2d0eMfCknvqudPlIjOa8l5H9aN1LemRNTHkaKcDwb5EUMqgAmicEBFQrNtEAYUH1XyEeIYGw0rkWdAj2/MqLoF2v2c16+ak2LjM4siDA3AISsAGp6ABrkETtAGD+AJvIBX49F4Nt6Nj5k1Z2Q9+BPGV/fHbqb4g=</latexit> Laplacian filter • The most popular approximation is the five-point formula which consists in convolving the image with the kernel • Based on the approximation of the 2nd derivative filter: [1, -2, 1] ( f ( n − 1) − f ( n )) − ( f ( n ) − f ( n + 1)) = f ( n − 1) − 2 f ( n ) + f ( n + 1)
Laplacian filter • The most popular approximation is the five-point formula which consists in convolving the image with the kernel =
Image patches as filters • Goal: find in an image • What’s a good similarity/distance measure between two patches? Source: D. Hoiem
Matching with filters Method 1: Filter the image with = ∑ h [ m , n ] g [ k , l ] f [ m k , n l ] + + k , l f = image g = filter What went wrong? Filtered Image Input Source: D. Hoiem
Matching with filters Method 2: Filter the image with zero-mean eye. = ∑ h [ m , n ] ( f [ k , l ] f ) ( g [ m k , n l ] ) − + + mean of f k , l True detections False detections Thresholded Image Filtered Image (scaled) Source: D. Hoiem Input
Matching with filters Method 3: Normalized cross-correlation. mean template mean image patch ( g [ k , l ] g )( f [ m k , n l ] f ) ∑ − + + − m , n k , l h [ m , n ] = 0 . 5 ⎛ ⎞ 2 2 ( g [ k , l ] g ) ( f [ m k , n l ] f ) ∑ ∑ ⎜ ⎟ − + + − ⎜ ⎟ m , n ⎝ ⎠ k , l k , l Source: D. Hoiem
Matching with filters Method 3: Normalized cross-correlation. True detections Thresholded Image Input Normalized X-Correlation Source: D. Hoiem
Other linear transformations • So far, we’ve focused on convolution • What about other linear transformations? • Change the basis e.g. Fourier transform U = F f
The Discrete Fourier transform Discrete Fourier Transform (DFT) transforms a signal f[n] into F[u] as: f [ n ] exp ( − 2 π j un N − 1 N ) ∑ F [ u ] = n =0 Discrete Fourier Transform (DFT) is a linear operator. Therefore, we can write: n u ? ? ? ? ? ? ? ? … ? exp ( − 2 π j un N ) F = f NxN array Source: Torralba, Freeman, Isola
Change of basis Example: g ( t ) = sin( 2 π f t ) + ( 1/3 )sin( 2 π ( 3f ) t ) = + Source: A. Efros
Change of basis Source: A. Efros
Change of basis = + = Source: A. Efros
Change of basis = + = Source: A. Efros
Change of basis = + = Source: A. Efros
Change of basis = + = Source: A. Efros
Change of basis = + = Source: A. Efros
<latexit sha1_base64="/zy0YhesJzEUJHslqENQdBq2EZ8=">ACF3icbVDLSsNAFJ3UV62vqks3g0Wom5JUQTdC1Y3LCvYBTS2T6aQdMpmEmRuhPyFG3/FjQtF3OrOv3H6WGjrgQuHc+7l3nu8WHANtv1t5ZaWV1bX8uFjc2t7Z3i7l5TR4mirEjEam2RzQTXLIGcBCsHStGQk+wlhdcj/3WA1OaR/IORjHrhmQguc8pASP1ipVLVydhLw3wBXaye5dLH0bY9RWhqZOlQYZdzW56sYcB3DcK5bsij0BXiTOjJTQDPVe8cvtRzQJmQqiNYdx46hmxIFnAqWFdxEs5jQgAxYx1BJQqa76eSvDB8ZpY/9SJmSgCfq74mUhFqPQs90hgSGet4bi/95nQT827KZwAk3S6yE8EhgiPQ8J9rhgFMTKEUMXNrZgOickETJQFE4Iz/IiaVYrzknFvj0t1a5mceTRATpEZeSgM1RDN6iOGoiR/SMXtGb9WS9WO/Wx7Q1Z81m9tEfWJ8/g7ie4A=</latexit> Frequency Spectra ∞ = 1 X k sin(2 π kt ) A k =1 Source: A. Efros
Visualizing the Fourier transform f [ n ] exp ( − 2 π j un For: N − 1 N ) 12 Imag ∑ u=1 F [ u ] = 11 13 N=16 10 14 n =0 15 9 n=0 exp ( α j) = cos( α ) + j sin( α ) 8 Real cos ( 2 π un N ) − j sin ( 2 π un 7 n=1 N ) n=2 6 n=3 5 n=4 Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 Imag F f = n=0…15 Real u=15 16x16 array Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 Imag F f = Real u=15 16x16 array Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 12 Imag 11 13 10 14 F f = 15 9 n=0 8 Real 7 n=1 u=15 n=2 6 16x16 array n=3 5 n=4 Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 12 Imag 11 13 10 14 F f = 15 9 n=0 8 Real 7 n=1 u=15 n=2 6 16x16 array n=3 5 n=4 Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 Imag F f = Real u=15 16x16 array Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 14 u=2 6 Imag 15 13 5 7 F f = n=0 4 8 12 Real u=15 n=1 3 16x16 array 9 11 n=2 10 Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 14 u=2 6 Imag 15 13 5 7 F f = n=0 4 8 12 Real u=15 n=1 3 16x16 array 9 11 n=2 10 Source: Torralba, Freeman, Isola
Visualizing the transform coefficients exp ( − 2 π j un N ) n=0 1 2 … n=15 For N=16 u=0 u=1 u=2 Imag F f = Real u=15 16x16 array Source: Torralba, Freeman, Isola
The inverse of the Discrete Fourier transform Discrete Fourier Transform (DFT): Its inverse: f [ n ] exp ( − 2 π j un F [ u ] exp ( 2 π j un N − 1 N − 1 N ) N ) f [ n ] = 1 ∑ ∑ F [ u ] = N n =0 u =0 n=0 1 2 … n=15 n=0 1 2 … n=15 u=0 u=0 u=1 u=1 u=2 u=2 F F = F f = u=15 u=15 NxN array NxN array Source: Torralba, Freeman, Isola
For images, the 2D DFT 1D Discrete Fourier Transform (DFT) transforms a signal f [ n ] into F [ u ] as: f [ n ] exp ( − 2 π j un N − 1 N ) ∑ F [ u ] = n =0 2D Discrete Fourier Transform (DFT) transforms an image f [ n,m ] into F [ u,v ] as: f [ n , m ] exp ( − 2 π j ( M ) ) N − 1 M − 1 un N + vm ∑ ∑ F [ u , v ] = n =0 m =0 Source: Torralba, Freeman, Isola
Recommend
More recommend
