Some Essentials of Data Analysis with Wavelets Slid Slides for the - PowerPoint PPT Presentation

Some Essentials of Data Analysis with Wavelets Slid Slides for the wavelet lectures of the course in data analysis at The f h l l f h i d l i Th Swedish National Graduate School of Space Technology Niklas Grip, Department of Mathematics, Luleå University of Technology Last update: 2009-11-12

1 1 a 0 (x) f(x) x 0 0

 (x)  (x) 1 1 0.8 0.8 0 6 0.6 0.6 0 6 0.4 0.4 0.2 0.2 0 0 -0.2 -0.2 -0.4 -0.4 -0.6 -0.6 -0.8 -0.8 -1 -1 -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 x x Old approximation Old approximation New approximation f(x) a 1 (x) 0 1/2 1 x 0 1 x

 (x)  (x) 1 1 0.8 0.8 0 6 0.6 0 6 0.6 0.4 0.4 0.2 0.2 0 0 -0.2 -0.2 -0.4 -0.4 -0.6 -0.6 -0.8 -0.8 -1 -1 -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 x x f(x) Old approximation a 2 (x) New approximation 0 1/4 2/4 3/4 1 0 1/2 1 x x

 (x)  (x) 1 1 0.8 0.8 0 6 0.6 0 6 0.6 0.4 0.4 0.2 0.2 0 0 -0.2 -0.2 -0.4 -0.4 -0.6 -0.6 -0.8 -0.8 -1 -1 -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 x x f(x) Old approximation a 3 (x) New approximation 0 1/4 2/4 3/4 1 0 1/8 2/8 3/8 4/8 5/8 6/8 7/8 1 x x

 (x)  (x) 1 1 0.8 0.8 0 6 0.6 0 6 0.6 0.4 0.4 0.2 0.2 0 0 -0.2 -0.2 -0.4 -0.4 -0.6 -0.6 -0.8 -0.8 -1 -1 -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 x x f(x) Old approximation a 4 (x) New approximation 0 2/16 4/16 6/16 8/16 10/16 12/16 14/16 1 0 1/8 2/8 3/8 4/8 5/8 6/8 7/8 1 x x x x

 (x)  (x) 1 1 0.8 0.8 0 6 0.6 0 6 0.6 0.4 0.4 0.2 0.2 0 0 -0.2 -0.2 -0.4 -0.4 -0.6 -0.6 -0.8 -0.8 -1 -1 -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 x x f(x) Old approximation a 5 (x) New approximation 0 4/32 8/32 12/32 16/32 20/32 24/32 28/32 1 0 2/16 4/16 6/16 8/16 10/16 12/16 14/16 1 x x

f(x) a 6 (x) 0 8/64 16/64 24/64 32/64 40/64 48/64 56/64 1 x

Wavelet bases { { } } n /2 n j j - y y = y y - The Haar basis is of the type The Haar basis is of the type ( ( x x k k ), ), ( ) ( ) x x 2 2 (2 (2 x x k k ) . ) . n k , k Such a basis is called a wavelet basis with scaling function j j y y and mother wavelet . Consequence : The scaling function gives a large scale approximation and the wavelets adds finer details (illustrated in next slide).

Orthonormal bases Both the Haar basis and the usual Fourier basis is a set of building blocks { } with the following properties e k å 2  · Î = Any function f L ( ) can be decomposed into a sum f c e . k k k · There is a simple formula for computing the coefficients: p p g ¥ Inner product ò = = c f x e x dx ( ) ( ) f e , k k k -¥ ì ï = 1 if k n ï · = í The building blocks are orthonormal : e e , k n ï ¹ 0 if k n ï î î Any such set of building blocks is called an ort honormal basis .

Good properites of the Haar wavelet basis : Good properites of the Haar wavelet basis : • Orthonormal (just like the Fourier basis). ·  Well localized Better suited for good approximation of small local g pp details in a signal with a small number of terms (contrary to the Fourier basis). Usually less desirable properties of the Haar wavelet basis : · ·   Discontinuities Discontinuities 1) Many terms needed for goo 1) Many terms needed for goo d approximation d approximation (=small "edges" in last slide ) of continuous signals. 2) Bad frequency localization (drawback in ) q y ( in time-frequency analysis (explained soon)).

MRA adds smoothness Natural question : N t l ti Are there any way to contruct a A h continuous ti , or even " arbitrarily smooth" (say, k times differentiable), well localized and orthonormal wavelet basis? Answer : Yes. T he construction is a generalization of the telescope sums in last lecture. Described in any wavelet book under the name multiresolution analysis lti l ti l i (MRA) . (MRA)

Extra bonus : It follows from the MRA theory that there is a special The fast wavelet transform algorithm for quick computation of the wavelet coefficients. The computation time is proportional to the signal length ( N ) and thus faster than the fast Fourier transform ( N log N ).

Pyramid algorithm / filter banks / Mallat’s algorithm

Example 1: Daubechies n scaling functions, n =1-12 p g , n=1 n=2 n=3 1.5 1.5 1.5 1 1 1 •Nonzero only in 0.5 0.5 0.5 0 0 0 0 0 0 the interval Daubechies scaling functions -0.5 -0.5 -0.5 0 5 10 0 5 10 0 5 10 n=4 n=5 n=6 [0, n-1 ]. 1.5 1.5 1.5 1 1 1 •For any k and 0.5 0.5 0.5 large enough n , 0 0 0 -0.5 -0.5 -0.5 0 5 10 0 5 10 0 5 10 the Daubechies n n=7 n=8 n=9 1.5 1.5 1.5 wavelet and 1 1 1 0.5 0.5 0.5 scaling function li f i 0 0 0 -0.5 -0.5 -0.5 is k times 0 5 10 0 5 10 0 5 10 n=10 n=11 n=12 1.5 1.5 1.5 differentiable. 1 1 1 0.5 0.5 0.5 0 0 0 -0.5 -0.5 -0.5 0 5 10 0 5 10 0 5 10

Corresponding Daubechies wavelets. Corresponding Daubechies wavelets. n=1 n=2 n=3 1 1 1 0 0 0 Daubechies wavelets -1 -1 -1 0 10 20 0 10 20 0 10 20 n=4 n=5 n=6 1 1 1 0 0 0 0 0 0 -1 -1 -1 0 10 20 0 10 20 0 10 20 n=7 n=8 n=9 1 1 1 0 0 0 0 0 0 -1 -1 -1 0 10 20 0 10 20 0 10 20 n=10 n=11 n=12 1 1 1 0 0 0 0 0 0 -1 -1 -1 0 10 20 0 10 20 0 10 20

Example 2: Spline wavelets of degree 2 p p g Translated scaling functions 2 •Exponential decay 0 ( l (slower than th −2 Spline wavelets −3 −2 −1 0 1 2 3 Daubechies, but Translated wavelets 2 still fast). 0 •Spline wavelets of S li l t f −2 degree n is n times −3 −2 −1 0 1 2 3 Translated and dilated (with factor 2) wavelets differentiable. 2 0 • n th degree • n th degree −2 polynomial in −3 −2 −1 0 1 2 3 Translated and dilated (with factor 4) wavelets intervals [k,k+1] 2 0 0 (scaling function) (scaling function) −2 and [k/2,(k+1)/2] −3 −2 −1 0 1 2 3 (wavelet).

Some threshold techniques q

Caruso wax roll example Source: http://www.fmah.com 1) Original, 2) Single pass denosied, 3) Removed noise, 4), second pass denoised seeking decorrelation between the noise model and the original file

L H L L L L H H H H

256 256 i 256x256 pixels, 256 greyscale l 256 l Whi White noise added i dd d Restored, daub4, reduced to 1,8 % of the original file size

FBI fingerprint example

Original image x 24 bit colours 847x683 pixels Image size: =1 66 MB 1.66 MB

JPEG-compressed image Compression: •65.8 times •JPEG

JPEG2000-compressed image Compression: •JPEG2000 •130 times

Original: Denoised: Movie example Removed noise: Source: http://www.fmah.com

Digital subscriber lines Digital subscriber lines

Multicarrier transmission examples: ADSL: Out now. About 2-8.5 megabits per second (Mbps) VDSL: (Originally) planned for 2001. VDSL: (Originally) planned for 2001. ADSL vs. VDSL From 5 Mbps in 1500 m long wires up to about 60 Mbps in 400 m long wires. p g (50 Mbps is enough for, for example, 8 digital TV channels or 2-4 high definition TV channels ) TV channels or 2 4 high definition TV channels.)

Maximum delay restrictions Maximum delay restrictions

Each symbol is built up of N basis functions The transmitted information N å å s c f f t (t)= ( ) ( ) ( ) k k k l k l k l k l , , = Choice of basis functions l 1 f must be well localized in time (because too k l k l , , long symbols introduce unacceptable delays). Wavelets can be used, but for this particular application, the short time Fourier transform pp , has some advantages and is used in VDSL.

Railway bridge strains Railway bridge strains

Channel A1 (  m/m), 7 level decomposition with haar wavelet. Channel A2 (  m/m), 7 level decomposition with haar wavelet. Approximation Approximation 20 10 Detaljer j Detaljer j 10 5 0 0 -10 -5 -20 -30 -10 0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45 Tid (timmar) Tid (timmar) Channel A4 (  m/m) Channel A4 (  m/m), 7 level decomposition with haar wavelet. Channel A5 (  m/m), 7 level decomposition with haar wavelet. Channel A5 (  m/m) 7 level decomposition with haar wavelet 7 level decomposition with haar wavelet 5 Approximation 20 Detaljer 10 0 0 -5 -10 -20 -10 Approximation Detaljer -30 -15 15 0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45 Tid (timmar) Tid (timmar) Channel A6 (  m/m), 7 level decomposition with haar wavelet. Channel A7 (  m/m), 7 level decomposition with haar wavelet. 30 30 Approximation Approximation Detaljer Detaljer 20 20 10 10 0 0 0 -10 -10 -20 0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45 Tid (timmar) Tid (timmar) Channel A8 (  m/m), 7 level decomposition with haar wavelet. Channel R1 (  m/m), 7 level decomposition with haar wavelet. 15 Approximation Approximation 5 Detaljer Detaljer 10 0 5 0 -5 -5 0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45 Tid (timmar) Tid (timmar)