A class of Laplacian multi-wavelets bases for high-dimensional data - PowerPoint PPT Presentation

MULTIVARIATE APPROXIMATION AND INTERPOLATION WITH APPLICATIONS 2013 A class of Laplacian multi-wavelets bases for high-dimensional data Nir Sharon Tel-Aviv University Joint work with Yoel Shkolnisky A part of PhD thesis under the supervision of Yoel Shkolnisky and Nira Dyn September 26, 2013 Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 1 / 24

Representing signals 4 3 2 1 0 −1 −2 −3 −4 0 1 2 3 4 5 6 7 1D signals – Fourier basis, wavelets, polynomials,. . . Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 2 / 24

Representing signals 4 3 2 1 0 −1 −2 −3 −4 0 1 2 3 4 5 6 7 1D signals – Fourier basis, wavelets, polynomials,. . . What to do in higher dimensions? Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 2 / 24

Representing signals 4 3 2 1 0 −1 −2 −3 −4 0 1 2 3 4 5 6 7 1D signals – Fourier basis, wavelets, polynomials,. . . What to do in higher dimensions? What to do for general data - images, documents, gene arrays, . . . ? Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 2 / 24

What is general data? Let X = { x i } N i =1 , x i ∈ R D , be a set of N points with two requirements: B A E D F C Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 3 / 24

What is general data? Let X = { x i } N i =1 , x i ∈ R D , be a set of N points with two requirements: 1 The set X is associated with a kernel function K : R D × R D → R + , and with the graph structure induced by K . B A E D F C Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 3 / 24

What is general data? Let X = { x i } N i =1 , x i ∈ R D , be a set of N points with two requirements: 1 The set X is associated with a kernel function K : R D × R D → R + , and with the graph structure induced by K . 2 X has an associated tree structure – analog of a dydic partition. B A E D F C Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 3 / 24

The goal Find N functions { φ n } N n =1 , φ n : X �→ R , such that � φ n , φ m � = δ n , m . We use � � f , g � = f ( x ) g ( x ) , ∀ f , g : X �→ R . x ∈X Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 4 / 24

The goal Find N functions { φ n } N n =1 , φ n : X �→ R , such that � φ n , φ m � = δ n , m . We use � � f , g � = f ( x ) g ( x ) , ∀ f , g : X �→ R . x ∈X Further requirements ◮ The construction must be applicable in cases where D (the dimension of each point in X ) is very large. ◮ It should allow for a sparse representation of a large family of functions. ◮ It must have a fast and numerically stable algorithm. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 4 / 24

Known solutions Two known solutions for general data ◮ Haar basis – Piecewise constant functions ◮ Fourier basis – Eigenvectors of the (graph) Laplacian Haar basis Fourier basis The eigenvectors of the graph Laplacian 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 5 / 24

Haar basis – general data Haar-like on graphs (Gavish, Nadler, and Coifman) Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 6 / 24

Haar basis – general data Haar-like on graphs (Gavish, Nadler, and Coifman) ◮ Simple, fast. Pros ◮ Applicable to high dimensional data. ◮ Poor representations of smooth functions. Cons Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 6 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . How to generalize? – Eigenvectors of the “graph Laplacian”. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . How to generalize? – Eigenvectors of the “graph Laplacian”. The graph Laplacian, in a nutshell: Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . How to generalize? – Eigenvectors of the “graph Laplacian”. The graph Laplacian, in a nutshell: For any set of points (in R D , on a manifold,. . . ), use kernel K to 1 construct a graph � x i − x j � 2 / 2 ε � � W i , j = K . Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . How to generalize? – Eigenvectors of the “graph Laplacian”. The graph Laplacian, in a nutshell: For any set of points (in R D , on a manifold,. . . ), use kernel K to 1 construct a graph � x i − x j � 2 / 2 ε � � W i , j = K . Normalize, e.g., 2 N L = I − B − 1 W , � B i , i = W i , j . j =1 Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Fourier basis Eigenfunctions of the Laplacian, e.g., ϕ ′′ = − λϕ . How to generalize? – Eigenvectors of the “graph Laplacian”. The graph Laplacian, in a nutshell: For any set of points (in R D , on a manifold,. . . ), use kernel K to 1 construct a graph � x i − x j � 2 / 2 ε � � W i , j = K . Normalize, e.g., 2 N L = I − B − 1 W , � B i , i = W i , j . j =1 Compute the eigenvectors. 3 Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 7 / 24

Graph Laplacian basis Graph Laplacian’s eigenvectors on meshes (Gabriel Peyr´ e) ◮ Efficient representation for smooth functions. Pros ◮ Applicable to high dimensional (almost arbitrary) data. ◮ Poor representation of non-smooth functions/rapidly changing Cons functions. ◮ Global basis functions. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 8 / 24

Let’s construct a new family of bases Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Let’s construct a new family of bases Orthogonal Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Let’s construct a new family of bases Orthogonal Multi-scale – basis elements of varying support. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Let’s construct a new family of bases Orthogonal Multi-scale – basis elements of varying support. A family of bases parameterized by k – controls the localization of the basis elements. Extreme cases ◮ k = 1 = ⇒ Haar basis ◮ k = N = ⇒ Fourier basis Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Let’s construct a new family of bases Orthogonal Multi-scale – basis elements of varying support. A family of bases parameterized by k – controls the localization of the basis elements. Extreme cases ◮ k = 1 = ⇒ Haar basis ◮ k = N = ⇒ Fourier basis Stable O ( k 2 N log N + T ( N , k ) log( N )) algorithm, where T ( N , k ) is the complexity of computing k top eigenvectors. Usually N ≫ k . Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Let’s construct a new family of bases Orthogonal Multi-scale – basis elements of varying support. A family of bases parameterized by k – controls the localization of the basis elements. Extreme cases ◮ k = 1 = ⇒ Haar basis ◮ k = N = ⇒ Fourier basis Stable O ( k 2 N log N + T ( N , k ) log( N )) algorithm, where T ( N , k ) is the complexity of computing k top eigenvectors. Usually N ≫ k . Building blocks: graph Laplacian and multi-resolution analysis. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 9 / 24

Construction overview Two phases: Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 10 / 24

Construction overview Two phases: 1 Define the vectors which span the approximation spaces V 0 ⊂ V 1 ⊂ · · · ⊂ V j , where V j = R N , with N the number of data points. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 10 / 24

Construction overview Two phases: 1 Define the vectors which span the approximation spaces V 0 ⊂ V 1 ⊂ · · · ⊂ V j , where V j = R N , with N the number of data points. 2 Apply a fast orthogonalization process to obtain V j = V 0 ⊕ W 0 ⊕ W 1 ⊕ · · · ⊕ W j − 1 , with W p ⊥ V p , W p ⊕ V p = V p +1 for p ≥ 0. Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 10 / 24

Phase one – V 0 ⊂ V 1 ⊂ · · · ⊂ V j , Nir Sharon (Tel-Aviv University) Laplacian multi-wavelets, MAIA 13 September 26, 2013 11 / 24