Hybrid sparse stochastic processes and the resolution of linear - PDF document

Hybrid sparse stochastic processes and 
 the resolution of linear inverse problems Michael Unser Biomedical Imaging Group EPFL, Lausanne, Switzerland Statistical Models for Shape and Imaging , March 11-15, 2019, Institut Poincaré, Paris Variational-MAP formulation of inverse problem Linear forward model noise y = Hs + n linear 
 model H n s Reconstruction as an optimization problem s rec = arg min k y � Hs k 2 + λ k Ls k p p = 1 , 2 , 2 p | {z } | {z } data consistency regularization − log Prob( s ) : prior likelihood � 2

An introduction to sparse stochastic processes EDEE Course � 3 Random spline: archetype of sparse signal Random weights { a n } i.i.d. and random knots { t n } (Poisson with rate λ ) Stochastic differential equation D s ( t ) = w ( t ) with boundary condition s (0) = 0 X w ( t ) = a n δ ( t − t n ) Innovation : n Formal solution = Compound Poisson process X s ( t ) = D − 1 w ( t ) = a n D − 1 { δ ( · − t n ) } ( t ) n X = b 1 + a n + ( t − t n ) n � 4

JqHiclVZdb9s2F W7r9j7aLo97oVY qALGkN2t2XrUKBAi 4F+pAlS9Ml8gJKupI5U5RKUrE9Qr9sv2SPe93+xC4l27WkZHAFOCF4D8+5vPeSl37GmdKu+9edu+ 9/8GH 21 uh9/8uln97bvf/5KpbkM4CxIeSpf+1QBZwLON McXmcSaOJzOPcnh9Z+fg1SsVT8oucZjBIaCxaxgGqcuto+ wkESBYQDUqTKBeBnSe73jWV2Zh5TBAvoXocUG5OiwfeyW/h17sk43SuiB4DkSkHk aEiRBmB cxitJX2ztu3y0/0h4MFoMdZ/EdX93f+tEL0yBPQOiAU6UuB26mR4ZKzQIORdfLFWQ0mNAYTLnrgvRwKiR KvEnNClnaziaKDVPfETaHaimzU7eZLvMdfT9yDCR5RpEUAlFOSc6JTaEJGQSAs3nhAYB+ptTjX7UGBSGC8In/R+GIzMGfg0adyAx0tMgTRIqQuNFNGF8HkJEc64L46loOe72uj1SZk4RhBI/5xy0eoz HmIw0eVWTZLHKQ8LI2O/MIOHbv/g24du0cJImC8wbh8B1Q9h6874PIdKBj2p+8mpj8WlIWFoqkgtuiCezDkYqwqzYvm/aB b9c2JF95uSI2bfydqG6zNqH2OeXyXeCD8f6jrBHhqVO4rKA9aY jx gr BvbMvtv/DpfUo4xlyjmex1JEvcmphN23K4ZWgtymcavMwVuZVcw1E/NbdR4td ZVKv7icjBajq3WnrmpTgY2JvbPgtMWY8lpK5/sDG5mbwhsqrEmUiVjqdHM 4t Jq0AMTa1Em+GTZzfX3lf0ZI6b4bX4BrvZp7aVC74SHkFnGYQM qJvcNSXp3/dRUmTgqzuqB935w098fER 1x0UQcYmUsjYdN48mascV9sWZs0R4x7oPUS4htHkdNDLzBi8qmRmPBSeDGthyN/UPEeOGbJ4ivL3iZ gVGUybkZ Pr2cr0rGl6sTK9aJpCJvTCFrbUnmOHZSDXt/C8lWh/ZtdjdsIqSWTWKrFJAzFpIaYNhJcmEFPrUG+/ MgpaDydcdlwM+wvRLE/gIAO+vh1PQXY9kWsx1XlYSe2+3KDpHUvLF8KSktcPcYa7A8b7mjqF2uXBRY68mBPWlfRMN TFmpL8A2aOz3CIjJPcyIAWyH2SRr+nuN7wjpcAu0TY qepdM6E/qrJiyz1+WwRjSl2NKRCFMk0zAPcM/okA2CD3oKIDAQksaSZti8 bkxaD4u2oNXw/7gUd/9ebjz9PHi4bHlfOl85TxwBs6B89Q5co6dMydw/nT+dv5x/u3sdY475 1fK+jdO4s1Xzi1r+P/ByVFhJY=</latexit> <latexit sha1_base64="sUc5qrFhIvsr51Rwv234bJX9R98=">A <latexit sha1_base64="KWQGjUuYKXxaMcV8rvHV2 mFaC8=">A JaXiclVZtb9s2EFb vcTeW7IBw7B9IeYa 4vGkN1u2ToUKJCiSIF+6JKlLRJ5ASWdZdYUpZJUHI/Tn9nX7Q/tN+xP7CjZrl6SwRXghOA9fJ7j3elOfsqZ0q7 z7XrN957/4MPtzrdjz7+5NP tnc+f6GSTAZwHCQ8ka98qoAzAcea Q6vUgk09jm89Gf71v7yHKRi fhVL1IYxzQSbMICqnHrbPtLjwnixVRPA8rNUf7dLe/wt/D2 XbPHbjFQ9qL4XLRc5bP87OdrZ+9MAmyGIQO FXqdOimemyo1CzgkHe9TEFKgxmNwBRu56SPWyGZJBJ/QpNit4ajsVKL2EekdVA1bXbzMt p ic/jg0Ta ZB KXQJONEJ8TGgIRMQqD5gtAgQH8zqtGPGoPCaED4cPDTaGymwM9B4w0kCJgHSRxTERpvQmPGFyFMaMZ1bjw1Wa27/W6fFKFXBKHEz gHrR7gvocYzFRxVRNnUcLD3MjIz83wrjvY+/6um7cwEhZLjDtAQPlDWNUZn2dQyqAndT859bE6NMQMTSWpRefEkxkHY1XhIl/9zxvEVn1z4qW3G1Lj5d+J2gZrM2qfYx7fJR4I/x/qOoGegsp8hRWEb1BuiPGmCusG7phd /ADHqlHGcuUc3Kz9Em9yaiEm29PjKwEuUrjSpm9tzLrmGsmFlfq3FvpVFVK/vx0OF6trdYdc1mdDG1M7J8lpy3GgtNWPukNL2dvCGyqUREpk7HSaOaZRTaTVoAYm1qJnWET53fX3pe0pM6bUlnl3cxTm8olHylawFEKAaOc2B6W8PL9r6owcZibdf/1fXPYvB8TJ3XESROxj5WxMu43jYcVY4v7pGJs0R4w7oPUK4idDQdNDLzBRmVTo7HgJHDjnWO3l4yKCBu+eYj4+oFnSZpjNGVMnjW5Hq9Nj5ump2vT06YpZEIvbWFL7QmOSAayeoUnrUT7F/Y8Zicsk0QuWiU2ayBmLcS8gfCSGCJqHervFg85Ao1vZ0TwtSYpzhei2O9AQAcDfLqeApzbItLTsvJECPZebhC3+sJq1Cst8fQUa3AwarijqZ9XmgUWOvLgTKq aLjQcxZqS3AfzZ0+YROySDIiAEchzk avs6ULhwugCSZ4EKEybzOhP6qGUt uxzViOYURzoSY pkEmYB3hkdskHwQc8B AZC0kjSFIc3fm4Mmx8X7cWL0WB4b+D+cr/36MHyw2PL+cb51rnlDJ095 Fz4Dx3jp3A+cP50/nL+Xvr385O56vO1yX0+rXlmS+c2tPp/QfuE2vI</latexit> Lévy processes: all admissible brands of innovations Generalized innovations : white Lévy noise with E { w ( t ) w ( t 0 ) } = σ 2 w δ ( t − t 0 ) ∈ S 0 ( R d ) D s = w (perfect decoupling!) White noise (innovation) Lévy process Brownian motion (Wiener 1923) 0 0 Gaussian 0.0 0.2 0.4 0.6 0.8 1.0 Integrator Z t w ( t ) s ( t ) Impulsive Compound Poisson d τ 0 0 0 0.0 0.2 0.4 0.6 0.8 1.0 S α S (Cauchy) Lévy flight 0 0 0.0 0.2 0.4 0.6 0.8 1.0 (Paul Lévy circa 1930) � 5 Generalized innovation model Theoretical framework: Gelfand’s theory of generalized stochastic processes Generic test function ϕ ∈ S ( R d ) plays the role of index variable L s = w Solution of SDE (general operator) 2 innovation process sparse stochastic process L − 1 s = L − 1 w White noise 
 1 3 w Whitening operator X = h ϕ , w i Y = h ϕ , s i = h ϕ , L − 1 w i = h L − 1 ∗ ϕ , w i L 4 Approximate decoupling Proper definition of 
 Regularization operator vs. wavelet analysis continuous-domain white noise Main feature: inherent sparsity (Unser et al , IEEE-IT 2014) (few significant coefficients) � 6

Ke3iclVbdbts2FJa7v8T7a7bL3RCLgyZBYsju mwdAhRIUSRALrp0abuErkFJRzYX6qckFdsj9Ap7mt1u7 GHGbBDyU4tKRlcAbZpno/fd8hzdA69VHClXfef1r0P vzo40/W1tuf vb5F1/e3/jqpUoy6cO5n4hEv aYAsFjONdcC3idSmCRJ+CVd3Vk7a+uQSqexL/oWQqDiI1iHnKfaZwabrQenISkY2jBZDyRQU7TMc8P6WmSvjH7PcqUxinFCeUxOR2m2/TsTbDTIXoMcbvz6yEVLB4JIBazRxShsvh/M9/k3iOTBapDuEKEFxIKEeFxyGOuQcxIwK+54riLnFIy4XpMtqMkQMch2CkXnNIHcD0jME2TG Kdk04 vEVsmyYRjNjOIbqvh8Y6n5OQenw0t9y6xpvuWMgODXAVwb+d4f1Nt+sWD2kOevPBpjN/ng831n6iQeJnETrnC6bUZc9N9cAwqbmP+2rT EHK/Cs2AlN4kJMtnApImEj8oG4xW8GxSKlZ5CEyYnqs6jY7eZvtMtPhDwPD4zT EPulUJgJohNiswKPW4Kv8dyZ76O/GdPoR4VB+UxAcNj9sT8wYxDXoHEHEmKY+EkUsTgwNGQRF7MAQpYJnRuqwsW4vdXeIkUyKoJQ4mVCgFaPcZ4iBnO3DEGUjRIR5EaOvNz09tzuwaM9N29gJMzmGLeLgPKDsGVnbDRLGfSk6qdgHr4vGiKeV2KPSZkJMFYVpvniN68RW/XVieferkiNm38vantYq1F7AuP4PueB8P+hrhJgKVCZpzCDsKbkBt+oscK8gV2z73a/xyXVU8Y0FYJ0Sp/U24xJ6Lxb0bcS5C6NO2UO3sncnLnm8exOnYcLnW Vkj+/7A0WY6u1a27Lk549E/s157TJWHDazCebvdvZawKraiyJlMFYaNTjzEc2klaAGBtaiZVhFef3b7wvaUmVN2VymXc1T20o53ykKAEvUvA5E8TWsESU7/+yCo/PcvtNqC1hnmfO6v j8U VcVFH GFmLIxHdePZkrHBfbFkbNAec+GB1AsIVkNzXMfAWyxUNjQaE06CMPQaq73kRavLzSHiqwuwx+Z4mjIip3Wupzemp3XTyY3p G6y7WpuCxpqz/DSwE ub+FZI9De1K7H6ARlkMi0kWJXNcRVAzGpIcpOax3a2i8e8gI0vp0je4sgKfYXovjvQED7X zaVAHeZOKRHpeZFwe2wxvXjxp1YXH5UVri6jHmYLdfc0czL18qFpjoyIM9aVlFw1RPeKAtwXdoXt8iPCSzJCMxYCvEPsmC3zKlC4cLIElCHMRBMqkyob/qiqe2XPYrRBOGLR2JMEQyCTIf94wO2UPwQE8AYjwIyUaSpdi8 brRq18umoOX/W7vYdf9ub/5 PH84rHmfON862w7PefAe IcO8+dc8dv/dH6s/VX6+/1f9ub7d32Xgm915qv+dqpPO1H/wGgiNC4</latexit> <latexit sha1_base64="sUc5qrFhIvsr51Rwv234bJX9R98=">A <latexit sha1_base64="mFLHMsp3AJPrKENid/anCo4EUqg=">A JaXiclVZtb9s2EFb vcTeW7IBw7B9IeYa 4vGkN1u2ToUKJCiSIF+6JKlLRJ5ASWdZdYUpZJUHI/Tn9nX7Q/tN+xP7CjZrl6SwRXghOA9fJ7j3elOfsqZ0q7 z7XrN957/4MPtzrdjz7+5NP tnc+f6GSTAZwHCQ8ka98qoAzAcea Q6vUgk09jm89Gf71v7yHKRi fhVL1IYxzQSbMICqnHrbPtLjwnixVRPA8rNUf7dLe/wt/D2 XbPHbjFQ9qL4XLRc5bP87OdrZ+9MAmyGIQO FXqdOimemyo1CzgkHe9TEFKgxmNwBRu56SPWyGZJBJ/QpNit4ajsVKL2EekdVA1bXbzMt p ic/jg0Ta ZB KXQJONEJ8TGgIRMQqD5gtAgQH8zqtGPGoPCaED4cPDTaGymwM9B4w0kCJgHSRxTERpvQmPGFyFMaMZ1bjw1Wa27/W6fFKFXBKHEz gHrR7gvocYzFRxVRNnUcLD3MjIz83wrjvY+/6um7cwEhZLjDtAQPlDWNUZn2dQyqAndT859bE6NMQMTSWpRefEkxkHY1XhIl/9zxvEVn1z4qW3G1Lj5d+J2gZrM2qfYx7fJR4I/x/qOoGegsp8hRWEb1BuiPGmCusG7phd /ADHqlHGcuUc3Kz9Em9yaiEm29PjKwEuUrjSpm9tzLrmGsmFlfq3FvpVFVK/vx0OF6trdYdc1mdDG1M7J8lpy3GgtNWPukNL2dvCGyqUREpk7HSaOaZRTaTVoAYm1qJnWET53fX3pe0pM6bUlnl3cxTm8olHylawFEKAaOc2B6W8PL9r6owcZibdf/1fXPYvB8TJ3XESROxj5WxMu43jYcVY4v7pGJs0R4w7oPUK4idDQdNDLzBRmVTo7HgJHDjnWO3l4yKCBu+eYj4+oFnSZpjNGVMnjW5Hq9Nj5ump2vT06YpZEIvbWFL7QmOSAayeoUnrUT7F/Y8Zicsk0QuWiU2ayBmLcS8gfCSGCJqHervFg85Ao1vZ0TwtSYpzhei2O9AQAcDfLqeApzbItLTsvJECPZebhC3+sJq1Cst8fQUa3AwarijqZ9XmgUWOvLgTKq aLjQcxZqS3AfzZ0+YROySDIiAEchzk avs6ULhwugCSZ4EKEybzOhP6qGUt uxzViOYURzoSY pkEmYB3hkdskHwQc8B AZC0kjSFIc3fm4Mmx8X7cWL0WB4b+D+cr/36MHyw2PL+cb51rnlDJ095 Fz4Dx3jp3A+cP50/nL+Xvr385O56vO1yX0+rXlmS+c2tPp/QfuE2vI</latexit> Description of sparse stochastic process Specification of spatial dependencies ∈ S 0 ( R d ) s = L − 1 w Whitening operator L ⇒ Specification of innovation (sparsity behavior) Canonical observation through a rectangular window 1 X id = h w, rect i h i , = ) X id = h w, rect i is infinitely divisible w = white noise with canonical Lévy exponent f ( ω ) = log E { e j ω X id } . Definition : A random variable X with generic pdf p id ( x ) is infinitely divisible (id) iff., for d any N ∈ Z + , there exist i.i.d. random variables X 1 , . . . , X N such that X = X 1 + · · · + X N . 1 i.i.d. X = h w, rect i h i , = h i + · · · + h i , , = 1 1 � 7 n n ⇒ Probability laws of sparse processes are id Analysis: go back to innovation process : w = L s Generic random observation: X = h ϕ , w i with ϕ 2 S ( R d ) or ϕ 2 L p ( R d ) (by extension) z }| { Linear functional: Y = h ψ , s i = h ψ , L − 1 w i = h L − 1 ∗ ψ , w i If φ = L − 1 ∗ ψ 2 L p ( R d ) then Y = h ψ , s i = h φ , w i is infinitely divisible R � � evy exponent f φ ( ω ) = ωφ ( x ) d x with (modified) L´ R d f e f φ ( ω ) − j ω y d ω Z p Y ( y ) = F − 1 { e f φ ( ω ) } ( y ) = ⇒ 2 π R = explicit form of pdf An Introduction to Unser and Tafti Sparse Stochastic Processes � 8