Quantum Chemistry Regression x = { z k (charge) , r k (position) } k - PowerPoint PPT Presentation

Quantum Chemistry Energy Regression with Scattering Transforms Matthew Hirn, Stéphane Mallat École Normale Supérieure Nicolas Poilvert Penn-State 1

Quantum Chemistry Regression x = { z k (charge) , r k (position) } k ≤ d ∈ R 4 d without quantum chemistry: very fast. 2

Linear Regression 3

Energy Properties • Invariant to permutations of the index k . 4

Overview • Coulomb kernel representations • Density functional approach to representation • From Fourier to wavelet energy regressions • Wavelet scattering dictionaries: deep networks without learning • Numerical energy regression results • Relations with image classification and deep networks 5

Coulomb Kernel 6

Density Functional Theory • Computes the energy of a molecule x from its electronic probability density ρ x ( u ) for u ∈ R 3 Organic molecules with Hydrogne, Carbon H 4 C 6 OS H 9 C 7 NO Nitrogen, Oxygen Sulfur, Chlorine H 3 C 6 NO 2 H 9 C 8 N 7

Density Functional Theory Kohn-Sham model: Z ρ ( u ) ρ ( v ) ρ ( u ) V ( u ) + 1 Z E ( ρ ) = T ( ρ ) + | u − v | dudv + E xc ( ρ ) 2 electron-electron Kinetic Exchange Molecular electron-nuclei Coulomb repulsion energy correlat. energy energy attraction At equilibrium: 8

Coulomb Interactions in Fourier • Coulomb potential energy: Diagonalized in Fourier: 9

Coulomb in Fourier Dictionary ω 2 ω 1 10

Large Scale Instabilities 11

Coulomb Multiscale Factorizations • Multiscale regroupment of interactions: For an error ✏ , interactions can be reduced to O (log ✏ ) groups ( Rocklin, Greengard) Fast multipoles 12

Scale separation with Wavelets rotated and dilated: ω 2 real parts imaginary parts ω 1 13

Wavelet Interference for Densities 14

Sparse Wavelet Regression ω 2 ω 1 Theorem: For any ✏ > 0 there exists wavelets with 15

Dictionaries for Quantum Energies 16

Atomization Density Approximate density ˜ ρ x ( u ) Electronic density ρ x ( u ) 17

Sparse Linear Regressions 18

x Fourier and Wavelets Regressions Regression: 8 Fourier 7 Wavelet Scattering �� 6 Coulomb | 5 − 4 | � 3 � 2 1 log 2 M 0 1 2 3 4 5 6 7 8 9 10 19 Model Complexity log ( M )

Energy Regression Results 20

Wavelet Dictionary Rotations θ 1 | ρ ∗ ψ j 1 , θ 1 ( u ) | Scales j 1 ρ ( u ) 21

Wavelet Dictionary Rotations θ 1 Scales j 1 | ρ ∗ ψ j 1 , θ 1 ( u ) | 22

Scattering Dictionary Rotations θ 1 Recover translation variability: | ρ ∗ ψ j 1 , θ 1 | ∗ ψ j 2 , θ 2 ( u ) Recover rotation variability: | ρ ∗ ψ j 1 , · ( u ) | ~ ψ l 2 ( θ 1 ) Scales j 1 | ρ ∗ ψ j 1 , θ 1 ( u ) | 23

Scattering Dictionary Rotations θ 1 Recover translation variability: | ρ ∗ ψ j 1 , θ 1 | ∗ ψ j 2 , θ 2 ( u ) Recover rotation variability: | ρ ∗ ψ j 1 , · ( u ) | ~ ψ l 2 ( θ 1 ) Scales j 1 Combine to recover roto-translation variabiltiy: || ρ ∗ ψ j 1 , · | ∗ ψ j 2 , θ 2 ( u ) ~ ψ l 2 ( θ 1 ) | | ρ ∗ ψ j 1 , θ 1 ( u ) | 24

Scattering Second Order | ρ ∗ ψ j 1 , θ 1 ( u ) | , j 1 fixed || ρ ∗ ψ j 1 , · | ∗ ψ j 2 , θ 2 ( u ) ~ ψ l 2 ( θ 1 ) | j 1 , l 2 fixed S c a l e s j 2 Rotations θ 2 25

Scattering Dictionary 26

Scattering Regression x Regression: 8 Fourier 7 Wavelet Scattering �� 6 Coulomb | 5 − 4 | � 3 � 2 1 log 2 M 0 1 2 3 4 5 6 7 8 9 10 27 Model Complexity log ( M )

Quantum Chemistry Energy Regression Results 28

From 2D to 3D Scattering M . Eickenberg , M . Hirn ω 2 ω 1 29

Reconstruction from Scattering Joan Bruna Original images of N 2 pixels: Order m = 2 Reconstruction from { k x k 1 , k x ? λ 1 k 1 , k | x ? λ 1 | ? λ 2 k 1 } : O (log 2 2 N ) coe ff .

Ergodic Texture Reconstructions Joan Bruna Original Textures 2D Turbulence Second order Gaussian Scattering: O (log N 2 ) moments E ( | x ? λ 1 | ) , E ( || x ? λ 1 | ? λ 2 | )

Digit Classification: MNIST Joan Bruna y = f ( x ) Linear Classifier S J x x Classification Errors LeCun et. al.

Complex Image Classification CalTech 101 data-basis: Edouard Oyallon Arbre de Joshua Castore Ancre Metronome Nénuphare Bateau S J x Linear Classif. x y Rigid Mvt. computes invariants Classification Accuracy Data Basis Deep-Net Scat.-2 CalTech-101 85% 80% CIFAR-10 90% 80%

Conclusion • Quantum energy regression involves generic invariants to rigid movements, stability to deformations, multiscale interactions • These properties require scale separations, hence wavelets. • Multilayer wavelet scattering create large number of invariants • Equivalent to deep networks with predefined wavelet filters • Knowing physics provides the invariants: can avoid learning representations 34