Anima Anandkumar BEYOND BLACK BOXES: INFUSING STRUCTURE INTO MACHINE LEARNING
TRINITY OF AI ALGORITHMS COMPUTE DATA 2
DEEP LEARNING IS DATA-HUNGRY STRUCTURE-INFUSED LEARNING Learning Data Priors = +
Learning Data Priors = + How to use structure and domain knowledge to design Priors? Examples of Priors Graphs/Tensors • • Symbolic rules Physical laws • Simulations • Generative models •
Learning Data Priors = + How to use structure and domain knowledge to design Priors? Examples of Priors Graphs/Tensors • • Symbolic rules Physical laws • Simulations • Generative models •
NEXT GENERATION AI FROM PREDICTION TO GENERATION DOG DOG y y x x 6
Generative Adversarial Networks 7
TURING TEST FOR FACE GENERATION http://www.whichfaceisreal.com/index.php 8
WHAT IS THE SOLUTION OF A GAN? GAN objective for loss function 𝑀 min max 𝑀 , Real Images Discriminator Loss Latent Generator Code
COMPETITION IN GANS Generator vs Discriminator optimization • Training GANs challenging : unstable and mode collapse • Standard optimization: alternating gradient descent • Fails even for simple case with bilinear objectives
A VERY SIMPLE GAN Current optimization methods:
COMPETITIVE GRADIENT DESCENT Florian Schäfer A NeurIPS 2019
INTUITIONS Opponent awareness in optimization Components in decision making: Belief about loss function 1. Uncertainty of environment 2. Anticipation of action of adversary 3.
COMPETITIVE GRADIENT DESCENT Linear for one player → Bilinear for two players 𝑧 𝑔 + 1 2 𝑔 𝑧 + 𝑧 𝑈 𝛼 𝑦 𝑙+1 − 𝑦 𝑙 = argmin 𝑦 𝑔 + 𝑦 𝑈 𝛼 𝑦 𝑔 + 𝑦 𝑈 𝐸 𝑦𝑧 2𝜃 𝑦 𝑈 𝑦 𝑧 + 1 2 𝑧 + 𝑧 𝑈 𝛼 𝑧 𝑙+1 − 𝑧 𝑙 = argmin 𝑧 + 𝑦 𝑈 𝛼 𝑦 + 𝑦 𝑈 𝐸 𝑦𝑧 2𝜃 𝑧 𝑈 𝑧 Local approximation is interactive! Nash equilibrium of local game
A VERY SIMPLE GAN CGD converges for all step sizes:
RESULTS ON W-GAN We use architecture intended for Best performance by WGAN WGAN-GP, no additional loss with Adaptive-CGD (no hyperparameter tuning regularization)
TAKE-AWAYS Competitive optimization in GANs Competition between generation and discriminator leads to instability and mode collapse Stabilization in competitive gradient descent through opponent awareness Implicit competitive regularization through stabilization State of art performance with no tuning and explicit penalties
DISENTANGLEMENT IN STYLEGANS Weili Nie Terro Karas Animesh Garg A Ankit Patel 18
CONTROLLABLE STYLEGAN ● Multi-resolution generator and discriminator ● Generator conditions on factor code ● Mapping network – conditioned styles – modulate each block in the synthesis network ● Encoder shares all layers with discriminator except last ○ to predict factor code. 19
SEMI-SUPERVISED LEARNING 20
DISENTANGLED LEARNING • Loss on encoder encourages disentanglement • Loss incorporates code of real images when available (semi-supervised) 21
5% OF LABELLED DATA ON CELEB-A (256X256) 22
1% OF LABELLED DATA ON ISAAC SIM (512X512) 23
TAKE-AWAYS Disentangled learning in StyleGAN Controllable photo-realistic generation in StyleGANs Disentanglement through reconstruction of style codes Semi-supervised learning with very little labeled data 24
Flow-based Generative Models 25
CONTINUOUS NORMALIZING FLOWS p(x) - Exact likelihood - Invertibility - Use ODE solvers p(z) 26
CONTINUOUS NORMALIZING FLOWS z l z L = x z = z 0 p 0 ( x 0 ) p l ( x l ) p L ( x L ) ✓ @f ( z ( t ); ✓ ◆ @ l og p ( z ( t )) ) 0 ) = z, @z ( t ) = f ( z ( t ) ,t ; ✓ = Tr z ( t ) @t @z ( t ) @t 27 ⇒
CONTINUOUS NORMALIZING FLOWS z l z L = x z = z 0 p 0 ( x 0 ) p l ( x l ) p L ( x L ) ✓ @f ( z ( t ); ✓ ◆ @ l og p ( z ( t )) ) 0 ) = z, @z ( t ) = f ( z ( t ) ,t ; ✓ = Tr z ( t ) @t @z ( t ) @t Ordinary Differential Equation 28 ⇒
NEURAL ODE MODELS FOR TIME SERIES 29
AI4PHYSICS: TURBULENCE FORECASTING VIA NEURAL ODE Gavin Portwood, Peetak Mitra, Mateus Dias Riberio, Tan Mihn Nguyen, Anima Anandkumar 30
MOTIVATION Fluid Turbulence is difficult to model: • Multi-scale: Dynamics of different scales non-linear and coupled • Direct numerical simulation (DNS) resolves all scales and hence, is expensive • Current reduced order models are heuristic, not high fidelity • Can neural ODEs help? 31
EXPERIMENTAL RESULTS Neural ODE predictions of evolution of dissipation rate are better • Neural ODE generalizes well on unseen test data • 32
TAKE-AWAYS Flow-based generative models Good alternatives to GANs when likelihood estimates are needed Ideal for scientific applications with underlying differential equations and need for uncertainty estimates Challenges in scaling 33
FEEDBACK GENERATIVE MODELS
NEXT GENERATION AI FROM PREDICTION TO GENERATION DOG DOG y y One model to do both? x x 35
Taking inspiration from Biological brains.. 36
HUMAN VISION: FEEDFORWARD & FEEDBACK Second-order predictions Hierarchical predictive coding First-order predictions First-order streams Prediction error (superficial pyramidal cells) Expectations (deep pyramidal cells) Second-order streams Modulatory backward connections “ Prediction error Excitatory (forward) connections (precision) Inhibitory (backward) connections Expectations (precision) s)” Visual input a ‘ ’ w Representational sharpening : “ [ a] s a Interaction between the feedforward and feedback connections are crucial for core object recognition in human vision 37 y—t s—h ” n– e– s— s ex—i g—a — –
DECONVOLUTIONAL GENERATIVE MODEL object latent category • Feedback network for variables deconvolution • Latent variables to overcome non-invertibility intermediate rendering Nhat Ankit Michael Tan Rich Ho Patel Jordan Nyugen Baranuik image
CONVOLUTIONAL NEURAL NETWORK WITH FEEDBACK y . . . . Yujia Sihui . Pinglei . Huang Dai Bao . . . . . . Doris Tan Rich x Tsaos Nyugen Baranuik CNN-F performs approximate belief propagation through feedforward CNN and feedback generative model 39
CNN-F CAN RECOVER CLEAN DATA Noise Blur Occlusion Input CNN-F Reconstruction
CNN-F YIELDS ROBUST CLASSIFICATION
TAKE-AWAYS Adding feedback to CNNs Biological inspiration can lead to more robust architectures Combining feedforward and feedback networks for iterative inference Robust prediction on degraded images
CONCLUSION Generative models are important in many applications • Photorealistic generation now possible • • Competitive optimization to stabilize GAN training Controlled disentangled generation in Style GANs • Continuous flow-based models for physical applications • • Brain inspired CNN with feedback • Outstanding challenges: How to combine generative models with simulations and • downstream taskss
Thank you 44
Recommend
More recommend
