sequential neural processes
play

Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung - PowerPoint PPT Presentation

Sep 29, 2022 •322 likes •888 views

Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung Son 3 Sungjin Ahn 1 1 Rutgers University 2 SAP 3 ETRI *Equal Contribution Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying

  1. Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung Son 3 Sungjin Ahn 1 1 Rutgers University 2 SAP 3 ETRI *Equal Contribution

  2. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  3. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  4. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  5. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  6. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  7. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  8. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  9. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  10. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  11. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  12. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  13. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  14. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  15. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  16. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  17. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  18. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  19. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  20. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  21. Sequential Neural Processes Transition Model Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  22. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  23. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  24. Sequential Neural Processes Meta-Transfer Learning. "We need not learn everything from current context but only use it to update our prior hypothesis."

  25. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  26. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  27. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  28. Demonstrations

  29. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 1st time-step without context

  30. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 10th time-step without context

  31. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 20th time-step without context. Beyond training time horizon

  32. Meta-Transfer Learning

  33. Meta-Transfer Learning

  34. Meta-Transfer Learning

  35. Comparing against GQN

  36. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  37. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  38. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  39. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  40. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  41. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  42. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  43. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  44. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  45. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  46. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  47. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

Recommend

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives Refinement and trace semantics Automaton view Refinement checking algorithm Failures Semantics 2 CSP Communicating Sequential Processes,

735 views • 57 slides
9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from

9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from

9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Andrej Karpathy, Bo Xie, Zsolt Kira Sequential and Temporal Data Many applications exhibited by dynamically changing states Language (e.g.

1.12k views • 91 slides
Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent Neural Networks Compact schematic drawing of standard multi-layer perceptron (MLP) output hidden input Modeling sequential data So far we

796 views • 50 slides
Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling Kyunghyun Cho New York University & Facebook AI Research Joint work with Jason Lee and Elman Mansimov Neural sequence modeling An arbitrary

461 views • 25 slides
Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Sequential Data with Neural Networks Recurrent Neural

303 views • 4 slides
Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank

Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank

Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank Neural Networks Modern Artificial Intelligence Most recent advances in AI use neural networks Especially true for language (NLP) What is a

366 views • 35 slides
Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides

Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides

Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides INRIA-IRISA, Rennes, France Outline The change detection (disorder) problem Overview of existing results Lordens criterion and the CUSUM

382 views • 23 slides
Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning

Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning

Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Sequential Decision Making In general: exponentially large decision tree s1 a b . 9 . 1 . 2 . 8 s2 s3 s12 s13 a b a b

364 views • 12 slides
Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and

Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and

CS11-747 Neural Networks for NLP Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and Sequential Data NLP and Sequential Data NLP is full of sequential data NLP and Sequential Data NLP is full of

2.05k views • 171 slides
Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department

Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department

Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department of Mathematical Sciences National Chengchi University January 21, 2016 24th Annual Workshop on Differential Equations This is a joint work with

1.03k views • 68 slides
Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba

Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba

Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba Toth Joint work with Harald Oberhauser Mathematical Institute, University of Oxford International Conference on Machine Learning, July 2020

292 views • 26 slides
Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making

Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making

Module 14 Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Markov Decision Processes MDPs: Fully Observable MDPs Decision maker

398 views • 21 slides
Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes Synchronous communication with global channels Channels are the only state that is shared between threads No shared objects, shared

667 views • 42 slides
Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes Synchronous communication with global channels Channels are the only state that is shared between threads No shared objects, shared

795 views • 33 slides
Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Artificial Intelligence 15-381 April 5, 2007 Sequential Decision Problems & Markov Decision Processes Recap of last lecture Reasoning over time - Markov Processes - Hidden Markov Models - modeling state transitions - probability of

321 views • 16 slides
Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and

Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and

CS11-747 Neural Networks for NLP Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and Sequential Data NLP is full of sequential data Words in sentences Characters in words Sentences in

672 views • 36 slides
CSCI 447/547 MACHINE LEARNING Outline Introduction Sequence Data Sequential Memory

CSCI 447/547 MACHINE LEARNING Outline Introduction Sequence Data Sequential Memory

Recurrent Neural Networks CSCI 447/547 MACHINE LEARNING Outline Introduction Sequence Data Sequential Memory Recurrent Neural Networks Vanishing Gradient LSTMs and GRUs Introduction Uses: Speech Recognition

616 views • 24 slides
Sequential Files : Outline ! Overview ! Ordered vs. Unordered ! Physical sequential Files !

Sequential Files : Outline ! Overview ! Ordered vs. Unordered ! Physical sequential Files !

Sequential Files : Outline ! Overview ! Ordered vs. Unordered ! Physical sequential Files ! Physical Linked Sequential Files ! COBOL rasitjutrakul Sequential Files " Logically the records are stored consecutively Physical sequential file

335 views • 22 slides
Mehlers Formula, Branching Processes, and Compositional Kernels of Deep Neural Networks

Mehlers Formula, Branching Processes, and Compositional Kernels of Deep Neural Networks

Mehlers Formula, Branching Processes, and Compositional Kernels of Deep Neural Networks Tengyuan Liang Hai Tran-Bach May 26, 2020 Motivations & Questions DNN and Kernels (Rahimi, Recht 08; Belkin et al 18; Jacot et al 19) .

964 views • 32 slides
Spiking neural models: from point processes to partial differential equations. Julien Chevallier

Spiking neural models: from point processes to partial differential equations. Julien Chevallier

Spiking neural models: from point processes to partial differential equations. Julien Chevallier Co-workers: M. J. Cceres, M. Doumic and P. Reynaud-Bouret LJAD University of Nice INRIA Sophia-Antipolis 2016/06/09 Introduction Thinning

993 views • 83 slides
Deep Neural Networks as Gaussian Processes Jaehoon Lee Google Brain Workshop on Accelerating the

Deep Neural Networks as Gaussian Processes Jaehoon Lee Google Brain Workshop on Accelerating the

Deep Neural Networks as Gaussian Processes Jaehoon Lee Google Brain Workshop on Accelerating the Search for Dark Matter with Machine Learning April 10, 2019 Based on Published in ICLR 2018, htups://arxiv.org/abs/1711.00165 Open source

501 views • 33 slides
Spiking neural models: from point processes to partial differential equations. J. Chevallier

Spiking neural models: from point processes to partial differential equations. J. Chevallier

Spiking neural models: from point processes to partial differential equations. J. Chevallier Advisors: P. Reynaud Bouret (Nice) and F. Delarue (Nice) Colloque jeunes probabilistes et statisticiens Les Houches 2016/04/18 Context Two scales

677 views • 49 slides
Random Sampling Florian Schoppmann August 24, 2010 Non-Sequential Sequential Sequential with

Random Sampling Florian Schoppmann August 24, 2010 Non-Sequential Sequential Sequential with

Random Sampling Florian Schoppmann August 24, 2010 Non-Sequential Sequential Sequential with Reservoir Sequential With Reservoir and Replacement . . . . . . . . . . . . Sampling Algorithms Input: List [ 1 . . . N ] Length of

555 views • 13 slides
Some New Developments in Sequential Analysis Extension of Optimality of Well Known Stopping

Some New Developments in Sequential Analysis Extension of Optimality of Well Known Stopping

University of Patras [ 1 ] Some New Developments in Sequential Analysis Extension of Optimality of Well Known Stopping Times Extension of Walds First Identity to Markov Processes George V. Moustakides Dept. Computer Engineering and

454 views • 21 slides

More recommend