Machine Learning for Environmental Grand Challenges Shakir Mohamed - PowerPoint PPT Presentation

Machine Learning for Environmental Grand Challenges Shakir Mohamed Research Scientist, DeepMind @shakir_za #AI4ER Cambridge

Principles to Products Advancing Climate and Fairness and Assistive Autonomous Healthcare Applications Science Energy Safety Technology systems World Objects and Reasoning Planning Explanation Rapid Learning Simulation Relations Information Prediction Uncertainty Information Gain Causality Probability Bayesian Hypothesis Estimation Principles Asymptotics Theory Analysis Testing Theory 2 Shakir Mohamed

Statistical Operations Estimation Hypothesis Inference and Learning Testing Comparison Summarisation Experimental Modelling Design Data Enumeration 3 Shakir Mohamed

Statistical Operations Inference Decision-making What we can What we can know about our data do with our data. Inference Comparison Summarisation Data Enumeration 4 Shakir Mohamed

Pa ru I: Pathways in Machine Learning Shakir Mohamed Research Scientist, DeepMind @shakir_za #AI4ER Cambridge

On Models Model: Description of the world, of data, A probabilistic model writes out these of potential scenarios, of processes. models using the language of probability Peak Bad Accident hour Weather Tra ffi c Sirens Jam Probabilistic models let you learn Most models in machine probability distributions of data. learning are probabilistic. 6 Shakir Mohamed

Statistical Inference Direct Indirect Learning Laplace Maximum Two Sample Method of approximation Likelihood Comparison Moments Principles Maximum a Variational Approx Bayesian Transpo ! ation posteriori Inference Computation methods Integr. Nested Max Mean Cavity Methods Laplace Approx Discrepency Expectation Markov chain Maximisation Monte Carlo Noise Sequential Contrastive Monte Carlo

Model Evidence Model evidence (or marginal likelihood, pa ru ition function) : Integrating out any global and local variables enables model scoring, comparison, selection, moment estimation, z normalisation, posterior computation and prediction. Learning principle: Model Evidence prob- f ( z ) q ( z ) f(z) p ( z ) func- ) di- Z wn p ( x ) = p ( x , z ) d z the are z x Integral is intractable in general Basic idea: Transform the and requires approximation. integral into an expectation over a simple, known distribution. 8 Shakir Mohamed

Variational Methods KL [ q ( z | y ) k p ( z | y )] Approximation class True posterior q φ ( z ) Deterministic approximation procedures with bounds on probabilities of interest. Fit the variational parameters. 9 Shakir Mohamed

Learning by Comparison Basic idea: Transform into q( x ) p*( x ) z learning a model of We compare the estimated the density ratio. distribution q(x) to the true f(z) distribution p*(x) using samples. x Learning principle: Two-sample tests Ratios p ( x (1) ) p ( x (2) ) p ∗ ( x ) q ( x ) = 1 p ∗ ( x ) = q ( x ) Interest is not in estimating the marginal probabilities, only in how they are related. 10 Shakir Mohamed

Estimation by Comparison Density Estimation Two steps by Comparison 1. Use a hypothesis test or H 0 : p ∗ = q θ vs. p ∗ 6 = q θ comparison to obtain some L ( θ , φ ) model to tells how data from our model di fg ers from observed data. Density Ratio Density Di ff erence r φ = p ∗ r φ = p ∗ − q θ 2. Adjust model to be tu er q θ match the data distribution using the comparison model Mixtures with identical moments B f [ r ∗ k r ] from step 1. Bregman Max Mean Moment Class Probability f-Divergence Divergence Discrepency Matching Estimation 11 Shakir Mohamed f ( u ) = u log u − ( u + 1) log( u + 1) Mohamed and Lakshminarayanan (2016)

Algorithms for Generative Models z z z ~ q( z | x ) Generator Inference Model Network p( x | z ) q( z | x ) x gen x real x ~ p( x | z ) D φ Data x Fully-observed auto- Prescribed latent variable Implicit latent variable models regressive models models and variational inference and estimation-by-comparison PixelCNN and Variational Generative Wavenet Autoencoders Adversarial Networks 12 Shakir Mohamed

Stochastic Optimisation Common gradient problem Z r φ E q φ ( z ) [ f θ ( z )] = r q φ ( z ) f θ ( z ) d z Typical problem areas • Sensitivity analysis 1. Pathwise estimator : Di fg erentiate the function f ( z ) • Generative models and inference 2. Score-function estimator : Di fg erentiate the • Reinforcement learning and control density q ( z|x ) • Operations research and inventory control • Monte Carlo simulation • Finance and asset pricing 13 Shakir Mohamed Mohamed et al. (2019)

Progress in Generative Models ImageNet Visual Quality of Independent Samples Conv. Generative Conv- Pixel VAE DRAW Adversarial Network DRAW RNN 14 Shakir Mohamed

Perception-Action Loops Biological Computational perception-action loop perception-action loop 15 Shakir Mohamed

Environment Simulation Action Action Action a t-1 a t a t+! Action-conditional and latent-only transitions. State State State … s t-1 s t s t+1 Grounded representations in actions and observations, using simulation to suppo ru grounding. m t-1 m t m t+1 Data Data Data x t-1 x t x t+1 16 Shakir Mohamed Chiappa et al. (2017)

Shakir Mohamed

Intrinsic Motivation Equip agents with mechanisms to produce and learn from internal rewards that can guide behaviour, when external rewards are absent. 1 1 2 3 Escaping a Predator True MI 4 5 6 6 18 Shakir Mohamed Mohamed and Rezende (2015)

Mnih et al. (2015)

AlphaZero Generalising AlphaGo to any 2-player game Fully general; No opening book; No endgame database; No heuristics; Starts from random All learned without any reference to past human games 20 Shakir Mohamed Silver et al. (2018)

Applications in Healthcare outcomes 2 clinician experience 3 1 Better clinical Enhance patient and Reduce costs Model AKI Predicted Optional longer history Data used by the model 24h 48h 72h 48h history New entry Time 6h unknown 24h Outpatient events Admission 21 Shakir Mohamed

Predicting Organ Failure Make predictions of AKI �������������������������� ���������������������������� up to 48hr ahead. ������������������� Provide a window for meaningful action. �������� For the most severe ���������������������� �������������������������� ������������������� cases, can detect up to ����������������������� ��������������� ������������������� 90% of cases. 22 Shakir Mohamed Tomasev et al. (2019)

Critical Practice for ML Consider the uses of our models. What are the dual uses of generative models. How do we think critically about these uses, educate, regulate, co-design these tools. 23 Shakir Mohamed Bansal et al. (2018)

Dual-uses and Value Alignment Future of Life Institute, Value alignment map 24

Neutrality and Universality Neutrality Traps Universality • The Po ru ability Trap: Failure to understand how repurposing ‘A mono-cultural view of ethics algorithmic solutions designed for one social context may be conceives itself as the only valid one. In order to avoid this kind of ethical inaccurate / do harm when applied to a di fg erent context. chauvinism and colonialism it is • The Formalism Trap: Failure to account for the full meaning of necessary that transcultural ethics social concepts such as fairness, which be resolved through arise from an intercultural dialogue mathematical formalisms. instead of thinking of itself as • The Ripple E fg ect Trap : Failure to understand how the inse ru ion universal without noticing its own of technology into an existing social system changes the cultural bias.’ Capurro, 2004 behaviours and embedded values of the pre-existing system . • The Solutionism Trap : Failure to recognise the possibility that the best solution to a problem may not involve technology. 25 Shakir Mohamed

Pa ru II: AI for Environmental Risk Shakir Mohamed Research Scientist, DeepMind @shakir_za #AI4ER Cambridge

Extreme Weather Events Segment Tropical Cyclones, Atmospheric Given CAM5 outputs of a tropical cyclone and its initial position, track its trajectory. Rivers from background Tools for data assimilation, analysis of NWP simulations, and new types of decision suppo ru . 28 Shakir Mohamed Mudigonda et al. (2017)

Hybrid Physical Process Modelling Predict future sea su rg ace temperature (SST) from previous synthetic SST data from NEMO (Nucleus for European Modeling of the Ocean) Physical Model: Advection-Di fg usion Equation Solution Key Idea: Predict w 29 Shakir Mohamed De Beznac et al. (2017)