Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To - PowerPoint PPT Presentation

Unifying Orthogonal Monte Carlo Methods From Kac’s Random Walks To Hadamard Multi Rademachers Krzysztof Choromanski, Mark Rowland Wenyu Chen, Adrian Weller

The Phenomenon of Orthogonal Monte Carlo Estimators Estimation task: Applications: ● dimensionality reduction (JLT-mechanisms) isotropic distribution ● scaling kernel methods (e.g. Gaussian) (random feature maps) ● hashing algorithms (e.g. LSH) ● (sliced) Wasserstein Standard MC approach: distances (WGANs, autoencoders...) ● reinforcement learning (ES algorithms) ● and many, many more...

The Phenomenon of Orthogonal Monte Carlo Estimators Sampling from the Estimation task: Haar measure on the O(d) group isotropic distribution (e.g. Gaussian) Expensive: O(n^3 time) The Orthogonal Trick: guarantees unbiasedness # of samples of the often implies better MC estimator <= dim accuracy

Towards Computational Efficiency: The Zoo of Approximate MCs

Towards Computational Efficiency: The Zoo of Approximate MCs ...

Towards Computational Efficiency: The Zoo of Approximate MCs ... ...

Towards Computational Efficiency: The Zoo of Approximate MCs ... size size N x N N/2 x N/2 ... Constraints: ● ●

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Kac’s random walk matrices Givens rotations Hadamard-Rademacher Chains Givens reflections reflection in the jth coordinate

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Kac’s random walk matrices Hadamard-Rademacher Chains

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Hadamard-MultiRademachers Butterfly Matrices

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Still more accurate estimator than unstructured MC baseline

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Log-Linear Time Complexity (unstructured MC baseline has quadratic)

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Analysis of the Total Variation Distance between Haar measure on d-sphere and measure induced by standard Kac’s random walk on d-sphere estimator estimated value Pillai, Smith 2016 Kac’s random walk on d-sphere mixes in O(d log d) steps

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Analysis of the Total Variation Distance between Haar measure on d-sphere and measure induced by standard Kac’s random walk on d-sphere estimator estimated value Pillai, Smith 2016 Kac’s random walk on d-sphere mixes in O(d log d) steps More careful analysis of the LHS

Maximum Mean Discrepancy Experiment How Does It Work In Practice ? Kernel Approximation via Random Features Reinforcement Learning via ES-methods Accuracy Computational Efficiency

Thank you for your attention !

Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To - PowerPoint PPT Presentation

Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To Hadamard Multi Rademachers Krzysztof Choromanski, Mark Rowland Wenyu Chen, Adrian Weller The Phenomenon of Orthogonal Monte Carlo Estimators Estimation task: Applications:

Monte Carlo Methods An introduction to Monte Carlo (MC) methods How to use MC methods

Chapter 5: Monte Carlo Methods Monte Carlo methods are learning methods Experience

Eligibility Traces Unifying Monte Carlo and TD key algorithms: TD( ), Sarsa( ), Q( )

Monte Carlo Methods Guojin Chen Christopher Cprek Chris Rambicure Monte Carlo Methods 1.

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Monte Carlo Estimation 7 January 2019 OSU CSE 1 Monte Carlo Methods Class of computational

Monte Carlo Methods and Area Estimates CS3220 - Summer 2008 Jonathan Kaldor Monte Carlo Methods

Monte Carlo Integration Monte Carlo methods use random numbers for sampling Although

Monte Carlo Methods Prof. Kuan-Ting Lai 2020/4/17 Monte Carlo Methods Learn directly from

Monte Carlo Methods Why Monte Carlo? Computation of number ( 1) n +1 = 1 1 2 + 1

Course on Inverse Problems Albert Tarantola Lesson VIII: Monte Carlo Methods Monte Carlo

An introduction to Monte Carlo methods in XRF analysis Tom Schoonjans Joint ICTP-IAEA school,

Planning and Optimization G4. Asymptotically Suboptimal Monte-Carlo Methods Gabriele R oger

Monte Carlo methods for magnetic systems Zoltn Nda Babe-Bolyai University Dept of

Kinetic Monte Carlo Methods C n e n o t i t e a r t f u o p r CC DC Kinetic

Monte Carlo Methods CS60077: Reinforcement Learning Abir Das IIT Kharagpur Sep 06 and 12, 2019

Monte Carlo Methods CS60077: Reinforcement Learning Abir Das IIT Kharagpur Oct 05 and 06, 2020

Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include:

Monte Carlo Methods Lecture notes for MAP001169 Based on Script by Martin Sk old adopted by

Monte Carlo Methods Lecture notes for MAP001169 Based on Script by Martin Sk old adopted by

Monte Carlo Methods and Simulating Quarks Michael Creutz Brookhaven Lab 1946: Stanislaus Ulam

Adapting quasi-Monte Carlo methods to simulation problems in weighted Korobov spaces Christian

Monte Carlo Methods in Particle Physics Bryan Webber University of Cambridge IMPRS, Munich