Learning Particle Physics by Example: Accelerating Science with - PowerPoint PPT Presentation

Learning Particle Physics by Example: Accelerating Science with Generative Adversarial Networks arXiv:1701.05927, arXiv:1705.02355 Luke de Oliveira Partner, Manifold Visiting Researcher, Lawrence Berkeley National Lab � @lukede0 � @lukedeo � lukedeo@manifold.ai � https://ldo.io S7666, May 11th, GPUTech 2017

Outline • High Energy Particle Physics / LHC Physics • How do we probe phenomena in the physical sciences? What problems does this introduce? • Location Aware GANs / CaloGANs • Applications to other domains

Why this matters • GANs provide a viable strategy for speeding up numerically intensive simulation - extends to other disciplines (medicine, weather, nuclear) • Basic science provides theory-driven success metrics — feedback loop for pure ML research

High Energy Physics at the Large Hadron Collider

Physics @ LHC • 27 km ring at the border of Switzerland and France • Accelerate bunches of protons at 99.999999% the speed of light - approx. 90 μ s per lap • 14TeV collision energy let’s us probe constituents, investigate rare decays, examine new theories (supersymmetry, extra dimensions, gravitons) • Many experiments: ATLAS , CMS, LHCb, ALICE, …

What does a proton-proton collision look like? 💦

What is a Particle Detector? ATLAS detector is a 3D camera with a temporal component that measures energy deposition in different materials and in different environments. What are we measuring? What does it look like?

Acquiring Labels • Unlike many fields we’ve seen here at GPUTech, we can’t label data in High Energy Physics • The most brilliant physicist cannot look at a detector read out and say what happened • Use a constructivist approach, start with theory , then simulate (we know labels from simulation)

Finding Physics using Simulation • To perform searches for new physics or precision measurements of known particles we simulate billions of collisions • Allows us to calculate yield, discovery significance, build classifiers • This is very hard - we always have all problems associated with transfer learning

High Energy Physics Simulation • Encode infinite-dimensional integrals, full Quantum Field Theory into Monte Carlo Simulation • Using approximations, can bound the time required • Can we do better?

Simulation Overview QFT / Theory Simulate proton-proton interactions Simulate showering Interaction with detector materials / geometry …

Simulation Overview QFT / Theory Simulate proton-proton interactions Simulate showering Interaction with CPU intensive detector materials / (~60% of worldwide geometry computing grid) …

Why is this hard? This is the first series of layers. We need to model interactions on the 10 -10 to resolve showering of particles

Trifecta of Issues Full Simulation is slow Hard to model tail- Detector simulation can take statistics minutes per event Time To get a few rare events in kinematically unfavorable regions, we often have to simulate a large number of events Petabytes of 
 Simulated Data Large amounts of simulated data needs Disk Rare to be stored 
 and transferred Space Events

Our Panacea Fast Portable Specialized

Our Panacea Fast Portable Specialized …GANs?

Location Aware Generative Adversarial Networks arXiv:1701.05927

Generative Adversarial Networks (GAN) Turn generative modeling into a two player, non-cooperative game. tries to tell fake/real tries to produce real looking samples

Defining a Step 0 - Jet Images Avoid producing full event, focus on jets - cone shaped collections of hadrons (e.g., electrons) forming showers η = 0 η = - ∞ η = + ∞

Defining a Step 0 - Jet Images Key insight - unroll detector, well behaved manifold [arXiv/{1407.5675,1511.05190}]

From Particle Physics � Computer Vision Jet Images are 2D images where pixel intensity is transverse momentum (component perpendicular to proton path) Single Jet Image Mean of 10 4 Jet Images

Step 0 Task • Generate Jet Images using traditional software (P YTHIA ) • Learn a generative model to reproduce jets with high fidelity • Fidelity defined as ability to reproduce physics quantities (manifolds) such as mass, other theoretically motivated quantities which we don’t have for most generative tasks • Provide a speedup over traditional generation

Defining Characteristics • Large (5 orders of magnitude) dynamic range • Sparse (7%) occupancy of images • Changing location of 1 pixel activation � wildly different characteristics in spacetime • Important features have high Lipschitz constant

Location Aware GAN (LAGAN) • Joint work with Michela Paganini (Yale) and Benjamin Nachman (Lawrence Berkeley National Lab) • arXiv:1701.05927 • Design domain-specific tweaks and modifications to GAN architecture • Use AC-GAN variant with 2 classes • QCD (background) and W bosons (signal) — well understood from Physics

LAGAN

Architectural Details • Latent space: • Perform Hadamard product between latent space and lookup table for conditioning (AC-GAN) • Rely on locally connected building blocks (i.e., convolutions without weight sharing) to leverage spacetime symmetries

LAGAN guidelines for sparse data • Activation functions Leaky ReLU ReLU (rec. in GAN lit.) (chosen to induce sparsity) • Batch Normalization Stabilize gradients, help with HDR • Minibatch Discrimination

Qualitative Assessment 5 random Jet Images nearest LAGAN-generated neighbor GAN-generated signal - background Real signal - background

Checking Against Theory Can a GAN recover these distributions without being trained on them?

Checking Physical Properties n- subjettiness jet mass

Our Solution Fast Portable Specialized

Our Solution Currently used Fast benchmark Our solution

Our Solution We condition on particle class, but can easily condition on mass, momentum, … Specialized

Our Solution Trained LAGAN weights: 20 MB Source code: 480 kB Portable

Can we go one step further? • We’ve shown that we can perform showering • Can we simulate how positron / pion / photon showers interact with materials? Most expensive part of simulation. • Large simulation package, G EANT 4, can take 5 mins per electron to generate its interaction with a detector.

CaloGAN arXiv:1705.02355

Solution: CaloGAN • Simulate subatomic particles interacting with calorimeter media • Shoot charged pions, positrons, and photons at cells of super-cooled liquid argon , use GAN to simulate what happens

How does an electron look in liquid argon? 2D slice - - Geant4, Pb Absorber, lAr Gap, 10 GeV e Geant4, Pb Absorber, lAr Gap, 10 GeV e 30 1000 1000 1000 1000 Cell Energy [MeV] direction [mm] Local Energy Deposit [MeV] direction [mm] 200 200 900 900 900 900 25 150 150 800 800 800 800 η η 100 100 700 700 700 700 20 50 50 600 600 600 600 0 15 0 500 500 500 500 400 400 400 400 50 50 − − 10 300 300 300 300 100 100 − − 200 200 200 200 150 150 − − 5 100 100 100 100 200 200 − − 0 0 0 0 0 200 150 100 50 0 50 100 150 200 200 150 100 50 0 50 100 150 200 − − − − − − − − Depth from Calorimeter Center [mm] Depth from Calorimeter Center [mm] What we can We simulate exact (x, y, z) read out is this

How does an electron look in liquid argon? 3 layers, unequal resolution

CaloGAN Architecture • Condition on Energy - express conservation of energy as a approx. sub-differentiable constraint • Same principles as LAGAN • Jointly train all three layers of a calorimeter

CaloGAN Generator • Three independent streams, one per calorimeter layer • Custom attention mechanism decides how much from one layer to carry to the next layer

CaloGAN Generator

Nearest Generated Samples (Positrons) Real 1st layer deposition CaloGAN 1st layer deposition Real 2nd layer deposition CaloGAN 2nd layer deposition Real 3rd layer deposition CaloGAN 3rd layer deposition

Matching Physical Features Faithfully reproduce energy-per-layer

Conditioning on Energy

GPU Based Speed-up Up to a 10 5 speed-up!

Other Applications • Recent successes with GANs in Cosmology • Interest in fluids, weather, medicine (nuclear) • Nuclear Physics (Pheno) • Combustion

The Future • Generative Models, GANs in particular, show promise for modeling exotic scientific / engineering phenomena • Let’s collaborate — at Berkeley Lab and Manifold, we’re interested in practical uses of GANs and Machine Learning in general

Reproducible Research • We have open-sourced our code, dataset, and analysis procedure for both works. • https://github.com/hep-lbdl/adversarial-jets • https://github.com/hep-lbdl/CaloGAN

Thanks!