learning execution through neural code fusion
play

Learning Execution through Neural Code Fusion Zhan Shi, Kevin - PowerPoint PPT Presentation

Aug 19, 2023 •15 likes •435 views

Learning Execution through Neural Code Fusion Zhan Shi, Kevin Swersky, Danny Tarlow, Paruhasarathy Ranganathan , Milad Hashemi Overview Proprietary + Confidential Motivation Background Neural Code Fusion Experimental Results

  1. Learning Execution through Neural Code Fusion Zhan Shi, Kevin Swersky, Danny Tarlow, Paruhasarathy Ranganathan , Milad Hashemi

  2. Overview Proprietary + Confidential ● Motivation ● Background ● Neural Code Fusion ● Experimental Results ● Conclusion 2 Confidential + Proprietary

  3. Proprietary + Confidential Motivation 3 Confidential + Proprietary

  4. Proprietary + Confidential Proprietary + Confidential 2% Pergormance/Year is the New Normal Source: Paruhasarathy Ranganathan, More Moore: Thinking Outside the (Server) Box 4

  5. Motivation Proprietary + Confidential Dynamic speculative execution ● Branch prediction, value prediction, cache replacement, ○ prefetching... 5 Confidential + Proprietary

  6. Motivation Proprietary + Confidential Dynamic speculative execution ● Branch prediction, value prediction, cache replacement, ○ prefetching... Static source code ● Variable naming, fjnding bugs, algorithm classifjcation, program ○ synthesis… Pergormance-related tasks: device mapping, thread coarsening, ○ throughput prediction... 6 Confidential + Proprietary

  7. Motivation Proprietary + Confidential Dynamic speculative execution ● Branch prediction, value prediction, cache replacement, ○ prefetching... Static source code ● Variable naming, fjnding bugs, algorithm classifjcation, program ○ synthesis… Pergormance-related tasks: device mapping, thread coarsening, ○ throughput prediction... Both views provide useful features ● 7 Confidential + Proprietary

  8. Example: a “Simple” Case for Branch Prediction Proprietary + Confidential for (i = 0; i < k; i++) { } 8 Confidential + Proprietary

  9. Example: a “Simple” Case for Branch Prediction Proprietary + Confidential Highly biased for (i = 0; i < k; i++) { } 9 Confidential + Proprietary

  10. Example: a “Simple” Case for Branch Prediction Proprietary + Confidential Highly biased for (i = 0; i < k; i++) { Branch history doesn’t help } 10 Confidential + Proprietary

  11. Example: a “Simple” Case for Branch Prediction Proprietary + Confidential Highly biased while(...){ generate k; for (i = 0; i < k; i++) { Branch history doesn’t help } } 11 Confidential + Proprietary

  12. Example: a “Simple” Case for Branch Prediction Proprietary + Confidential Highly biased while(...){ generate k; for (i = 0; i < k; i++) { Branch history doesn’t help } } ● Jump out when “close enough” ● Predictable if we knew the relation [Static] i and k are compared [Dynamic] values of i and k 12 Confidential + Proprietary

  13. Proprietary + Confidential Background: Graph Neural Networks 13 Confidential + Proprietary

  14. Background: Graph Neural Networks Proprietary + Confidential Typical deep learning operates ● on IID data points. 14 Confidential + Proprietary

  15. Background: Graph Neural Networks Proprietary + Confidential What if the data points had relational information? ● Battaglia et al., 2018 15 Confidential + Proprietary

  16. Background: Graph Neural Networks Proprietary + Confidential Message passing ● Input graph 16 Confidential + Proprietary

  17. Background: Graph Neural Networks Proprietary + Confidential Message passing ● Step 0 Input graph 17 Confidential + Proprietary

  18. Background: Graph Neural Networks Proprietary + Confidential Message passing ● Step 0 Step 1 Input graph 18 Confidential + Proprietary

  19. Background: Graph Neural Networks Proprietary + Confidential Message passing ● Step 0 Step 1 Step 2 Input graph 19 Confidential + Proprietary

  20. Background: Graph Neural Networks Proprietary + Confidential Message passing ● Step 0 Step 1 Step 2 Input graph GRU GRU 20 Confidential + Proprietary

  21. Proprietary + Confidential Programs as Graphs Allamanis et al., 2017 21 Confidential + Proprietary

  22. Proprietary + Confidential Representing Static and Dynamic Information Graphs are an efgective representation for static code ● How do we generally represent dynamic information in a ● model? 22 Confidential + Proprietary

  23. Proprietary + Confidential Neural Code Fusion 23 Confidential + Proprietary

  24. Full System Proprietary + Confidential 24 Confidential + Proprietary

  25. Assembly vs Source Code Proprietary + Confidential ● Highly structured 25 Confidential + Proprietary

  26. Assembly vs Source Code Proprietary + Confidential ● Highly structured 26 Confidential + Proprietary

  27. Assembly vs Source Code Proprietary + Confidential ● Highly structured ● Directly relate data to program semantics 27 Confidential + Proprietary

  28. Assembly vs Source Code Proprietary + Confidential ● Highly structured ● Directly relate data to program semantics ● Easy to use for architecture tasks 28 Confidential + Proprietary

  29. Code Fusion Graph Representation Proprietary + Confidential 29 Confidential + Proprietary

  30. Dynamic Tasks: Control Flow and Data Flow Proprietary + Confidential Control fmow (branch prediction) ● predict whether a branch statement will be taken or not taken. ● Set branch instruction node to be the target node. ● Binary classifjcation ● 30 Confidential + Proprietary

  31. Dynamic Tasks: Control Flow and Data Flow Proprietary + Confidential Control fmow (branch prediction) ● predict whether a branch statement will be taken or not taken. ● Set branch instruction node to be the target node. ● Binary classifjcation ● Data fmow (prefetching) ● predict which address will be accessed next. ● Set src node to be the target node. ● Predict 64-bit address ● 31 Confidential + Proprietary

  32. Multi-Task Representation Proprietary + Confidential Many other static/dynamic tasks can be defjned on the ● graph simultaneously Value prediction, indirect branch prediction, memory ○ disambiguation, caching… 32 Confidential + Proprietary

  33. Dynamic Snapshots Proprietary + Confidential Snapshots ● The values of the set of variable nodes ○ Captured during program execution ○ Used to initialize the graph neural network ● 33 Confidential + Proprietary

  34. Representation Study Proprietary + Confidential Number “3” in difgerent representations ● Categorical: [1, 0, 0, 0] ○ Scalar: 3 ○ Binary: 11 ○ 34 Confidential + Proprietary

  35. Representation Study Proprietary + Confidential ● Correctly predict when to jump out for(k=0; k < n; k+=3){ Sample k values as training data ● for (i = 0; i < k; i++) { } } 35 Confidential + Proprietary

  36. Representation Study: Proprietary + Confidential Results ● Binary > scalar > categorical ○ 36 Confidential + Proprietary

  37. Proprietary + Confidential Experimental Results 37 Confidential + Proprietary

  38. Experimental Setup Proprietary + Confidential Benchmarks ● SPEC06 INT ○ Tasks ● Dynamic: control fmow (branch prediction) and data fmow (prefetching) ○ Static: algorithm classifjcation ○ Offmine evaluation for both NCF and baselines ● 70% training ○ 30% testing ○ 38 Confidential + Proprietary

  39. Control-fmow (Branch Prediction) and Data-fmow Proprietary + Confidential (Prefetching) 39 Confidential + Proprietary

  40. Algorithm Classifjcation Proprietary + Confidential Test the usefulness of the learned representation ● We pre-train our GNN on the control-fmow task ● A simple linear SVM model ● We get 96% vs 95.3% (50M lines of LLVM IR ) using 200k lines of ● assembly with no external data sources. 40 Confidential + Proprietary

  41. Summary Proprietary + Confidential NCF combining static and dynamic information ● creates useful representations ○ Difgerent from the traditional dynamic models in architecture ● Data is usually purely dynamic ○ Model is history-based ○ Enhances static models with dynamic program behavior ● Learned representation can also transfer to a unseen static task ○ 41 Confidential + Proprietary

  42. Thank you! Questions?

Recommend

Flicker: Flicker: Minimal TCB Code Execution Minimal TCB Code Execution Jonathan M. McCune

Flicker: Flicker: Minimal TCB Code Execution Minimal TCB Code Execution Jonathan M. McCune

Flicker: Flicker: Minimal TCB Code Execution Minimal TCB Code Execution Jonathan M. McCune Carnegie Mellon University March 25, 2008 Bryan Parno, Arvind Seshadri Adrian Perrig, Michael Reiter 1 Password Reuse People often use 1

748 views • 49 slides
CPSC 213 Read assembly code anything that can be determined before execution (by compiler)

CPSC 213 Read assembly code anything that can be determined before execution (by compiler)

Learning Goals 1 Learning Goals 2 Big Ideas: First Half Memory Read Assembly Static and dynamic Endianness and memory-address alignment CPSC 213 Read assembly code anything that can be determined before execution (by

373 views • 6 slides
Artificial Neural Networks for Multimodal Information Fusion Friedhelm Schwenker Institute of

Artificial Neural Networks for Multimodal Information Fusion Friedhelm Schwenker Institute of

Artificial Neural Networks for Multimodal Information Fusion Friedhelm Schwenker Institute of Neural Information Processing University of Ulm Cairo University April 9, 2010 Schwenker UUlm ANN Informationfusion Outline Artificial neural

707 views • 47 slides
Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks Hardik Sharma Georgia Institute of Technology Jongse Park Arm, Inc. Naveen Suda Liangzhen Lai University of California, San Diego

275 views • 24 slides
Rate vs temporal code about synchrony Learning objectives: Learning objectives: To

Rate vs temporal code about synchrony Learning objectives: Learning objectives: To

Rate vs temporal code about synchrony Learning objectives: Learning objectives: To understand how synchrony can be measured To gain an understanding of a simple neural model of synchronization To gain an understanding of a simple neural

592 views • 25 slides
Order Is A Lie Are you sure you know how your code runs ? Order in code is not respected by

Order Is A Lie Are you sure you know how your code runs ? Order in code is not respected by

Order Is A Lie Are you sure you know how your code runs ? Order in code is not respected by Compilers Processors ( out-of-order execution) SMP Cache Management Understanding execution order in a multithreaded context is out of reach

600 views • 48 slides
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware Florian

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware Florian

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware Florian Tramr (joint work with Dan Boneh) Intel, Santa Clara August 30 th 2018 Trusted execution of ML: 3 motivating scenarios 1. Outsourced ML Data

781 views • 21 slides
Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the stimulus? What aspects of the stimulus does the system encode? (capacity is limited) Mark van Rossum What information can be extracted from spike trains:

779 views • 16 slides
Dynamic Code Generation and Execution for Monte Carlo Simulations Vaivaswatha Nagaraj Steve

Dynamic Code Generation and Execution for Monte Carlo Simulations Vaivaswatha Nagaraj Steve

Dynamic Code Generation and Execution for Monte Carlo Simulations Vaivaswatha Nagaraj Steve Karmesin Talk ID: 23282 Outline Introduction Code Generation Compilation &amp; Execution Results Conclusion and Future Work

518 views • 30 slides
Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to

Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to

Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to a given stimulus Decoding (homunculus): Given response, what was the stimulus? Mark van Rossum Given firing pattern, what will be the motor

191 views • 15 slides
Structured Fusion Networks for Dialog Shikib Mehri*, Tejas Srinivasan*, Maxine Eskenazi Language

Structured Fusion Networks for Dialog Shikib Mehri*, Tejas Srinivasan*, Maxine Eskenazi Language

Structured Fusion Networks for Dialog Shikib Mehri*, Tejas Srinivasan*, Maxine Eskenazi Language Technologies Institute, Carnegie Mellon University Code: https://github.com/shikib/structured_fusion_networks Motivation Neural systems show strong

1.09k views • 66 slides
Figure 1: Visual representation of M 3 Fusion . II. D ATA The study was carried out on Reunion

Figure 1: Visual representation of M 3 Fusion . II. D ATA The study was carried out on Reunion

M 3 Fusion : A Deep Learning Architecture for Multi- { Scale/Modal/Temporal } satellite data fusion P. Benedetti, D. Ienco, R. Gaetano, K. Os e, R. Pensa and S. Dupuy Abstract Modern Earth Observation systems provide sensing data at different

107 views • 9 slides
Exploiting Out-of-Order-Execution Processor Side Channels to Enable Cross VM Code Execution Sophia

Exploiting Out-of-Order-Execution Processor Side Channels to Enable Cross VM Code Execution Sophia

Exploiting Out-of-Order-Execution Processor Side Channels to Enable Cross VM Code Execution Sophia DAntoine REcon 2015 The Cloud 06/19/2015 Exploiting Out-of-Order-Execution 2/46 Cloud Computing (IaaS) Virtual instances Hypervisors

491 views • 46 slides
JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs Eunji

JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs Eunji

JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs Eunji Jeong , Sungwoo Cho, Gyeong-In Yu, Joo Seong Jeong, Dong-Jin Shin, Byung-Gon Chun Demo 1 Introduction Challenge Solution Results Deep Neural

452 views • 21 slides
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of

Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of

Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information Sudha Rao 1 and Hal Daum III 1,2 1 University of Maryland College Park 2 Microsoft Research New York CODE + DATA :

1.54k views • 111 slides
Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
1 Real Neural Learning Artificial Neuron Model Model network as a graph with cells as nodes

1 Real Neural Learning Artificial Neuron Model Model network as a graph with cells as nodes

Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to understand natural biological systems CS 391L: Machine Learning through computational modeling. Massive parallelism allows

92 views • 6 slides
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Code &amp; Data : github.com/snakeztc/NeuralDialog-LAED Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation Tiancheng Zhao, Kyusong Lee and Maxine Eskenazi Language Technologies Institute, Carnegie

730 views • 28 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
System Architectures and Techniques for Efficient, Secure, and Trusted Code Execution Mario

System Architectures and Techniques for Efficient, Secure, and Trusted Code Execution Mario

System Architectures and Techniques for Efficient, Secure, and Trusted Code Execution Mario Werner May 7, 2020 Graz University of Technology Why do we care about code? www.tugraz.at 10:20 July 18 W i r e l e s s - G A D S L G

599 views • 49 slides
Out line Neural net wor ks Percept r on Neural Net works Supervised learning

Out line Neural net wor ks Percept r on Neural Net works Supervised learning

Out line Neural net wor ks Percept r on Neural Net works Supervised learning algorit hms f or neur al net works J uly 7, 2005 Reading: R&amp;N Ch 20.5 CS 486/ 686 Univer sit y of Wat erloo 2 CS486/686 Lecture Slides (c)

85 views • 5 slides
Learning Discourse-level Diversity for Neural Dialog Models Using Conditional Variational

Learning Discourse-level Diversity for Neural Dialog Models Using Conditional Variational

Learning Discourse-level Diversity for Neural Dialog Models Using Conditional Variational Autoencoders Tiancheng Zhao, Ran Zhao and Maxine Eskenazi Language Technologies Institute Carnegie Mellon University Code&amp;Data:

585 views • 29 slides

More recommend