JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution - PowerPoint PPT Presentation

JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs Eunji Jeong , Sungwoo Cho, Gyeong-In Yu, Joo Seong Jeong, Dong-Jin Shin, Byung-Gon Chun Demo 1

Introduction Challenge Solution Results Deep Neural Networks Images From: http://www.mdpi.com/ https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/ Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7 Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017 https://skymind.ai/wiki/generative-adversarial-network-gan https://en.wikipedia.org/wiki/Reinforcement_learning https://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23 2

Introduction Challenge Solution Results Deep Neural Networks Images From: http://www.mdpi.com/ https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/ Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7 Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017 https://skymind.ai/wiki/generative-adversarial-network-gan https://en.wikipedia.org/wiki/Reinforcement_learning https://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23 Deep Learning (DL) Frameworks 3

Introduction Challenge Solution Results Deep Neural Networks Images From: http://www.mdpi.com/ https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/ Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7 Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017 https://skymind.ai/wiki/generative-adversarial-network-gan https://en.wikipedia.org/wiki/Reinforcement_learning https://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23 Imperative DL Frameworks Symbolic DL Frameworks 1.x 2.0 4

Introduction Challenge Solution Results Symbolic DL Frameworks Imperative DL Frameworks ✓ Build a Symbolic Graph ✓ Directly Execute the Computations ✓ Execute the Graph x def build_graph(g): def linear(x): W x = g.placeholder(float) return W * x + b linear = g.add(g.mul(W, x), b) Mul linear(x_data) b build_graph (graph) run_graph (graph, x_data) Add 1.x 2.0 5

Introduction Challenge Solution Results Symbolic DL Frameworks Imperative DL Frameworks + Easy to Optimize + Direct Execution: Easy to Program & Debug + Compiler Optimization Pros + Parallel Execution of Operations + Deploy on GPU, Cluster, Mobile,... - Decoupled View: - Hard to Optimize Cons Hard to Program & Debug 6

Introduction Challenge Solution Results JANUS: Combining the Best of Both Worlds x Imperative DL Program 3 Transparent def foo(x): Mul Conversion tmp = mul(3, x) 2 return add(tmp, 2) Add Symbolic DL Graph “Easy Programmability” “High Performance” 7

Introduction Challenge Solution Results ? Imperative DL Program with Dynamic Features for item in sequence: state = Cell(state, item) outputs += [state] Symbolic DL Graph Dynamic Control Flow ● Dynamic Types ● Impure Functions ● ... ● 8

Introduction Challenge Solution Results state ● Correct Merge Next ? i<N ● Slow Imperative DL Program Switch Cell with Dynamic Features Symbolic DL Graph for item in sequence: state = Cell(state, item) outputs += [state] 9

Introduction Challenge Solution Results state ● Correct Merge Next ? i<N ● Slow Imperative DL Program Switch Cell with Dynamic Features Symbolic DL Graph for item in sequence: state = Cell(state, item) state outputs += [state] Cell ● Fast ● Incorrect Cell Cell Symbolic DL Graph 10

Introduction Challenge Solution Results Solution: Speculative Graph Generation and Execution ● [Performance] Speculatively Specialize the Graph ○ Make reasonable assumptions based on the execution history ( Profiling ) ○ Run specialized graph (Common Case) ● [Correctness] Validate Assumptions ○ Fallback if an assumption is broken (Rare Case) 11

Overall Workflow on JANUS Introduction Challenge Solution Results Imperative DL Program for item in sequence: state = rnn(state, item) outputs += [state] Imperative Executor Python Interpreter . Pre-defined DL Operations . 12

Overall Workflow on JANUS Introduction Challenge Solution Results Imperative DL Program len: 3 for item in sequence: state = rnn(state, item) outputs += [state] Profiler Imperative Executor Python Interpreter . Pre-defined DL Operations . 13

Overall Workflow on JANUS Introduction Challenge Solution Results Symbolic DL Graph Imperative DL Program len:3 state Graph for item in sequence: state = rnn(state, item) Generator len == 3 Cell outputs += [state] ? Cell Assert Cell Symbolic Graph Executor Python Interpreter . Pre-defined DL Operations . 14

Overall Workflow on JANUS Introduction Challenge Solution Results Symbolic DL Graph Imperative DL Program len:3 state for item in sequence: state = rnn(state, item) len == 3 Assumption Cell outputs += [state] ? Failure Cell Assert Cell Symbolic Graph Executor Python Interpreter . Pre-defined DL Operations . 15

Overall Workflow on JANUS Introduction Challenge Solution Results Symbolic DL Graph Imperative DL Program len:3 state for item in sequence: state = rnn(state, item) len == 3 Cell outputs += [state] ? Cell Assert Cell Symbolic Graph Executor Python Interpreter . Pre-defined DL Operations . 16

Overall Workflow on JANUS Introduction Challenge Solution Results Imperative DL Program len: ? for item in sequence: state = rnn(state, item) outputs += [state] Profiler Imperative Executor Python Interpreter . Pre-defined DL Operations . 17

Overall Workflow on JANUS Introduction Challenge Solution Results Symbolic DL Graph Imperative DL Program len: ? state Graph for item in sequence: state = rnn(state, item) Generator i<N outputs += [state] Merge Next Switch Cell Symbolic Graph Executor Python Interpreter . Pre-defined DL Operations . 18

Introduction Challenge Solution Results ImageNet Test Error with ResNet50 36 GPUs Symbolic JANUS Imperative 3.4x Faster Convergence Time 19

Introduction Challenge Solution Results Normalized Training Throughput Single Machine Imperative JANUS Symbolic LeNet CNN ResNet-50 Inception-v3 LSTM 47.6x over RNN Imperative LM TreeRNN TreeNN TreeLSTM A3C 96.0% of DRL Symbolic PPO AN GAN PIX2PIX Imp. 20

Thank You! 21