tvm
play

TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS - PowerPoint PPT Presentation

Nov 17, 2023 •1.23k likes •1.46k views

TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS stack Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA AutoVTA Hardware FPGA ASIC Fleet

  1. μ TVM Deep Learning on Bare-Metal Devices Pratyush Patel

  5. No OS stack

  6. Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA AutoVTA Hardware FPGA ASIC Fleet

  7. Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Most bare-metal devices LLVM, CUDA VTA AutoVTA do not support LLVM Hardware FPGA ASIC Fleet

  8. Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet

  9. Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Upstreamed! LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet

  10. Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Upstreamed! LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet Many other backends

  11. μ TVM builds upon AutoTVM μ TVM Runtime μ Device C Code Generator API send program run optimize

  12. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run

  13. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code infer.c

  14. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code vendor 
 gcc infer.c infer.o

  15. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

  16. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API custom loader run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

  17. A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API send program custom loader run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

  18. Next steps • Iron out interfaces with actual hardware. • Optimize with AutoTVM. • Support restricted and configurable model sizes. • Enable custom data types such as fixed-point precision formats. Get in touch! Pratyush Patel — patelp1@cs.uw.edu

Recommend

More recommend