Trillium: The Code is the IR Amogh Akshintala, Hangchen Yu , Arthur - PowerPoint PPT Presentation

Trillium: The Code is the IR Amogh Akshintala, Hangchen Yu , Arthur M. Peters, Christopher J. Rossbach

Brief Overview GPGPU Virtualization End-to-end comparison of prior approaches Lessons learnt: ● Virtual ISA unnecessary ● Para-virtual API remoting only feasible option A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #2

General purpose computing on GPUs Scientific Computing ↑↑↑ Performance Gain CPU GPU Machine Learning A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #3

The need to virtualize GPUs P3.2xlarge 1x V100, $2,200/mo Performance P2.xlarge 1x K80, $650/mo g3s.xlarge 1x M60, $540/mo Cost Credits to BitFusion Inc. A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #4

The need to virtualize GPUs P3.2xlarge 1x V100, $2,200/mo Virtual V100 0.75x V100, $1,500 Performance Virtual V100 0.5x V100, $1,000 P2.xlarge 1x K80, $650/mo g3s.xlarge 1x M60, $540/mo Cost Credits to BitFusion Inc. A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #5

Existing techniques are impractical Sharing Interposition Isolation Compatibility Slowdown Pass Through 1x ❌ Full-virtualization Everybody sacrifices something User-mode API Remoting Para-virtualization A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #6

Sharing Interposition Isolation Compatibility Slowdown Full-virtualization ✔︐ ✔︐ ✔︐ ❌ 100x VM Application Vendor Library Native stack Vendor Library Vendor Driver Vendor Driver Virtual GPU GPU Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #8

Sharing Interposition Isolation Compatibility Slowdown User-mode API Remoting ✔︐ / ❌ ✔︐ ❌ ❌ 1.5x VM Application Custom User-mode Library Custom API Server Native stack Vendor Library Vendor Driver GPU Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #11

Sharing Interposition Isolation Compatibility Slowdown Para-virtualization ✔︐ ✔︐ ✔︐ ❌ 10x VM Application Custom User-mode Library Native stack Custom Guest Driver Vendor Library Custom Virtual GPU Vendor Driver Interface Translator GPU Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #13

End-to-end performance comparison Y. Suzuki, S. Kato, H. Yamada, K. Kono, “GPUvm: Why Not Virtualizing GPUs at the Hypervisor?”, ATC’14 A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #18

More details in the paper... A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #19

Everybody sacrifices something Sharing Interposition Isolation Compatibility Slowdown Full-virtualization ✔︐ ✔︐ ✔︐ ❌ 100x ✔︐ / ❌ User-mode API Remoting ✔︐ ❌ ❌ 1.5x Para-virtualization 10x ✔︐ ✔︐ ✔︐ ❌ A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #20

Para-virtual API-remoting... Sharing Interposition Isolation Compatibility Slowdown Full-virtualization ✔︐ ✔︐ ✔︐ ❌ 100x ✔︐ / ❌ User-mode API Remoting ✔︐ ❌ ❌ 1.5x Para-virtualization 10x ✔︐ ✔︐ ✔︐ ❌ Para-virtual API remoting ✔︐ ✔︐ ✔︐ 1.5x A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #22

SVGA2: TGSI as IR and vISA VM Application OpenGL/DX11 Libraries GLSL Code Native stack vmwgfx.ko Vendor Library TGSI SVGA Device Vendor Driver Translator GPU Native ISA Hypervisor TGSI, Tungsten Graphics Shader Infrastructure, is an intermediate language for describing shaders. A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #26

Xen-SVGA: computing support VM Application Mesa3D OpenCL Library OpenCL Code Native stack TGSI LLVM Compiler Vendor Library TGSI TGSI Nouveau Nouveau Native ISA Xen-SVGA Device GPU Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #31

Xen-SVGA: computing support VM Application Mesa3D OpenCL Library OpenCL Code Native stack TGSI LLVM Compiler Vendor Library TGSI TGSI Nouveau Nouveau Native ISA Xen-SVGA Device GPU Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #32

vISA - Boon or bane? PTX, a low-level parallel thread execution virtual machine and instruction set architecture (ISA). A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #33

Trillium: Eliminates vISA VM Application Mesa3D OpenCL Library Native stack Vendor Library OpenCL Code Nouveau Nouveau Native ISA GPU Trillium Device Hypervisor A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #37

So are we done? A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #39

Checkpoint Sharing Interposition Isolation Compatibility Slowdown ✔︐ / ❌ User-mode API Remoting ✔︐ ❌ ❌ 1.5x Para-virtualization ✔︐ ✔︐ ✔︐ ❌ 10x Trillium 2.4x ✔︐ ✔︐ ✔︐ ❌ ✓ Virtual ISA is unnecessary ✓ Para-virtual API remoting system performs better A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #40

Para-virtual API-remoting... Sharing Interposition Isolation Compatibility Slowdown ✔︐ / ❌ User-mode API Remoting ✔︐ ❌ ❌ 1.5x Para-virtualization ✔︐ ✔︐ ✔︐ ❌ 1-10x Trillium 2.4x ✔︐ ✔︐ ✔︐ ❌ ✕ Interposing too low in the stack ✕ API-specific A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #41

Para-virtual API-remoting... Sharing Interposition Isolation Compatibility Slowdown ✔︐ / ❌ User-mode API Remoting ✔︐ ❌ ❌ 1.5x Para-virtualization ✔︐ ✔︐ ✔︐ ❌ 1-10x Trillium 2.4x ✔︐ ✔︐ ✔︐ ❌ AvA ✔︐ ✔︐ ✔︐ ✔︐ <1.5x ✕ Interposing too low in the stack ✕ API-specific A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #42

Automatic virtualization of accelerators Para-virtual VM CL.h + Annotations Application API Stack Generated User-mode Library Native stack AvA Guest Driver Vendor Library AvA Virtual Device Vendor Driver Generated API Server Accelerators Hypervisor H. Yu, A. M. Peters, A. Akshintala, C. J. Rossbach, Automatic Virtualization of Accelerators, HotOS’19 A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #45

Preliminary development effort Type APIs LoC Time Difficulty ★★★★ GPUvm Full-virt 1 20 000 Years ★★★★ SVGA2 Para-virt 2 MANY! Years Automatic ★ AvA Para-virtual 9 OpenCL: 835 Days API Remoting A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #46

Conclusion Lessons ● Virtual ISA is unnecessary ○ Decouple device virtualization from GPU ISA virtualization ● Para-virtual API remoting can lead to better performance and properties New para-virtual API remoting system: AvA ● “No IR” enables interposition at user -mode APIs ● Compatibility is compensated A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium , Virt’19 #47