M 3 : Taking Microkernels to the Next Level Nils Asmussen FOSDEM, - PowerPoint PPT Presentation

barkhauseninstitut.org M 3 : Taking Microkernels to the Next Level Nils Asmussen FOSDEM, 02/02/2020, Brussels

About Me Nils Asmussen PhD last year at the OS chair of the TU Dresden Low-level system programming and microkernels Worked on several microkernel-based OSes in the past Escape, own hobby OS (presented here in 2013): https://github.com/Nils-TUD/Escape NRE, userland for NOVA: https://github.com/TUD-OS/NRE M 3 , presented today: https://github.com/TUD-OS/M3 Since 2019 at the Barkhausen Institut 2 / 23

Barkhausen Institut Research institute in Dresden, founded end of 2017 Currently about 30 people Low-latency and secure IoT systems Focus on research and demonstrators 2 / 23

Barkhausen Institut Research institute in Dresden, founded end of 2017 Currently about 30 people Low-latency and secure IoT systems Focus on research and demonstrators RF Design Privacy Wireless MPSoC OS Lab 1 / 23

Motivation Microkernel-based systems have proven valuable for several objectives Security Robustness Real time Flexibility Recently, new challenges are coming from the hardware side Heterogeneous systems Third-party components Security issues of complex general-purpose cores 2 / 23

Heterogeneous Systems Demanded by performance and energy requirements Big challenge for OSes: single shared kernel on all cores does no longer work OSes need to be prepared for processing elements with different feature sets 3 / 23

Third-party Components Market pressure forces us to integrate third-party components We should not trust these components Currently, ofen no isolation between them Bug in such a component can compromise whole system (see Broadcom incident) 4 / 23

Security Issues of Complex General-purpose Cores 20 known atacks (and counting ...) Allow to leak private data, sometimes bypassing all security measures of the core Mitigations exist, but these are complex and costly These security holes have been lurking in CPUs for many years Should we still trust these complex cores to properly enforce the isolation between different sofware components? 5 / 23

Microkernel-based System as Foundation Application Application Service Service Microkernel Core Core Core 6 / 23

Microkernel-based System as Foundation Application Application Service Service Management Microkernel Core Core Core 6 / 23

Microkernel-based System as Foundation Application Application Service Service Management Microkernel Enforcement Core Core Core 6 / 23

Microkernel-based System as Foundation Application Application Service Service Management Microkernel Enforcement Core Core Core FPGA TPU GPU 6 / 23

Outline 1 The New System Architecture M 3 : The Operating System 2 3 What are the Benefits? 7 / 23

Outline 1 The New System Architecture M 3 : The Operating System 2 3 What are the Benefits? 8 / 23

Hardware/Operating System Co-Design 9 / 23

Hardware/Operating System Co-Design Core GPU TPU Core Core FPGA 9 / 23

Hardware/Operating System Co-Design Core GPU TPU Core Core FPGA 9 / 23

Hardware/Operating System Co-Design Core GPU TPU Key ideas: TCU as new hardware TCU TCU TCU component Core Core FPGA TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design Core GPU TPU Key ideas: TCU as new hardware TCU TCU TCU component Core Core FPGA TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design App App Kernel Core GPU TPU Key ideas: TCU as new hardware TCU TCU TCU component Kernel on dedicated PE App Core Serv Core FPGA Serv TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design Manage App App Kernel Core GPU TPU Key ideas: TCU as new hardware TCU TCU TCU Enforce Enforce Enforce component Kernel on dedicated PE Kernel manages, App Core Serv Core FPGA Serv TCU enforces Enforce TCU Enforce TCU Enforce TCU 9 / 23

Hardware/Operating System Co-Design Takes µ -kernels to the next level: App App Kernel Core GPU TPU TCU as secure foundation TCU TCU TCU App Core Serv Core FPGA Serv TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design Takes µ -kernels to the next level: App App Kernel Core GPU TPU TCU as secure foundation Heterogeneity: TCU TCU TCU Uniform interface App Core Serv Core FPGA Serv TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design Takes µ -kernels to the next level: App App Kernel Core GPU TPU TCU as secure foundation Heterogeneity: TCU TCU TCU Uniform interface Untrusted HW comp.: Protected by TCU App Core Serv Core FPGA Serv TCU TCU TCU 9 / 23

Hardware/Operating System Co-Design Takes µ -kernels to the next level: App App Kernel Core GPU TPU TCU as secure foundation Heterogeneity: TCU TCU TCU Uniform interface Untrusted HW comp.: Protected by TCU App Core Serv Core FPGA Serv Side channels: Physical isolation TCU TCU TCU 9 / 23

Communication TCU provides endpoints to: App App Kernel Core GPU TPU Access memory (contiguous range, byte granular) TCU TCU TCU M App Core Serv Core DRAM TCU TCU 10 / 23

Communication TCU provides endpoints to: App App Kernel Core GPU TPU Access memory (contiguous range, byte granular) TCU TCU TCU M S S Receive messages into a receive buffer Send messages to a App Core Serv Core receiving endpoint DRAM TCU TCU R R 10 / 23

Communication TCU provides endpoints to: App App Kernel Core GPU TPU Access memory (contiguous range, byte granular) TCU TCU TCU M S S Receive messages into a receive buffer Send messages to a App Core Serv Core receiving endpoint DRAM Replies for RPC TCU TCU R R 10 / 23

Isolation Kernel tile User tile User tile TCU-based isolation: App App Kernel Core GPU TPU Additional protection layer TCU TCU TCU User tile User tile App Core Serv Core DRAM TCU TCU 11 / 23

Isolation Kernel tile User tile User tile TCU-based isolation: App App Kernel Core GPU TPU Additional protection layer TCU TCU TCU User tile User tile App Core Serv Core DRAM TCU TCU 11 / 23

Isolation Kernel tile User tile User tile TCU-based isolation: App App Kernel Core GPU TPU Additional protection layer TCU TCU TCU M S S Only kernel tile can establish communication User tile User tile channels App Core Serv Core DRAM TCU TCU R R 11 / 23

Isolation Kernel tile User tile User tile TCU-based isolation: App App Kernel Core GPU TPU Additional protection layer TCU TCU TCU M S S Only kernel tile can establish communication User tile User tile channels User tiles can only use App Core Serv Core DRAM established channels TCU TCU R R 11 / 23

Outline 1 The New System Architecture M 3 : The Operating System 2 3 What are the Benefits? 12 / 23

OS Design M 3 : M icrokernel-based syste m for het. m anycores M 3 FS Kernel (or L4 ± 1) Implemented from scratch in Rust and C++ Drivers, filesystems, etc. implemented on user tiles pipes App Kernel manages permissions, using capabilities TCU enforces permissions (communication, memory access) App App Kernel is independent of other tiles 13 / 23

M 3 System Call User tile Kernel tile App PE Kernel PE TCU TCU R 14 / 23

M 3 System Call User tile Kernel tile App PE Kernel PE TCU TCU S R 14 / 23

M 3 System Call User tile Kernel tile App PE Kernel PE TCU TCU S R 14 / 23

M 3 System Call User tile Kernel tile App PE Kernel PE TCU TCU S R 14 / 23

M 3 System Call User tile Kernel tile App PE Kernel PE TCU TCU S R 14 / 23

OS Service Access Client Server TCU TCU DRAM 15 / 23

OS Service Access File Protocol: Used for: files, pipes, ... Client Server TCU TCU DRAM 15 / 23

OS Service Access File Protocol: Used for: files, pipes, ... Client Server Data in memory TCU TCU DRAM 15 / 23

OS Service Access File Protocol: Used for: files, pipes, ... Client Server req(in/out) Data in memory Msg channel between client and server TCU S R TCU req(in) for next input piece resp(pos,len) req(out) for next output piece DRAM 15 / 23

OS Service Access File Protocol: Used for: files, pipes, ... Client Server req(in/out) Data in memory Msg channel between client and server TCU S R TCU M req(in) for next input piece resp(pos,len) req(out) for next output piece Server configures client’s memory EP DRAM 15 / 23

OS Service Access File Protocol: Used for: files, pipes, ... Client Server req(in/out) Data in memory Msg channel between client and server TCU S R TCU M req(in) for next input piece resp(pos,len) req(out) for next output piece Server configures client’s memory EP DRAM Client accesses data via TCU 15 / 23

Outline 1 The New System Architecture M 3 : The Operating System 2 3 What are the Benefits? 16 / 23

Example System Kernel Core Core Core Core Core 5G TCU TCU TCU TCU TCU TCU 17 / 23

Example System Monitor FS Net Kernel Control Core Core Core Core Core 5G TCU TCU TCU TCU TCU TCU 17 / 23