Scaling container policy management with kernel features Joe - PowerPoint PPT Presentation

Scaling container policy management with kernel features Joe Stringer Cilium.io Linux Plumbers 2019, Lisbon, Portugal Joe Stringer Scaling container policy with eBPF Sep 11, 2019 1 / 29

Overview 1 Background 2 Deploying fast datapaths fast 3 Identity-based security 4 Layer 7 security Joe Stringer Scaling container policy with eBPF Sep 11, 2019 2 / 29

Background Kubernetes Architecture 101 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 3 / 29

Background Kubernetes networking plugins Plumb local connectivity (CNI) Connect remote nodes Services / loadbalancing Network policy https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/ Joe Stringer Scaling container policy with eBPF Sep 11, 2019 4 / 29

Background Cilium Agent runs on each node Native eBPF dataplane Identity-based security Scalable Joe Stringer Scaling container policy with eBPF Sep 11, 2019 5 / 29

Background What does it mean to scale? Manage cluster interactions Minimize unnecessary events Reduce event sizes ... Optimize work within the node Apply datapath changes efficiently https://cilium.io/blog/2019/04/24/cilium-15 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 6 / 29

Deploying fast datapaths fast Joe Stringer Scaling container policy with eBPF Sep 11, 2019 7 / 29

Deploying fast datapaths fast BPF plumbing Joe Stringer Scaling container policy with eBPF Sep 11, 2019 8 / 29

Deploying fast datapaths fast ELF Templating Joe Stringer Scaling container policy with eBPF Sep 11, 2019 9 / 29

Deploying fast datapaths fast 1K nodes: Scaling to 60k pods Joe Stringer Scaling container policy with eBPF Sep 11, 2019 10 / 29

Deploying fast datapaths fast Future directions Optimize verifier execution: O ( n ) → O (1) Support code path templatization Joe Stringer Scaling container policy with eBPF Sep 11, 2019 11 / 29

Identity-based security Joe Stringer Scaling container policy with eBPF Sep 11, 2019 12 / 29

Identity-based security Joe Stringer Scaling container policy with eBPF Sep 11, 2019 13 / 29

Identity-based security Policy example a p i V e r s i o n : "cilium.io/v2" kind: CiliumNetworkPolicy d e s c r i p t i o n : "Restrict deathstar access to empire ships" metadata: name: "deathstar -ingress" spec: e n d p o i n t S e l e c t o r : matchLabels: org: empire c l a s s : d e a t h s t a r i n g r e s s : - fromEndpoints: - matchLabels: org: empire toPorts: - p o r t s : - port: "80" p r o t o c o l : TCP https://docs.cilium.io/en/stable/gettingstarted/http/ Joe Stringer Scaling container policy with eBPF Sep 11, 2019 14 / 29

Identity-based security Label selectors 12345 {org:empire, class:deathstar} 12468 {org:empire, class:tiefighter} 12465 {org:alliance, class:xwing} 12345 {org:empire, class:deathstar} matchLabels: org: empire 12468 {org:empire, class:tiefighter} matchLabels: 12345 {org:empire, class:deathstar} org: empire class: deathstar https://cilium.io/blog/2019/08/20/cilium-16 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 15 / 29

Identity-based security Datapath Configuration: Ingress Joe Stringer Scaling container policy with eBPF Sep 11, 2019 16 / 29

Identity-based security Datapath Configuration: Egress Joe Stringer Scaling container policy with eBPF Sep 11, 2019 17 / 29

Identity-based security Datapath Configuration: Egress Joe Stringer Scaling container policy with eBPF Sep 11, 2019 17 / 29

Layer 7 security Joe Stringer Scaling container policy with eBPF Sep 11, 2019 18 / 29

Layer 7 security L7 is the new L4 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 19 / 29

Layer 7 security Rejecting traffic in a protocol-aware manner Joe Stringer Scaling container policy with eBPF Sep 11, 2019 20 / 29

Layer 7 security Rejecting traffic in a protocol-aware manner Joe Stringer Scaling container policy with eBPF Sep 11, 2019 20 / 29

Layer 7 security Rejecting traffic in a protocol-aware manner Joe Stringer Scaling container policy with eBPF Sep 11, 2019 20 / 29

Layer 7 security Rejecting traffic in a protocol-aware manner cilium-agent –http-403-msg="..." Joe Stringer Scaling container policy with eBPF Sep 11, 2019 20 / 29

Layer 7 security Rejecting traffic in a protocol-aware manner cilium-agent –http-403-msg="..." Joe Stringer Scaling container policy with eBPF Sep 11, 2019 20 / 29

Layer 7 security Datapath Configuration: L3 flow Joe Stringer Scaling container policy with eBPF Sep 11, 2019 21 / 29

Layer 7 security Datapath Configuration: L7 flow Joe Stringer Scaling container policy with eBPF Sep 11, 2019 22 / 29

Layer 7 security Datapath Configuration: L7 flow Joe Stringer Scaling container policy with eBPF Sep 11, 2019 22 / 29

Layer 7 security L7 Configuration: Past Per-endpoint configuration A: 192.0.2.3:33333 ; B: 192.0.2.4:80 ; L: 192.0.2.1:12345 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 23 / 29

Layer 7 security L7 Configuration: Past A: 192.0.2.3:33333 ; B: 192.0.2.4:80 ; L: 192.0.2.1:12345 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 23 / 29

Layer 7 security L7 Configuration: Past A: 192.0.2.3:33333 ; B: 192.0.2.4:80 ; L: 192.0.2.1:12345 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 23 / 29

Layer 7 security L7 Configuration: Past A: 192.0.2.3:33333 ; B: 192.0.2.4:80 ; L: 192.0.2.1:12345 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 23 / 29

Layer 7 security L7 Configuration: Past A: 192.0.2.3:33333 ; B: 192.0.2.4:80 ; L: 192.0.2.1:12345 Joe Stringer Scaling container policy with eBPF Sep 11, 2019 23 / 29

Layer 7 security L7 Configuration: Present Joe Stringer Scaling container policy with eBPF Sep 11, 2019 24 / 29

Layer 7 security L7 Configuration: Present Joe Stringer Scaling container policy with eBPF Sep 11, 2019 24 / 29

Layer 7 security L7 Configuration: Present Joe Stringer Scaling container policy with eBPF Sep 11, 2019 24 / 29

Layer 7 security L7 Configuration: Present Joe Stringer Scaling container policy with eBPF Sep 11, 2019 24 / 29

Layer 7 security L7 Configuration: Present Joe Stringer Scaling container policy with eBPF Sep 11, 2019 24 / 29

Layer 7 security L7 Configuration: Proposal Joe Stringer Scaling container policy with eBPF Sep 11, 2019 25 / 29

Layer 7 security L7 Configuration: Proposal Joe Stringer Scaling container policy with eBPF Sep 11, 2019 25 / 29

Layer 7 security L7 Configuration: Socket redirect Joe Stringer Scaling container policy with eBPF Sep 11, 2019 26 / 29

Layer 7 security Socket assign: Hiccup BPF progs are attached at TC ingress skb_orphan() invoked directly before PREROUTING 1 https://www.mail-archive.com/netdev@vger.kernel.org/msg303851.html 2 https://www.mail-archive.com/netdev@vger.kernel.org/msg304057.html Joe Stringer Scaling container policy with eBPF Sep 11, 2019 27 / 29

Layer 7 security Socket assign: Hiccup BPF progs are attached at TC ingress skb_orphan() invoked directly before PREROUTING TC folks are already carrying hacks for this 1 1 https://www.mail-archive.com/netdev@vger.kernel.org/msg303851.html 2 https://www.mail-archive.com/netdev@vger.kernel.org/msg304057.html Joe Stringer Scaling container policy with eBPF Sep 11, 2019 27 / 29

Layer 7 security Socket assign: Hiccup BPF progs are attached at TC ingress skb_orphan() invoked directly before PREROUTING TC folks are already carrying hacks for this 1 Just move to ____dev_forward_skb() 2 ? 1 https://www.mail-archive.com/netdev@vger.kernel.org/msg303851.html 2 https://www.mail-archive.com/netdev@vger.kernel.org/msg304057.html Joe Stringer Scaling container policy with eBPF Sep 11, 2019 27 / 29

Layer 7 security Summary Minimize processing cost Number of events Cost for each event Separate concerns: Policy vs addressing Frontload expensive operations ... while keeping runtime costs low Joe Stringer Scaling container policy with eBPF Sep 11, 2019 28 / 29

Thank you More information https://cilium.io https://cilium.io/slack https://github.com/cilium/cilium https://twitter.com/ciliumproject Joe Stringer Scaling container policy with eBPF Sep 11, 2019 29 / 29

Thank you More information https://cilium.io https://cilium.io/slack https://github.com/cilium/cilium https://twitter.com/ciliumproject Joe Stringer Scaling container policy with eBPF Sep 11, 2019 29 / 29

Socket redirect Joe Stringer Scaling container policy with eBPF Sep 11, 2019 30 / 29