FOSDEM’17 Brussels, 2017-02-04 Stateful packet processing with eBPF: An implementation of OpenState interface Quentin Monnet < quentin.monnet@6wind.com > @qeole
Quentin Monnet (6WIND) | BEBA — OpenState and eBPF Agenda 2/34 I’ll speak at FOSDEM Will talk about: ⋅ OpenState, 40% ⋅ eBPF, 60% Me
Quentin Monnet (6WIND) | BEBA — OpenState and eBPF Agenda 3/34 Coming too! I’ll speak at FOSDEM I’ll have a talk Will talk about: on eBPF! ⋅ OpenState, 40% ⋅ eBPF, 60% Daniel B. Me
Quentin Monnet (6WIND) | BEBA — OpenState and eBPF Agenda 4/34 Coming too! I’ll speak at FOSDEM I’ll have a talk Will talk about: on eBPF! ⋅ OpenState, 40% 70% ⋅ eBPF, 60% 30% Daniel B. Me
Quentin Monnet (6WIND) | BEBA — OpenState and eBPF SDN: hosts, VMs, programmable switches, controllers… 5/34 Controller Internet Switch Router Hypervisor
Most packets goes through the “shortcut” dataplane Two paths for dataplane Some packets are sent as exceptions—this generally includes stateful processing Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 6/34 SDN controller Switch Standard path Shortcut
Can we make the switch “smarter”, without loosing SDN benefits? What about: bringing back some control into the switch? How could we abstract stateful packet processing, in such a way the controller can easily set up the switches? Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 7/34 SDN controller Switch Standard path OpenState path
Objectives: Wire-speed-reactive control/processing tasks inside the switches Centralized control Scalability Platform-independent From January 2015 to March 2017 (27 months) Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 8/34 Horizon 2020 More info at http://www.beba-project.eu/
BEBA: Who? Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 9/34
BEBA switch Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 10/34 Open Packet Processor BEBA (registers and boolean conditions) switch OpenState InSP (stateful processing) (packet generation)
OpenState: stateful packet processing Forwarding depends on traffic previously observed 1 Lookup for flow state 2 Lookup for action associated to flow state, perform action 3 Update state to new value So we need two tables: a state table and a table for actions: XFSM table (eXtended Finite State Machine) Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 11/34 State update Packet pattern State Action State table XFSM table
Case study: port knocking Clients see port 22 of the server as closed To access port 22, they first have to send a secret packet sequence to that port Our example secret sequence: UDP packet on port 1111, 2222, 3333 then 4444 Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 12/34 10.2.2.2 10.1.1.1 Clients Switch Server 10.3.3.3
Case study: port knocking Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 13/34 Initial state Any other packet; UDP or timeout packet on port 1111 Step 1 Step 2 Step 3 UDP UDP packet packet on port 2222 on port 3333 UDP packet on port 4444 All TCP packets to Connection on port 22 are forwarded. TCP port 22 Packets to other is open ports are dropped.
State table Tracks current state for each flow Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 14/34 Flow matching pattern State … … IP src = any DEFAULT Packet pattern State State table
XFSM table To state and “event” pattern, associates action and “next state” Quentin Monnet (6WIND) | BEBA — OpenState and eBPF “Next state” is used to update the entry for this flow in the state table 15/34 Flow matching pattern Actions State Event Action Next state … … … … DEFAULT UDP dst port = 1111 Drop STEP_1 STEP_1 UDP dst port = 2222 Drop STEP_2 STEP_2 UDP dst port = 3333 Drop STEP_3 STEP_3 UDP dst port = 4444 Drop OPEN OPEN TCP dst port 22 Forward OPEN OPEN Port = * Drop OPEN … … … … * Port = * Drop DEFAULT State Action XFSM table
State table update The state of the flow is updated for each packet, thus unrolling the Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 16/34 port knocking sequence Flow matching pattern State … … IP src = 10.3.3.3, IP dst = 10.1.1.1 STEP_1 … … IP src = any DEFAULT State update State table XFSM table
Can we implement that with eBPF? Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 17/34
eBPF ~ extended Berkeley Packet Filter Assembly-like language, based on cBPF (packet filtering) Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 18/34 Programs come from user space, run in the kernel C source code ELF-compiled BPF bpf_prog.c bpf_prog.o tc LLVM/clang User program Userspace bpf() syscall Kernel cls_bpf Maps JIT Network stack Packets tc ingress tc egress Net device Net device
Stateful eBPF Default behavior: program is run to process a packet, no state preserved on exit However: eBPF Maps (kernel 3.18+): helpers → Let’s use hash maps for OpenState tables! ( https://github.com/qmonnet/pkpoc-bpf ) Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 19/34 • Memory area accessible from eBPF program through specific kernel • Arrays, hash maps (and several other kinds) • Persistent across multiple runs of an eBPF program • Can be shared with other eBPF programs • Can be shared with userspace applications
openstate.h state; Quentin Monnet (6WIND) | BEBA — OpenState and eBPF }; int32_t next_state; int32_t action; struct XFSMTableVal { }; uint16_t dst_port; uint16_t src_port; l4_proto; uint8_t int32_t /* State table */ struct XFSMTableKey { /* XFSM table */ }; int32_t state; struct StateTableVal { }; uint32_t ip_dst; uint32_t ip_src; uint16_t ether_type; struct StateTableKey { 20/34
portknocking.c : State table lookup if (state_val) { Quentin Monnet (6WIND) | BEBA — OpenState and eBPF goto end_of_program; } goto xfsmlookup; */ * for current event. /* If we found a state, go on and search XFSM table for this state and current_state = state_val->state; struct StateTableVal *state_val = map_lookup_elem(&state_table, &state_idx); /* [Truncated] /* State table lookup */ state_idx.ip_dst = ntohl(ip->dst); state_idx.ip_src = ntohl(ip->src); struct StateTableKey state_idx; state_idx.ether_type = ntohs(ethernet->type); */ * ports. * addresses; since we will need it at next step, also extract UDP src and dst * Parse headers and make sure we have an IP packet, extract src and dst 21/34
portknocking.c : XFSM table lookup, state table update, action /* Return action code */ Quentin Monnet (6WIND) | BEBA — OpenState and eBPF } } return TC_ACT_UNSPEC; default : return TC_ACT_OK; case ACTION_FORWARD: return TC_ACT_SHOT; case ACTION_DROP: switch (xfsm_val->action) { map_update_elem(&state_table, &state_idx, &new_state, BPF_ANY); /* Set up the key */ struct StateTableVal new_state = { xfsm_val->next_state }; /* Update state table entry with new state value */ if (xfsm_val) { struct XFSMTableVal *xfsm_val = map_lookup_elem(&xfsm_table, &xfsm_idx); /* Lookup */ xfsm_idx.dst_port = dst_port; xfsm_idx.l4_proto = ip->next_protocol; = current_state; xfsm_idx.state 22/34 xfsm_idx.src_port = 0;
Compile and run One would compile the complete program into eBPF with: $ clang -O2 -emit-llvm -c openstate.c -o - | \ llc -march=bpf -filetype=obj -o openstate.o … and attach it with, for example: # tc qdisc add dev eth0 clsact # tc filter add dev eth0 ingress bpf da obj openstate.o One more thing: initialize the maps (user-space program with bpf() syscall) Alternative method: bcc’s Python wrappers provide an easier way to initialize maps, to compile and to inject programs Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 23/34
Result Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 24/34
Second case study: token bucket Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 25/34
Token bucket algorithm Quentin Monnet (6WIND) | BEBA — OpenState and eBPF 26/34 Bucket capacity: 4 tokens Packet Token generation: 1/Q Q W T0 Case 1 (normal traf fi c) time Forward Tmin Tmax W_next Packet W Case 2 (light traf fi c) time Forward Tmin Tmax W_next Packet W Case 3 (heavy load) time Drop Tmin Tmax W_next
Model side: Open Packet Processor Extension of OpenState: Quentin Monnet (6WIND) | BEBA — OpenState and eBPF ( https://github.com/qmonnet/tbpoc-bpf ) → Case 3 → Case 1 cond2 = (t ≤ Tmax) conditions: For token bucket: registers for Tmin and Tmax , then evaluate 27/34 • With global and per-flow registers • Registers evaluated with a set of conditions • XFSM table lookup must also match on conditions • cond1 = (t ≥ Tmin); • cond1 == true && cond2 == true • cond1 == true && cond2 == false → Case 2 • cond1 == false
Recommend
More recommend
