A networked-FPGA platform o ff ering fm exible Ethernet switching from Layer 1 all the way to full SDN via P4 Matthew Knight and Marc Durrenberger Networked FPGA platforms Opening up network packet visibility Increasing network flexibility
Who we are … GLOBAL HEADQUARTERED OPERATIONS FOUNDED IN 2013 SYDNEY Clients Globally Patents & Trademarks Major Verticals 100+ Protected by Financial Services Data Centers patented technology Defense & Government Telecoms Gaming & Media
What we do… Leverage Build Develop FPGA Switching Network Technology Hardware Applications AND AND AND • Layer 1 switches – patching , broadcast and tapping • Low latency order-entry -- Muxing • Nanosecond-precision aggregation tap • FPGA platform for application development
Agenda • An overview of Layer 1 switching • Metamako device architecture and telemetry • Software-defined networking • The network data plane - now programmable • A high-level overview of P4 packet processing directives • The Metamako P4/SDNet sample application • The SDNet data plane implementation • SDNet user-engines and scalability • Summary
An overview of Layer 1 switching • Physical layer switching acting at the Physical Medium Dependent Sublayer (PMD) • Switching function has no knowledge of byte encoding, framing etc. • Why is it useful? • Allows ports to be replicated - tapping, port redistribution • Extremely fast (5 ns port-to-port) and low-jitter • Can perform media conversion e.g.: • Fibre to copper • multi-mode to single-mode fibre • multi/single-mode/copper to DWDM - no transceiver vendor-locking • In Metamako devices, full Ethernet signal and clock regeneration on every port
Metamako device architecture • Integrated Layer 1+ switch • 1 or 3 Xilinx UltraScale™ or Ultrascale+™ FPGAs in 1 or 2 RU x2 MMP FPGA FPGA • FPGA Development Kit available • 48 ports per RU x4 MMP x4 MMP • x86_64 management processor FPGA • Runs Linux x14 10GbE x14 10GbE • Leverages the InfluxData open-source TICK Stack for x56 10GbE telemetry Layer 1 • L1+ switch ports status/statistics CPU Switch • Ethernet-connected to L1 switch fabric • Direct PCIe connections to FPGA(s) 1 or 2 RU Chassis • Toolchain can generate rpm which loads FPGA, drivers, apps
Telemetry: Actionable real-time counters and alerting Linux e.g. SDNet cpu mem net custom control plane Precon fj gured on every device disk swap … custom data plane • Real-time streaming counters from: • Linux • L1+ Port Counters Telegraf • Easily extensible to custom SDNet implementation Kapacitor InfluxDB Feeding local In fm uxData Stack • Telegraf - collection agent Layer 1+ • InfluxDB - time-series database Port Statistics • Kapacitor - alerting engine Accessible in real-time via Web Apps e.g. • Grafana • Chronograf
Software-defined networking What is SDN? Control Plane Control Plane Control Plane •The separation of the network control plane from the data plane and allowing the Data Plane Data Plane Data Plane control plane to serve more than one device What are the key bene fj ts? Control Plane •Abstraction of the data plane implementation e.g. OpenFlow, VMware NSX, •Programmatic dynamic control and Cisco ACI management of the network via a single interface Data Plane
The network data plane What does it do? • Moves packets around networks • Logic usually implemented in an ASIC • Knows how to parse standard network protocols and act upon fields within them via tables usually controlled by the control plane How does this fj t in to SDN? • As long as the ASIC supports all the required functions within the SDN control plane and can interface with it, all is good How does new networking functionality become available? • Often driven by one or more of the ASIC/switch vendors • ASIC design-to-market becoming more flexible but release cycles are still measured in years
The network data plane - now programmable P4 • Network-specific programming language allowing network data plane functionality to be defined • Open-source standard maintained by the P4 Language Consortium • Complete flexibility in defining packet protocols e.g. Software-defined Software-defined Control Plane Control Plane • Support a completely new type of overlay network • Packet headers can be rewritten on-the-fly e.g. OpenFlow, Existing or VMware NSX, Custom API Xilinx SDNet Cisco ACI • The SDNet language is conceptually similar to P4 and has the same benefits Software-defined Fixed Data Plane Data Plane • SDNet is far more than a functional specification as it also provides a compiler to translate specifications into working FPGA programmable logic • Furthermore, SDNet provides a P4 to SDNet translator
A high-level overview of P4 packet processing directives Parser • Receives incoming packets Arbiter • Follows rules telling it how to parse multiple levels of packet header • Parsed fields written to a struct which is passed to the next block Match-action pipeline Parser • Permits fields and derived fields from the parsed headers to be used to make packet forwarding decisions • Provides access to lookup tables Match-action pipeline • Tables can be: • exact, ternary or lpm (longest-prefix match) Deparser • Allows information to be sent to the control plane • Fields in headers may be rewritten Deparser Demux • Prepends headers to the packet data • Headers may be unchanged, changed or entirely new
The Metamako P4/SDNet sample application Based on the P4/NetFPGA project • 8 port reference design 10GbE 10GbE x16 The data plane • 16 10GbE interfaces connected to physical FPGA transceivers in-turn, connected to the Layer 1 switch SDNet Xilinx FPGA The control plane API • 1 virtual interface connected via PCIe and SDNet PCI Express drivers to the management processor Management Processor • Data is exchanged via raw frames across this Virtual Ethernet interface interface for control plane • Python and C++ APIs available on the management processor
The SDNet data plane implementation Packet ingress 10GbE 10GbE x16 • Packets from each 10GbE port are muxed into the Out Out packet bus connected to the SDNet Block The packet bus Output Management Demux • The SDNet block receives and (optionally) forwards Processor Packet bus packets over a single packet bus PCI Express Virtual Ethernet interface for SDNet • The width is configurable up to 1 024 bits control plane Block • The Metamako example SDNet implementation Packet bus uses a 256 bit packet bus Input Mux Packet egress • Packets are consumed from the packet bus leaving 10GbE 10GbE the SDNet block and demuxed to the relevant x16 In In 10GbE port(s)
The control plane 10GbE 10GbE Per-packet metadata x16 Out Out • The P4/SDNet functional specification includes the ability to define per-packet metadata Output Management • This metadata accompanies the packet through the Demux Processor SDNet block Packet bus PCI Express Virtual Ethernet • It can be logically split into: interface for SDNet Control Bus control plane Block • The metadata required by the mux/SDNet/demux Packet bus pipeline to process the packet e.g. the input port and Input following a lookup, the output port(s) Mux • Any set of fields that can be populated with data or state e.g. key not found in table destined for the control plane 10GbE 10GbE x16 • Implemented in SDNet as a separate bus In In
SDNet user engines and scalability The Concept 10GbE 10GbE x16 • User engines are essentially custom RTL modules Out Out that interface with the SDNet block and can be inserted in the packet processing pipeline Output Management Demux • They allow the implementor to perform custom User Processor Engine Packet bus processing of every frame e.g. PCI Express Virtual Ethernet Core interface for SDNet • Encryption/decryption control plane Block User • Deep packet inspection Engine Packet bus • Tra ffi c Shaping/Quality of Service Core Input Mux P4 Support 10GbE 10GbE • The P4 language supports extern objects which x16 In In Xilinx’s translator maps to SDNet user engines
SDNet user engines and scalability Raw throughput 10GbE 10GbE x64 • Heavily dependent upon the design meeting timing Out Out • With a relatively straightforward processing pipeline AXI running on a large UltraScale+™ FPGA, somewhere Crossbar between 0.5 and 1 Tbps could well be possible Packet busses SDNet SDNet SDNet SDNet Fundamental Trade-o ff s Block Block Block Block • An FPGA is typically clocked an order of magnitude lower than an ASIC Packet busses AXI • The latest and greatest switching ASICs are apparently Crossbar capable of 12.8 Tbps • No ASIC today provides the programability of a P4/ 10GbE 10GbE x64 SDNet impementation In In
Summary • Software-defined networking is here to stay • Though SDN introduces a programmable network control plane, the data plane is still largely defined by ASIC functionality • With P4 and SDNet, a programmable switching data plane is now possible leveraging FPGA programmable logic • Metamako devices provide a powerful, production-ready platform for switching • Metamako has built and is demonstrating a sample P4/SDNet switching application
Recommend
More recommend
