SF-TAP: Scalable and Flexible Traffic Analysis Platform Running on - PowerPoint PPT Presentation

SF-TAP: Scalable and Flexible Traffic Analysis Platform Running on Commodity Hardware Yuuki Takano, Ryosuke Miura, Shingo Yasuda Kunio Akashi, Tomoya Inoue 1 in USENIX LISA 2015 NICT, JAIST (Japan)

Table of Contents 2 1. Motivation 2. Related Work 3. Design of SF-TAP 4. Implementation of SF-TAP 5. Performance Evaluation 6. Conclusion

Motivation (1) as Python, Ruby, C++, for many purposes (IDS/ IPS, forensic, machine learning). reconstruction (quite complex). 3 • Programmable application level traffic analyzer • We want … • to write traffic analyzers in any languages such • **not** to write codes handling TCP stream • modularity for many application protocols.

4 Motivation (2) • High speed application level traffic analyzer • We want … • to handle high bandwidth traffic. • to handle high connections per second. • horizontal and CPU core scalable analyzer.

Motivation (3) 5 • Running on Commodity Hardware • We want … • open source software. • not to use expensive appliances.

Related Work libprotoident SF-TAP (application traffic detector) (flow oriented analyzer) (low level traffic capture) l7-filter nDPI libnids 6 SCAP [IMC 2012] GASPP [USENIX ATC 2014] pcap DPDK netmap [USENIX ATC 2012] BPF [USENIX ATC 1993] + modularity and scalability

High-level Architecture Core Scaling SF-TAP Cell SF-TAP Cell SF-TAP Cell SF-TAP Cell Intra Network Core Scaling Core Scaling Core Scaling Horizontal Scaling Cell Incubator Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer Analyzer 10GbE The Internet Flow Abstractor of SF-TAP CPU 7 CPU CPU CPU CPU Flow Abstractor CPU CPU CPU CPU Flow Abstractor CPU CPU CPU CPU Flow Abstractor CPU CPU CPU 10GbE

8 Design Principle (1) • Flow Abstraction • abstract flows by application level protocols • provide flow abstraction interfaces like /dev, /proc or BPF • for multiple programming languages • Modular Architecture • separate analyzing and capturing logic • easily replace analyzing logic

Design Principle (2) computer resources be core scalable for efficiency 9 • Horizontal Scalable • analyzing logic tends to require many • volume effect should solve the problem • CPU Core Scalable • both analyzing and capturing logic should

Design of SF-TAP (1) Packet Forwarder Separator Plane separated traffic SF-TAP Cell L3/L7 Sniffer SSL Proxy etc... other SF-TAP cells IP Packet Defragmenter L2 Bridge mirroring traffic IP Fragment Flow Handler defined 4 planes Analyzer Plane application level analyzers Forensic, IDS/IPS, etc… Abstractor Plane flow abstraction Separator Plane flow separation Capturer Plane traffic capturing (ordinary tech.) (users of SF-TAP implements here) (we implemented) Separator Identifier 10 L7 Loopback I/F NW I/F HTTP I/F TLS I/F Flow Abstractor Flow Classifier TLS Analyzer HTTP Analyzer HTTP Proxy TCP and UDP Handler filter and classifier rule DB Flow Forensic IDS/IPS etc... Application Protocol Analyzer etc... TCP Default I/F UDP Default I/F Analyzer Plane Abstractor Plane Capturer Plane SF-TAP Cell Incubator (we implemented)

Design of SF-TAP (2) IP Fragment Flow Separator handle fragmented packets IP Fragment Handler layer 2 frame capture layer 2 bridge Packet Forwarder Handler Packet Forwarder SF-TAP Cell Incubator L2 Bridge other SF-TAP cells separated traffic Separator Flow Incubator SF-TAP Cell 11 separate flows to multiple Ifs

Design of SF-TAP (3) TCP Default I/F nothing to do for UDP identify flows by IP and port reconstruct TCP flows IP Packet Defragmenter Flow Identifier TCP and UDP Handler Defragmenter IP Packet Identifier Flow UDP Default I/F L7 Loopback I/F SF-TAP Flow Abstractor rule classifier filter and Handler TCP and UDP Classifier Flow Flow Abstractor TLS I/F HTTP I/F NW I/F 12 defragment IP packets if needed

Design of SF-TAP (4) rule regular expressions classify flows by Flow Classifier Defragmenter IP Packet Identifier Flow UDP Default I/F TCP Default I/F L7 Loopback I/F classifier SF-TAP Flow Abstractor filter and Handler TCP and UDP Classifier Flow Flow Abstractor TLS I/F HTTP I/F NW I/F 13 output to abstraction IFs

14 Implementation (updated from the paper) • SF-TAP cell incubator • C++11 • it uses netmap, available on FreeBSD • SF-TAP flow abstractor • C++11 • it uses pcap or netmap • available on Linux, *BSD, and MacOS • Source Code • https://github.com/SF-TAP • License • 3-clauses BSD

Performance Evaluation (1) 15 packet drop against connections per second (pcap) 4K 10K 50K

Performance Evaluation (1) (2) α->β β α to β and γ to β to γ cell incubator γ β α 10 GbE x 2 1 GbE x 12 (3) α->γ (3) α->β (1) α->γ 16 ideal 1024 512 256 128 64 fragment size (bytes) 16 12 8 4 0 Mpps forwarding performance of SF-TAP cell incubator γ

Other Features protocol analysers incubator 17 • L7 Loopback interface for encapsulated flows • Load balancing mechanism for application • Separating and mirroring modes of SF-TAP cell • See more detains in our paper

Conclusion analysis. in our experiments. 18 • We proposed SF-TAP for application level traffic • SF-TAP has following features. • flow abstraction • running on commodity hardware • modularity • scalability • We showed SF-TAP has achieved high performance