A Framework for Rule Processing in Reconfigurable Network Systems - PDF document

A Framework for Rule Processing in Reconfigurable Network Systems Michael Attig and John Lockwood Washington University in Saint Louis Applied Research Laboratory Department of Computer Science and Engineering May 1, 2005 Rule Processing Framework – FCCM 2005 1 Outline • Overview • Background • Architecture • Results • Summary Rule Processing Framework – FCCM 2005 2

Rule Processing Overview • Rule processing & intrusion detection • TCP Flow Processing • Header Processing • Payload Scanning alert tcp any 110 � any any (msg:"Virus - Possible MyRomeo Worm"; flow:established; content:"myromeo.exe"; nocase; classtype:misc- activity; sid:723; rev:6;) • Snort Rules (version 2.2 Sept 2004) – 2464 Rules – 292 Headers – 2107 Signatures – 233 Regular Expressions Rule Processing Framework – FCCM 2005 3 Rule Characteristics • 2464 unique rules Unique Signature Distribution • 292 unique header rules 180 160 – 168 are “header-only” Number of Occurrences 140 • 2107 unique signatures 120 100 – 97% less than 32 bytes 80 60 – Spread across 2296 of rules 40 • 233 regular expressions 20 0 – Snort rules always contain 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 Signature Length (bytes) static signature also Unique Signature Occurences • Few signatures associated 160 with many rules 140 Number of occurrences 120 – 83% found in single rule 100 – Only 18 associated with 80 more than 10 rules 60 40 • 10 header rules can match at 20 once (pessimistic) 0 1 151 301 451 601 751 901 1051 1201 1351 1501 1651 1801 1951 2101 signature number Snort version 2.2 (Sept 2004) Rule Processing Framework – FCCM 2005 4

Fully Functional Rule Processing String Matching Comparators [Sourdis fccm’04] ? Partitioning [Baker fccm’04] TCAMs [Yu hoti’04] ? ? Bloom Filters [Attig fccm’04] Complete Rule ? DFAs [Moscola fccm’03] Processing Header Classification BV-TCAM [Song fpga’05] NFAs [Clark fccm’04] ? ? TCP Flow Rule Processor Reconstruction [Schuehler fpl’04] Pipelining [Cho fccm’04] Rule Processing Framework – FCCM 2005 5 Rule Processing Framework Overview TCP/IP Data Local Order TCP flows Local Internet Area Internet Area Network for inspection by Network payload scanner(s) Intrusion Detection System h header modules determine matching Local Area Internet header rules Local Area Internet Network Matching header Network and string IDs sent to rule p payload modules search for static processor using signatures and standard interface Intrusion Detection regular expressions System Rule processor uses Focus of this header and payload Talk match criteria to determine rule matches Rule Processing Framework – FCCM 2005 6

Rule Processor FPGA (1) (2) (3) (5) (6) (7) Software Communication Wrapper Matching Criteria Communication Wrapper Rule Processing Framework – FCCM 2005 7 Example R1: Alert tcp any 80 � any 125 (content:“string1”; content:“string2”;) Algorithmically: R1: H1 Λ C1 Λ C2 Rule Processing Framework – FCCM 2005 8

Rule Processing Example TCP/IP Data flow H1 C1 C2 C1 Rule Processing Framework – FCCM 2005 9 Rule Processor FPGA (1) (2) (3) (5) (6) (7) Software Communication Wrapper Rule X Header flow H1 H1 R1 R1 H1 C1 C2 Headers +1 C1 Reset Matching Criteria Communication Wrapper Rule Processing Framework – FCCM 2005 10

Implementation Environment • Xilinx Virtex 2000E FPGA – 12% LUTs – 25% Slices – 93% of Block RAMs – 80.6 MHz • Stacked configuration allows chaining processing Rule Processing Framework – FCCM 2005 11 Worst Case Throughput • Worst case when signature associated with many rules detected • Only 18 signatures in more than 10 rules • Worst case signature – |00 00 00 00| in 135 rules Framework Throughput • Scenario: 3000 – Back-to-back 44 byte TCP packets 2500 from different Throughput (Mbps) 2000 flows and |00 00 00 00| as 1500 payload 1000 – Worst case assumes Worst-case 7 million attack 500 Average Case packets per second 0 0 250 500 750 1000 1250 1500 Packet Size (Bytes) Rule Processing Framework – FCCM 2005 12

Intrusion Detection of WashU’s Backbone Network Matching 12+ Byte Signatures Matching 4 Byte Signatures • Observe ~10,000 total string matches per second on WashU’s backbone network (~250-300 Mbps) • Scaling to 2.5 Gbps, only ~100,000 string matches per second Rule Processing Framework – FCCM 2005 13 Next Generation FPGA Projections • More block RAM • Faster place & route • Parallel copies of pipeline – Multiple IDs per clock cycle • QDR SRAMs Rule Processor Relative Improvement 7 • 6x improvement 6 to throughput 5 VirtexE 4 Virtex2 3 Virtex4 2 1 0 Rule Frequency Throughput Rule Processing Framework – FCCM 2005 14

Related Work Throughput Function Group and Component Device Logic Cells (Gbps) Flow GaTech Stream Assembler Virtex 1000 876 (10%) 3.2 Monitoring Northwestern U. Flow Monitor Virtex2-8000 - 48.3 WashU TCP Processor Virtex4 140 22,100 (35%) 10.3 Header Processing WashU BV-TCAM Virtex4 100 4,200 (10%) 10 Payload Crete Pre-decoded CAMs Virtex2-6000 64,268 (95%) 9.7 Scanning GaTech Decoder Trees Virtex2-8000 54,890 (81%) 7 Tokyo Trie-based Hash Virtex2-6000 2,365 (7%) 10 UCLA Packet Filters Spartan 3 2000 15,202 (37%) 3.2 USC Partitioning Virtex2 Pro 15,010 (15%) 4.5 WashU Bloom Filters Virtex4 100 35,850 (85%) 20.4 Correlation WashU Rule Processor Virtex4 100 40,200 (95%) 15.9 Rule Processing Framework – FCCM 2005 15 Contributions • Development of large-scale Rule Processing Framework – Bridge between component processing and rule processing – Supports up to 32,768 rules • Rule processing framework capable of 2.5 Gbps throughput on FPX – Projected to 15.9 on latest Virtex 4 • Rule processor operated on TCP flows – Context information stored for over 2 million simultaneous flows Rule Processing Framework – FCCM 2005 16

Acknowledgments • Research Sponsors – Global Velocity – Boeing • ARL Faculty & Students http://arl.wustl.edu/projects/fpx/reconfig.htm Rule Processing Framework – FCCM 2005 17 Questions? Rule Processing Framework – FCCM 2005 18

Communication Wrapper Interface Between Devices Between Software/Hardware Rule Processing Framework – FCCM 2005 19 Example R1: Alert tcp any 80 � any 125 (content:“string1”; content:“string2”;) R2: Alert tcp any 8080 � any 1024 (content:“string1”;) R1: H1 Λ C1 Λ C2 R2: H2 Λ C1 Rule Processing Framework – FCCM 2005 20

Rule Processor FPGA Communication Wrapper X X CIDs RIDs CIDs BRAM H2 H1 H1 R1 R1 R2 H1 C1 +1 C2 C1 Reset Communication Wrapper Rule Processing Framework – FCCM 2005 21 Adding Modules • Accept and act upon IP packets • Communicate match criteria using communication wrapper interface – Provide deterministic interfaces – Abstract transport protocol • Software Configuration using communication wrapper • Represent matching criteria as ID numbers • Allows combination of techniques – Take advantage of best characteristics • General classifiers vs. field-specific headers • Static strings vs. regular expressions Rule Processing Framework – FCCM 2005 22

Evaluation • Recall Rule IDs are inserted into pipeline based on matching signatures 80 Million Throughput versus Rule Look-ups per Second rule IDs per second 3000 2500 Throughput (Mbps) 2000 1500 1000 500 0 1.E+00 1.E+02 1.E+04 1.E+06 1.E+08 Rule Look-ups per Second Rule Processing Framework – FCCM 2005 23 Additional Rules Supported • Virtex 2 – 120 of 144 Block RAMs (18 Kbits each) – 2 copies of pipeline • 10 BRAM in stage 2 • 10 BRAM in stage 5 • 40 BRAM in stage 6 – 184,320 rules supported • Virtex 4 – 216 of 240 Block RAMs (18 Kbits each) – 4 copies of pipeline • 9 BRAM in stage 2 • 9 BRAM in stage 5 • 36 BRAM in stage 6 – 165,888 rules supported Rule Processing Framework – FCCM 2005 24