Jaal: Towards Network Intrusion Detection at ISP Scale A. Aqil, K. Khalil , A. Atya, E. Paplexakis, University of California S. Krishnamurthy, KK. Ramakrishnan Riverside T. Jaeger Penn State University P. Yu, A. Swami US Army Research Lab 1
Is IDS Needed at ISP Scale? Increasing number of network attacks Distributed, span entire WANs Unnoticed until too late Mirai botnet Mirai exploits vulnerable devices spread across the internet to launch DDoS Sep 2016: Krebs on Security (620 Gbit/s), OVH (1Tbit/s) Oct 2016: multiple attacks on Dyn, affected Twitter, Github, Airbnb, Netflix, others Nov 2016: Liberia’s internet infrastructure 2
Is IDS Needed at ISP Scale? Simple two step attack: scan then flood Hardcoded default passwords control vulnerable devices (scanning a large set of IP addresses) Compromised devices also repeat the scan Launch coordinated attack on targets at the bot master signal Inherently difficult to detect Scanning activity observable only at ISP level “DDoS prevention works best deep in the network, where the pipes are the largest and the capability to identify and block the attacks is the most evident” Bruce Schneier, security expert 3
ISP Scale IDS is Challenging State of the art NIDS (e.g., Snort, Bro) are effective But expect to inspect all packets Works only at enterprise scale Problematic at ISP scale: Multiple ingress/egress points To create global view required for analysis, information collected from multiple vantage points needs to be aggregated Challenge: how to aggregate? 4
Aggregation Approach I Copy and forward to central engine Simple, but lead to performance degradation Performance degradation as traffic replication increases 100 Avg decrease in throughput Percentage decrease Worst decrease in throughput Drop in accuracy 50 70% Tput loss 0 0 10 20 30 40 50 60 70 80 90 100 Percentage of traffic replicated 5
Aggregation Approach II Sample and forward to central engine Already used by ISPs for heavy-hitter detection Efficient but achieves poor detection accuracy for general attacks Attack Reservoir Sampling Distributed SYN Flood 54% Sock Stress 60% SSH Brute Force 42% 6
Aggregation Approach III Create sketches and forward to central engine Targeted measurement approach Strong resource/accuracy guarantees Lacks generality: need one sketch for every measurement task For TCP/IP header, need 2 18 different sketches to capture all possible measurements 7
Jaal Design Goals Design an ISP-scale NIDS that: Can detect wide array of attacks requiring global view, using signatures similar to Snort’s Focus on TCP/IP header-based attacks Does not require copying and forwarding raw packets (minimizes bandwidth overhead) 8
Jaal Overview III- Flow assignment: • Assigns flows to monitors • Load balancing en source Load Load Flow Assign. Info. Info. olumes [ 32 , 42 ]. I- Monitors: eater than • Filter target flows Assignments egularly; this Assignments Packet Filtering Packet Filtering • Process packet Summatization Summatization (can result batches, create only way Monitor summaries Monitor clusters for peak Summaries Summaries Inference that, a large II- Inference engine: NIDS are inade- Decision • Collects summaries Rules to copying • Performs pattern matching 9
Summarization Goal: produce a representative summary of packets Enables high accuracy detection of attacks using general signature Light weight, low BW overhead all flows all flows packets batch batch assigned summary flows packets mode 10 fields mode
Summarization (cont.) Two step summarization osed to SVD to reduce fields mode X = U Σ V T , ¯ Clustering to reduce packets mode eliminate small singular values batch packets summary mode SVD k-means k n centroids counts fields p r mode 11
Inference Summary Individual S m counter- 1 or S m S a 2 Config. Aggregator Config. summaries τ d , τ c h, τ v to question vectors d. q Q NIDS Similarity Translator ¯ Postprocessor Rules X Estimator Inference Engine Alert Alert, Feedback Q (3) Collect individual summaries (push or pull) Transform NIDS rules (normalization, marking irrelevant fields) Estimate similarity Feedback: request finer grained summary to improve performance Estimate variance (e.g. port scans, DDoS) 12
Flow Assignment Requirements: Cover all flows Each flow is processed by exactly one monitor (for correct operation) Balance load to the extent possible Simple/Fast algorithm Challenge: Flows can start/terminate at any time, vary in packet rate Packet rates unknown a priori Solution: Model as constrained online load balancing problem Simple greedy algorithm, (empirically) close to optimal 13
Evaluation Implemented on in-house high performance SDN-testbed Two Realistic RocketFuel topologies (~350 routers) Complex topologies created by instantiating Open vSwitches instances connected via virtual links Two ISP backbone traces from MAWI group as background traffic + inject malicious traffic Five different attacks: most DoS: SYN flood common DDoS: distributed SYN flood attack Port scans: distributed port scans classes Brute forcing: distributed SSH brute forcing Sockstress 14
Evaluation (cont.) 98% average TPR @ 9% FPR and only 35% BW overhead (with feedback) Summarization parameters n , k , r set by studying ROC curves Each point has a different n: batch size, r: rank, k: centroids r = 14 retains most information in fields mode n ≥ 600, k ≥ 0.2n, r ≥ 12 enables high detection accuracy 15
Evaluation (cont.) Simulating Mirai progression Scanning on ports 23, 2323 Randomly select a source node + 150 vulnerable nodes Jaal detects the scan with 95% accuracy Number of Infected devices vs time Unchecked infections 150 Number of infected devices Remaining infected devices after Jaal 100 50 0 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Time (s) 16
Conclusion ISP scale NIDS is needed in the face of large scale attacks State of the art NIDS inadequate at ISP scale Jaal presents a major step forward in developing ISP scale NIDS Uses dimensionality reduction and clustering Centralized pattern matching on packet summaries Achieves high detection accuracy at low bandwidth overhead 17
Thanks! 18
Recommend
More recommend
