Application-Integrated Data Collection for Security Monitoring Magnus Almgren and Ulf Lindqvist System Design Lab, SRI International The work described here was funded by DARPA under contract number F30602-99-C-0149 and F30602-98-C-0059. The views herein are those of the authors and do not necessarily reflect the views of the supporting agency.
Introduction How many network IDSs can detect this? How many network IDSs can detect this? 2
Outline Part One Traditional NIDS and HIDS Host-located data collection Application-integrated module Proof-of-concept implementation Implementation architecture and data flow Part Two 3
Traditional Network IDS Problem areas: Advantages: Encrypted traffic Passive, non-invasive Evasion tricks Hidden Network speed Can monitor multiple hosts from one Session/transaction location reconstruction and statefulness Timely preemption difficult 4
Traditional Host IDS Data sources Audit data – useful, but limited insight into application data Application/system log files – limited content, disk space management Usually, data produced after the fact Blind to most network-level attacks 5
Host-located Real-time Event Data Collection on Multiple Levels Application Application-integrated IDS OS IDS analyzing audit trail (system calls) Network Network IDS (located on network or host) Security violations manifest themselves differently on different levels The data sources are complementary 6
Application-integrated Module Advantages Disadvantages Unencrypted Tailored for a specific information application Independent of network Invasive: Could impact speed application performance and stability Detailed information available True session and � Complements data transaction decoding collection on other and reconstruction levels Opportunity for preemptive capability 7
Proof-of-concept: One type of application Web server Web is a popular, ubiquitous service Allowed through most firewalls Existing EMERALD analysis engine for HTTP Many Web servers allow custom extensions Apache Web server Apache ~60% of market according to Netcraft Open-source, well-documented module interface 8
Implementation Architecture and Data Flow 1. The Web server Host receives a request Web server 2. Module produces Data EMERALD collection libraries transaction data module 3. Message is sent to Transaction records as EMERALD messages eXpert-HTTP 4. eXpert-HTTP eXpert-HTTP Alerts performs analysis 9
Outline Part Two Inside the Apache Server Performance evaluation Evasion techniques Problems with this approach Future work Related approaches Conclusions 10
Inside the Apache Server Apache uses a request loop Hooks are available in all phases of the loop Our module is hooked to the logging phase Currently no feedback from analysis unit � Passive data collection [Stein 1999] 11
Performance Experiment Setup Goal: Measure impact on user experience Used WebLoad from RadView Software Set up to request a single URL repeatedly Measured transaction round-trip time Each run was 60 minutes with 10 virtual clients on a single physical host Static page: text and one image Dynamic page: CGI program 12
Performance Results: Static Page 50 KB text + 12 KB JPEG Round-trip (s) No IDS W/ IDS Impact median 1.486 1.517 2.1% average 1.499 1.521 1.5% std dev 0.057 0.059 13
Performance Results: Dynamic Page Execution of a CGI program Round-trip (s) No IDS W/ IDS Impact median 1.192 1.229 3.1% average 1.195 1.238 3.6% std dev 0.034 0.048 14
Evasion Techniques Using lower protocol GET / HTTP 1.1 3. Host: victim levels [Ptacek and Content-Length: 3 Newsham] 123GET /cgi-bin/phf Crafting ambiguous The evasion HTTP request techniques work GET /cgi-bin/phf Tab 1. because Web servers and NIDS decode GET /%00cgi-bin/phf 2. them differently 15
Problems 1) Invasive 2) Application-specific Because the module A module must be must run within the written specifically for server application, it every application you could impact stability want to monitor Testing is difficult Every application has its own interface (or none) Server applications typically do not run from for customized module batch input But: For many services, Forced to use scripts only a few major brands rather than data files 16
Future Work Other Web server Great potential for brands improved analysis iPlanet – prototype ready In many cases, knows how the request was serviced Other services (e g document or CGI FTP program) SMTP (sendmail) Could detect evasion Remote access (telnet, attempts rlogin) Knows the exact local Databases filename request refers to Check expected control flow of program 17
Preemption: Two-tiered approach Problem: Complete analysis in module ⇒ performance hit Analysis on separate host ⇒ excludes preemption Solution: Two-tier analysis Module contains simple (fast) analysis engine: hash of suspicious source/CGI program eXpert-HTTP performs complete (slower) analysis and keeps a global state eXpert updates the module’s (simple) knowledge Three options for server module: serve request, deny request, or wait for further analysis 18
Per-request Granularity Single requests can be stopped or delayed (awaiting analysis) Select requests for analysis depending on the type (static, dynamic, directory) Compare with a packet-filtering firewall IP address/port granularity Can only block, not delay 19
Related Approaches mod_id by Burak Dayioglu (www.dayioglu.net) Performs simple CGI name matching inside module (development discontinued) TripWire for Web pages Limited to stopping altered content from being served (MD5 checksums) Interfacing Trusted Applications with Intrusion Detection Systems Marc Welz and Andrew Hutchison, RAID 2001 20
Conclusions Application-integrated data collection for IDS complements data from other levels and locations Addresses the three most severe problems NIDS are currently facing: Encryption Evasion Network speed Prototype integrated with Apache and with EMERALD infrastructure and analysis engine 21
Recommend
More recommend
