apache as a malware scanning proxy
play

Apache As A Malware-Scanning Proxy Jeremy Stashewsky, Sophos Plc. - PowerPoint PPT Presentation

Feb 14, 2024 •205 likes •895 views

Apache As A Malware-Scanning Proxy Jeremy Stashewsky, Sophos Plc. http://www.sophos.com/ jeremys@ca.sophos.com Overview The case: building an appliance product Apache HTTPD proxy architecture Malware scanning: challenges and

  1. Apache As A Malware-Scanning Proxy Jeremy Stashewsky, Sophos Plc. http://www.sophos.com/ jeremys@ca.sophos.com

  2. Overview • The case: building an appliance product • Apache HTTPD proxy architecture • Malware scanning: challenges and solutions • Where do we go from here: improving Apache. Slide Contents (c) 2006 Sophos Plc.

  3. Apache as a Proxy • Solid reverse and forward proxy • Decent performance • Variety of AAA modules • 2.2.x: cache modules now stable Slide Contents (c) 2006 Sophos Plc.

  4. Basic Apache Architecture Input Filter Chain Request authn modules mod_disk_cache mod_cache Client mod_proxy Origin Server Response Output Filter Chain CACHE_OUT – hit CACHE_SAVE – miss Slide Contents (c) 2006 Sophos Plc.

  5. Basic Scanning • New output filter captures bytes • Spools to temporary storage – If not cached • Scans with an external program • Safe? Let it through • Unsafe? Show a block page Slide Contents (c) 2006 Sophos Plc.

  6. Problems with Basic Scanning • Launches an external program • Stopping-up latency – Client time-outs – Indefinite content-length – Unhappy users Slide Contents (c) 2006 Sophos Plc.

  7. Alternatives to Launching A Scanner • Worker MPM? – Load engine in child process, scanner threads – Bad: thread crash kills process • ICAP (RFC 3507) scanner? • Custom external scanner – Unix/TCP daemon accepts scan commands Slide Contents (c) 2006 Sophos Plc.

  8. Custom External Scanner • Safety from problem files • Local IPC traffic – No body transfer overhead • Global fairness • Wrapping a Library – Apache w/ protocol filters – Stand-alone daemon with APR Slide Contents (c) 2006 Sophos Plc.

  9. Stream Scanning? • Interesting stuff at EOF – Viruses often append themselves – Many file formats put “Index” at EOF • Just don't send the bad part? – Interpreted – Auto-repair • Disinfection Slide Contents (c) 2006 Sophos Plc.

  10. “Stopping-up” Effect Proxy Normal Proxy: Client Proxy Scanning Scan Proxy: Client t0 t1 t2 Slide Contents (c) 2006 Sophos Plc.

  11. Time-Outs • Client: 60-300 seconds – Highly browser/user-agent dependent • Users: 4-7 seconds – Depends on content; HTML is a bit longer – Speed of Internet pipe important Slide Contents (c) 2006 Sophos Plc.

  12. Keep the Client Happy • Trickle “H... T... T... P... /... 1... ” – Some Clients more willing to wait if data flowing – Tricky: protocol filter • Trickle headers • Pause before body • Trickle body? Dangerous Slide Contents (c) 2006 Sophos Plc.

  13. Keep the User Happy • “Patience Page” • Download, scan and store • Provide link to stored file – E-mail notification? Slide Contents (c) 2006 Sophos Plc.

  14. Patience Page Problems • Right-click, “Save As...” – User: “Corrupt files! Argh!!” – IE: no Referer header – All: no Referer header when entering URL in Address bar – No good workaround • Non-visual Clients (e.g. wget) – Response codes help Slide Contents (c) 2006 Sophos Plc.

  15. A Patience Page in Apache • Send 403, some content • Keep both Client and Origin sockets open • JavaScript sent to Client • Provide download link when done! • Maintain caching? Make an output filter after CACHE_OUT. Slide Contents (c) 2006 Sophos Plc.

  16. Advanced Scanning • “Safe” file type bypass – Can also increase TPS at cost of security • Stream scanning for media – Detect exploits, embedded scripts – Users can “tolerate” streaming media stopping • Incremental scanning – Archives/containers Slide Contents (c) 2006 Sophos Plc.

  17. Architecture Recap Input Filter Chain Request authn modules mod_disk_cache mod_cache Client mod_proxy Origin Server Response Output Filter Chain CACHE_OUT – hit Scanning CACHE_SAVE – miss Daemon VSCAN_OUT – always Slide Contents (c) 2006 Sophos Plc.

  18. Moving Beyond Scanning • Why waste time scanning if you know it's infected? • Interoperability Bugs? Slide Contents (c) 2006 Sophos Plc.

  19. Add URI-based Policy • Blocking an unsafe URI – Save CPU -> more TPS – Combat 0-day & suspected Malwares • Bypass local or trusted sites – Workarounds – Improve Performance • Apache “auth checker” module Slide Contents (c) 2006 Sophos Plc.

  20. A First Step: URI Text File • Linear search – Load into apr_hash ? • Text is easy to patch • Doesn't scale well past a few thousand entries • Key Problem: URIs have structure – string searching doesn't map well Slide Contents (c) 2006 Sophos Plc.

  21. URIs: Relational Database • Good idea if a central database is required • Findings: – Good: apr-util DBI – Good: Reasonable update speed – Bad: Slow lookup time hurts TPS – Bad: Heavyweight Slide Contents (c) 2006 Sophos Plc.

  22. URIs: Simple Database • Data is hierarchical -> search trie • DBM files – apr_dbm in apr-util • Pre-Compiled Hash • Findings: – DBM: faster than relational, small updates – Hash: faster still, but big/slow updates Slide Contents (c) 2006 Sophos Plc.

  23. URIs: Domain Hashing • bucket = substr(hash(domain),...) – Similar to mod_disk_cache's implementation • Splits up database • 12 bits are sufficient for 10 6 domains – 4096 buckets Slide Contents (c) 2006 Sophos Plc.

  24. Hash Domains & Simple DBs • Fast; kept under O(log(n)) • Bucketing keeps indexes small • Binary-diff for distribution • Scales to at least 10 6 entries from experience Slide Contents (c) 2006 Sophos Plc.

  25. Architecture Recap Input Filter Chain URI database Request authn modules policy module mod_disk_cache Client mod_cache Origin Server Response mod_proxy Output Filter Chain CACHE_OUT – hit Scanning CACHE_SAVE – miss Daemon VSCAN_OUT – policy Slide Contents (c) 2006 Sophos Plc.

  26. User Interface • Apache, mod_ssl, mod_php – Administrative and End-user UI • Block and Error pages – Internal redirect to PHP • Patience Page – PHP generates the content to disk one-time – Make file apr_bucket Slide Contents (c) 2006 Sophos Plc.

  27. Where do we go from here? • Transparent Proxy • HTTPS Scanning • mod_cache, mod_disk_cache improvements • mod_proxy improvements Slide Contents (c) 2006 Sophos Plc.

  28. Transparent Proxy • OS redirects traffic • Key: Provide missing info to apache – Fixup-phase module? • Hostname? – Reverse-lookup: unreliable – HTTP/1.1 “Host” header – Resolve, check against destination IP Slide Contents (c) 2006 Sophos Plc.

  29. HTTP over TLS/SSL • Certificate checking – List of trusted CAs • Dynamic Cert generation – Keep Subject, replace Issuer, sign – User must trust Issuer • Transparent? – Grab cert to get hostname! Slide Contents (c) 2006 Sophos Plc.

  30. HTTPS: Social Issues • HTTPS sites can get hacked • Have cert != legitimate • Don't trust proxy to scan? – Policy bypass for individuals • Don't trust admin? – Access your bank from home Slide Contents (c) 2006 Sophos Plc.

  31. Improving mod_cache • Disk cache expiry: needs improvement – Disk cache can grow too large • Cacheability correctness bugs – Apache-Test suite would be handy Slide Contents (c) 2006 Sophos Plc.

  32. Improving mod_cache • Store meta-data with objects – Expiry meta-data – Scan caching & revalidation • Multi-layer cache providers – Scan revalidation as a top-level cache provider – Performance Slide Contents (c) 2006 Sophos Plc.

  33. Improving mod_proxy • Persistent connections! • Limiting connections to an Origin • Overall throughput – Maybe best handled by OS' QoS Slide Contents (c) 2006 Sophos Plc.

  34. Conclusions • Apache: Not just a good web server! – Clear, modular design • Key Challenges Covered: – Stopping-up – Keeping the User Happy – URI-based Policy – Apache improvements Slide Contents (c) 2006 Sophos Plc.

  35. Apache As A Malware-Scanning Proxy Jeremy Stashewsky, Sophos Plc. http://www.sophos.com/ jeremys@ca.sophos.com Slide Contents (c) 2006 Sophos Plc. 1 A bit of background: Sophos develops integrated threat management solutions to protect against malware, spam, and policy abuse. I'm a technical lead developer on a project to build a malware- scanning web gateway appliance (using Apache). This presentation is about some of the core challenges we faced and solutions we tried when building the appliance.

  36. Overview • The case: building an appliance product • Apache HTTPD proxy architecture • Malware scanning: challenges and solutions • Where do we go from here: improving Apache. Slide Contents (c) 2006 Sophos Plc. 2 Our Rough Appliance specs: - 2 to 4 GB Ram - 1 CPU (possibly dual-core) ~ 3GHz - small SATA disks Had to choose between Linux and FreeBSD. Decided to use Linux as it showed better performance with mod_disk_cache.

  37. Apache as a Proxy • Solid reverse and forward proxy • Decent performance • Variety of AAA modules • 2.2.x: cache modules now stable Slide Contents (c) 2006 Sophos Plc. 3 The availability of AAA modules and the modularity of Apache 2's design is what attracted us to use it in our product. Secondarily, we've also got a tradition of using Open Source Software wherever possible – an influence from when the Vancouver office was ActiveState.

Recommend

Misleading and Defeating Importance- Scanning Malware Propagation Guofei Gu 1 , Zesheng Chen 1 ,

Misleading and Defeating Importance- Scanning Malware Propagation Guofei Gu 1 , Zesheng Chen 1 ,

Misleading and Defeating Importance- Scanning Malware Propagation Guofei Gu 1 , Zesheng Chen 1 , Phillip Porras 2 , Wenke Lee 1 1 Georgia Institute of Technology 2 SRI International Outline Background White Hole: Design & Operation

238 views • 20 slides
Git Your YARA For Nothing and Your Malware for Free Automating Scanning with thousands of YARA

Git Your YARA For Nothing and Your Malware for Free Automating Scanning with thousands of YARA

<location, date> < Location , date> Reversing Labs // June 2020 Git Your YARA For Nothing and Your Malware for Free Automating Scanning with thousands of YARA rules Cooper Quintin - Senior Security Researcher - EFF Threat Lab

1.01k views • 22 slides
Apache 2 mod_ssl by example ApacheCon 2005 Mads Toftum mads@apache.org Agenda Getting

Apache 2 mod_ssl by example ApacheCon 2005 Mads Toftum mads@apache.org Agenda Getting

Apache 2 mod_ssl by example ApacheCon 2005 Mads Toftum mads@apache.org Agenda Getting started Certificates Access control Proxy solutions Performance Building mod_ssl The Apache 1.3 + mod_ssl way Download mod_ssl and

339 views • 30 slides
The Hidden Gem Jim Jagielski @jimjag About Me Apache Software Foundation

The Hidden Gem Jim Jagielski @jimjag About Me Apache Software Foundation

Apache httpd v2.4 Reverse Proxy The Hidden Gem Jim Jagielski @jimjag About Me Apache Software Foundation Co-founder, Director, Member and Developer Director Outercurve, MARSEC-XL, OSSI, OSI (ex) Developer Mega

863 views • 45 slides
I n t e r n s L i g h t n i n g T a l k s Proxy editing PiTiVi Proxy editing

I n t e r n s L i g h t n i n g T a l k s Proxy editing PiTiVi Proxy editing

I n t e r n s L i g h t n i n g T a l k s Proxy editing PiTiVi Proxy editing Anton Belka, antonbelka@gmail.com GUADEC 2013 The TM Conference GUADEC Proxy editing What is proxy editing? Proxy editing is the ability to

1.75k views • 143 slides
Apache Traffic Server & Lua Kit Chan (kichan@yahoo-inc.com) Agenda Intro Rationale

Apache Traffic Server & Lua Kit Chan (kichan@yahoo-inc.com) Agenda Intro Rationale

Apache Traffic Server & Lua Kit Chan (kichan@yahoo-inc.com) Agenda Intro Rationale Details Security Future 11/16/2014 2 Apache Traffic Server Fast, scalable and extensible HTTP/1.1 compliant caching proxy server

1.52k views • 45 slides
MySQL Proxy Making MySQL more flexible Jan Kneschke jan@mysql.com MySQL Proxy proxy-servers

MySQL Proxy Making MySQL more flexible Jan Kneschke jan@mysql.com MySQL Proxy proxy-servers

MySQL Proxy Making MySQL more flexible Jan Kneschke jan@mysql.com MySQL Proxy proxy-servers forward requests to backends and can transform, handle or block them released under the GPL see http://forge.mysql.com/wiki/MySQL_Proxy

945 views • 27 slides
Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware virus trojan horse etc. and how do we fight it? AV software Firewalls Filtering Patching Writing more secure software

540 views • 42 slides
Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware virus trojan horse etc. and how do we fight it? AV software Firewalls Filtering Patching Writing more secure software

570 views • 22 slides
Android Malware Analysis on Attacks and Defense Android malware Android malware With the

Android Malware Analysis on Attacks and Defense Android malware Android malware With the

Android Malware Analysis on Attacks and Defense Android malware Android malware With the explosive growth of mobile device market and usage, there is an increasing number of malicious mobile applications targeting these devices and

819 views • 44 slides
Procopiou Anna 931769 Professor: Dr. Athanasopoulos Elias 1 Articles 1. Detecting Malware

Procopiou Anna 931769 Professor: Dr. Athanasopoulos Elias 1 Articles 1. Detecting Malware

Procopiou Anna 931769 Professor: Dr. Athanasopoulos Elias 1 Articles 1. Detecting Malware Domains at the Upper DNS Hierarchy -M.Antonakakis et. al 2. ZMap: Fast Internet-wide Scanning and Its Security Applications -Z. Durumeric et. al 2

1.17k views • 89 slides
Software In-Security Buffer overflows Overview Web Application security Malware OS

Software In-Security Buffer overflows Overview Web Application security Malware OS

Lecture overview Table of contents: Introduction to SW security Software In-Security Buffer overflows Overview Web Application security Malware OS security topics Code scanning Emmanuel Benoist Taught by Ulrich

479 views • 13 slides
Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal

Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal

Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal paul@gtisc.gatech.edu Agenda Modern Malware Obfuscations, Server-side Polymorphism, Collection Volume Malware Analysis Detection

504 views • 27 slides
Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that

Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that

Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that does something that causes harm to a user, computer or network, including viruses, trojan horses, worms, rootkits, scareware and spyware. Malware

119 views • 9 slides
GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI

GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI

GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI CHRISTIAAN SCHADE TWENTE SECURITY LAB UNIVERSITY OF TWENTE THE NETHERLANDS AGENDA Something about malware Malware internals Current analysis

398 views • 27 slides
A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND

A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND

A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND CONTAINMENT FOR LARGE NETWORKS CHRISTIAAN SCHADE TWENTE SECURITY LAB UNIVERSITY OF TWENTE THE NETHERLANDS MALWARE WARS In the last

313 views • 14 slides
Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer

Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer

Overview Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer immunization Kjell Jrgen Hole Simula@UiB 4. Epidemiological model 5. Malware halting analysis 6. Malware halting method Last updated

672 views • 29 slides
Apache Felix Web Console Carsten Ziegeler | cziegeler@apache.org ApacheCon NA 2014 About

Apache Felix Web Console Carsten Ziegeler | cziegeler@apache.org ApacheCon NA 2014 About

Apache Felix Web Console Carsten Ziegeler | cziegeler@apache.org ApacheCon NA 2014 About cziegeler@apache.org @cziegeler RnD Team at Adobe Research Switzerland Member of the Apache So fu ware Foundation Apache Felix and Apache

725 views • 26 slides
Bug hunting with Apache Lucene Uwe Schindler Apache Lucene PMC & Apache Software Foundation

Bug hunting with Apache Lucene Uwe Schindler Apache Lucene PMC & Apache Software Foundation

Bug hunting with Apache Lucene Uwe Schindler Apache Lucene PMC & Apache Software Foundation Member uschindler@apache.org http://www.thetaphi.de, http://blog.thetaphi.de @ThetaPh1 SD DataSolutions GmbH , Wtjenstr. 49, 28213 Bremen,

859 views • 51 slides
Integrating Apache Camel with Apache Syncope Dr. Colm higeartaigh, Talend. Speaker

Integrating Apache Camel with Apache Syncope Dr. Colm higeartaigh, Talend. Speaker

Integrating Apache Camel with Apache Syncope Dr. Colm higeartaigh, Talend. Speaker Introduction Introducing Apache Syncope Apache Syncope basics Apache Syncope is an Open Source system for managing digital identities in enterprise

929 views • 33 slides
How Apache works JB Onofr <jbonofre@apache.org> Who am I JB Onofr

How Apache works JB Onofr <jbonofre@apache.org> Who am I JB Onofr

How Apache works JB Onofr <jbonofre@apache.org> Who am I JB Onofr <jbonofre@apache.org> @jbonofre | http://blog.nanthrax.net Fellow /Software Architect at Talend Member of the Apache Software Foundation (started in 2004)

821 views • 45 slides
C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS @ardalis | ardalis.com | weeklydevtips.com t h s What problems does proxy solve? Objectives How is the proxy pattern structured? Apply the

779 views • 23 slides
How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement

How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement

How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement Yoshihiro Oyama University of Tsukuba 1 Background Many malware programs execute operations for analysis evasion Detection of

346 views • 31 slides
The mod_proxy Cookbook A collection of proxy recipes to suit your discerning palate Daniel

The mod_proxy Cookbook A collection of proxy recipes to suit your discerning palate Daniel

The mod_proxy Cookbook A collection of proxy recipes to suit your discerning palate Daniel Ruggeri Who is This Guy? About Daniel Ruggeri Infrastructure guy with a love for code DRuggeri <at> apache.org Standard Disclaimer

608 views • 45 slides

More recommend