automatically identify malware capabilities
play

automatically identify malware capabilities Joshua Saxe, Rafael - PowerPoint PPT Presentation

Dec 18, 2022 •9 likes •239 views

CrowdSource: Applying machine learning to web technical documents to automatically identify malware capabilities Joshua Saxe, Rafael Turner, Kristina Blokhin, Jose Nazario Invincea Labs A DARPA Cyber Fast Track research effort Approved for

  1. CrowdSource: Applying machine learning to web technical documents to automatically identify malware capabilities Joshua Saxe, Rafael Turner, Kristina Blokhin, Jose Nazario Invincea Labs A DARPA Cyber Fast Track research effort Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 1 official policy or position of the Department of Defense or the U.S. Government.

  2. The inspiration behind CrowdSource… - The Internet is rife with text that combines example code with natural language description of its functionality - Why not use this data to train machine learning models to automatically reverse engineer software? - Such an approach harnesses the web “crowd,” which holds more knowledge than the mind of any one malware reverse engineer - As the web changes such an approach would automatically stay up to date with the latest programming idioms and APIs Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect 2 the official policy or position of the Department of Defense or the U.S. Government.

  3. Sound crazy? Is this even possible? … what are the research questions? KEY RESEARCH QUESTIONS Typical web technical document: What judgments can we make about the capability profile of a malware sample based on this entirely automatic approach? How does this approach compare with systems that rely on explicit encodings of expert knowledge to automatically analyze malware? Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 3 official policy or position of the Department of Defense or the U.S. Government.

  4. Initial evidence that it’s feasible … We took 53 malware samples, unpacked them, and took the union of the function names appearing in their Import Address Tables. Then we downloaded the entire body of Stack Overflow postings (6.5 million in all), loaded them into a database and indexed their text using a full text indexing system (SQLite3, to be precise). Finally, we counted the number of posts in which each symbol appears. Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 4 official policy or position of the Department of Defense or the U.S. Government.

  5. Promising results … Overall 77.6% of the function call names found in the malware appeared somewhere in the Stack Overflow posts. The mean number of posts for the function calls was 3195.78, with a standard deviation of 37034.2. Punchline: the DLL functions called by a sample of malware binaries are discussed explicitly on the web If we could mine these web documents, could we automatically say something about what the malware does? 5 Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

  6. Extracting useful information from the mapping: semantic networks for malware symbols CREATING A SIMPLE SEMANTIC MAP OF THE MALWARE API Our method is based on co-occurrence of a malware sample’s function call names within 20-word windows within the StackOverflow posts. By calculating overall call occurrence as well as pairwise co-occurrence relationships, we build up a network of co-occurrence probabilities. This statistical relationship strongly suggests functional and semantic dependence. The edge weight between two imported function calls is computed by the following equation, which is equivalent to the minimum probability of “call A” appearing given the appearance of “call B” and vice versa: Here InternetOpenA and InternetConnectA occur within 20 words of each other, so we add “1” to their co-occurrence count. Next InternetCloseHandle and HttpOpenRequestA occur within 20 words of each other so we add “1” to their co-occurrence count as well. Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official 6 policy or position of the Department of Defense or the U.S. Government.

  7. StackOverflow Based Semantic Network for One “ Kbot ” IRC bot GRAPHICAL CLUSTER STRUCTURE This example and most others exhibit a graph in which almost all nodes are mutually reachable Graphical cluster structure aligns with intuitive sense of shared meaning and functional dependency between symbols Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 7 official policy or position of the Department of Defense or the U.S. Government.

  8. StackOverflow Based Semantic Network for One “ Kbot ” sample Graph depicting Stack Overflow post co-occurrence relationships for strings in a single “ Kbot ” IRC bot sample Zoomed in view: network component? Zoomed in view: edge labeled with post tags Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 8 official policy or position of the Department of Defense or the U.S. Government.

  9. The next step, actionable intelligence: Explicit recognition of malware capabilities • Basic idea: – We have a list of predefined capabilities ( takes screenshot , logs keystrokes ) – And a set of textual strings that we observe in a malware sample, such as file paths, registry keys and function names – We would like to know P( capability|symbol ) for each capability given each symbol observed in the sample • Research problem: Can we somehow compute these probabilities by training on the StackOverflow corpus? Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the 9 official policy or position of the Department of Defense or the U.S. Government.

  10. Computing capability profiling model based on StackOverflow posts • To compute P(capability|symbol) • To learn our model, pull out all symbols occurring in the malware corpus under analysis • Compute P(capability|symbol) for every possible capability to symbol pair, caching them in a database as we go • After this training phase, finding the probability of a capability given a symbol is a single constant time lookup of a mapping between symbol/capability and probability • Computing probability for a capability given a string of symbols can be performed as follows : Probability that all symbol “sensors” are wrong Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official 10 policy or position of the Department of Defense or the U.S. Government.

  11. StackOverflow approach allows for minimal work in defining capabilities • Defining our capability patterns in a configuration file In contrast to rules based approaches, defining our patterns takes very little work • Because StackOverflow is a living corpus, our capability definitions will stay up to date with new APIs and programming trends • Preliminary empirical results indicate system accuracy is on par with expert rules based approaches but with vastly less work to create rules Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official 11 policy or position of the Department of Defense or the U.S. Government.

  12. Using the approach outlined above, our model “learns” what function calls are associated with what malware capabilities 12 Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

  13. • By linking back to StackOverflow titles, tags and posts, model is also “self - documenting” • In other words, the model can show why it “thinks” certain malware string symbols are associated with certain malware capabilities, by referencing the StackOverflow posts The probability of the compression/decompression Symbols found in the malware corpus capability given the symbol Some example post titles in which both the symbol and the topic co-occur 13 Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

  14. Visual results, per sample: Below, automated analysis results for a SpyEyes malware binary Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official 14 policy or position of the Department of Defense or the U.S. Government.

  15. Automated results for the Kbot IRC bot Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official 15 policy or position of the Department of Defense or the U.S. Government.

  16. How accurate is all this? The answer in the form of ROC curves: • Test dataset: ~300 Windows binaries, ~300 malware samples with known capabilities • All samples came unpacked or we unpacked them ourselves • We are assuming that an unpacking technology is deployed before running the CrowdSource approach… Perfect results detecting IRC capability 16 Approved for Public Release, Distribution Unlimited. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

Recommend

Embracing the new threat: towards automatically, self-diversifying malware Mathias Payer

Embracing the new threat: towards automatically, self-diversifying malware Mathias Payer

Embracing the new threat: towards automatically, self-diversifying malware Mathias Payer <mathias.payer@nebelwelt.net> UC Berkeley and (soon) Purdue University Image (c) http://ucrtoday.ucr.edu/9768/assassin-bugs Malware landscape is

555 views • 27 slides
Automatically Generating User Interfaces Adapted To Users Motor And Vision Capabilities

Automatically Generating User Interfaces Adapted To Users Motor And Vision Capabilities

Automatically Generating User Interfaces Adapted To Users Motor And Vision Capabilities Krzysztof Z. Gajos, Jacob O. Wobbrock and Daniel S. Weld Road Map Introduction Interface generation as optimization Modeling motor capabilities

1.57k views • 123 slides
Automatically Evading Classifiers A Case Study on PDF Malware Classifiers Weilin Xu

Automatically Evading Classifiers A Case Study on PDF Malware Classifiers Weilin Xu

Automatically Evading Classifiers A Case Study on PDF Malware Classifiers Weilin Xu David Evans Yanjun Qi University of Virginia Machine Learning is Solving Our Problems Fake Fake Spam IDS

1.07k views • 34 slides
AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max

AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max

AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max Wolotsky, Insu Yun, and Taesoo Kim Georgia Institute of Technology, July 27, 2017 Ab About t Us SSLab (@GT) Focusing on s ystem and security

1.25k views • 72 slides
When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware

When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware

When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware Detectors Charles Smutz Angelos Stavrou George Mason University Motivation Machine learning used ubiquitously to improve information security

814 views • 30 slides
Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware virus trojan horse etc. and how do we fight it? AV software Firewalls Filtering Patching Writing more secure software

570 views • 22 slides
Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware

Malware What is malware? Malware: malicious software worm ransomware adware virus trojan horse etc. and how do we fight it? AV software Firewalls Filtering Patching Writing more secure software

540 views • 42 slides
Android Malware Analysis on Attacks and Defense Android malware Android malware With the

Android Malware Analysis on Attacks and Defense Android malware Android malware With the

Android Malware Analysis on Attacks and Defense Android malware Android malware With the explosive growth of mobile device market and usage, there is an increasing number of malicious mobile applications targeting these devices and

819 views • 44 slides
Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal

Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal

Entrapment: Tricking Malware with Transparent, Scalable Malware Analysis Paul Royal paul@gtisc.gatech.edu Agenda Modern Malware Obfuscations, Server-side Polymorphism, Collection Volume Malware Analysis Detection

504 views • 27 slides
Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that

Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that

Getting started with malware analysis Judith van Stegeren Definitions Malware : any software that does something that causes harm to a user, computer or network, including viruses, trojan horses, worms, rootkits, scareware and spyware. Malware

119 views • 9 slides
GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI

GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI

GOODWARE DRUGS FOR MALWARE: ON-THE-FLY MALWARE ANALYSIS AND CONTAINMENT DAMIANO BOLZONI CHRISTIAAN SCHADE TWENTE SECURITY LAB UNIVERSITY OF TWENTE THE NETHERLANDS AGENDA Something about malware Malware internals Current analysis

398 views • 27 slides
A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND

A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND

A CUCKOOS EGG IN THE MALWARE NEST ON-THE-FLY SIGNATURE-LESS MALWARE ANALYSIS, DETECTION AND CONTAINMENT FOR LARGE NETWORKS CHRISTIAAN SCHADE TWENTE SECURITY LAB UNIVERSITY OF TWENTE THE NETHERLANDS MALWARE WARS In the last

313 views • 14 slides
Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer

Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer

Overview Malware Halting 1. Malware 2. Software diversity Part I: Method Development 3. Computer immunization Kjell Jrgen Hole Simula@UiB 4. Epidemiological model 5. Malware halting analysis 6. Malware halting method Last updated

672 views • 29 slides
How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement

How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement

How does Malware Use RDTSC? A Study on Operations Executed by Malware with CPU Cycle Measurement Yoshihiro Oyama University of Tsukuba 1 Background Many malware programs execute operations for analysis evasion Detection of

346 views • 31 slides
Anti-anti-malware (part 3) / Stack Smashing 1 last time various kinds of armored malware

Anti-anti-malware (part 3) / Stack Smashing 1 last time various kinds of armored malware

Anti-anti-malware (part 3) / Stack Smashing 1 last time various kinds of armored malware malware that evades analysis 2 logistics: LEX homework detect push ret pattern using fmex probably easier than TRICKY 3 upcoming exam next Wednesday

496 views • 48 slides
Android Malware Adventures Mert Can Cokuner Krat Ouzhan Aknc Android Malware

Android Malware Adventures Mert Can Cokuner Krat Ouzhan Aknc Android Malware

DeepSec IDSC Android Malware Adventures Mert Can Cokuner Krat Ouzhan Aknc Android Malware Adventures Agenda INTRODUCTION ANDROID MALWARE COMMAND & CONTROL QUESTIONS & ANSWERS 2 1 2 3 4 Android Malware

1.01k views • 64 slides
Malware Defense I TDDD17 Information Security, Second Course Ulf Kargn Department of

Malware Defense I TDDD17 Information Security, Second Course Ulf Kargn Department of

Malware Defense I TDDD17 Information Security, Second Course Ulf Kargn Department of Computer and Information Science Linkping University Malware Defense Agenda 2 Two lectures Lecture I : Malware basics, malware on the PC,

621 views • 40 slides
FIGHTING MALWARE WITH MACHINE LEARNING Edward Raff Jared Sylvester Mark McLean Need ML for

FIGHTING MALWARE WITH MACHINE LEARNING Edward Raff Jared Sylvester Mark McLean Need ML for

FIGHTING MALWARE WITH MACHINE LEARNING Edward Raff Jared Sylvester Mark McLean Need ML for Malware Amount of malware is growing exponentially Anti-virus and signature based approaches are reactionary, dont work for novel malware

127 views • 11 slides
CO 445H MALWARE AND VIRUSES Dr. Benjamin Livshits Malware: Different Types 2 Spyware is

CO 445H MALWARE AND VIRUSES Dr. Benjamin Livshits Malware: Different Types 2 Spyware is

CO 445H MALWARE AND VIRUSES Dr. Benjamin Livshits Malware: Different Types 2 Spyware is software that aids in gathering A virus is a computer program that is information about a person or organization capable of making copies of itself

716 views • 56 slides
CS7038 - Malware Analysis - Wk03.1 Malware Taxonomy and Terminology Coleman Kane

CS7038 - Malware Analysis - Wk03.1 Malware Taxonomy and Terminology Coleman Kane

CS7038 - Malware Analysis - Wk03.1 Malware Taxonomy and Terminology Coleman Kane kaneca@mail.uc.edu January 24, 2017 Why Classification is Important Malware classification helps us in multiple ways. Here are a couple key ways:

787 views • 16 slides
On Static Malware Detection Tayssir Touili LIPN, CNRS & Univ. Paris 13 Motivation: Malware

On Static Malware Detection Tayssir Touili LIPN, CNRS & Univ. Paris 13 Motivation: Malware

On Static Malware Detection Tayssir Touili LIPN, CNRS & Univ. Paris 13 Motivation: Malware Detection The number of new malware exceeds 75 million by the end of 2011, and is still increasing. The number of malware that produced

1.34k views • 91 slides
Pharmacist and Pharmacy Techician Learning Objectives 1. Identify common industry branding

Pharmacist and Pharmacy Techician Learning Objectives 1. Identify common industry branding

10/18/19 1 Pharmacist and Pharmacy Techician Learning Objectives 1. Identify common industry branding themes. 2. Identify your pharmacy strengths to craft marketing messages that build on your capabilities. 3. Describe geotargeting and

664 views • 53 slides
Introduction to Security Malware Ming Chow (mchow@cs.tufts.edu) Twitter: @0xmchow Learning

Introduction to Security Malware Ming Chow (mchow@cs.tufts.edu) Twitter: @0xmchow Learning

Introduction to Security Malware Ming Chow (mchow@cs.tufts.edu) Twitter: @0xmchow Learning Objectives By the end of this week, you will be able to: Describe types of malware See certain malware behaviors Scan and analyze malware

883 views • 25 slides
Lecture 12 Malware Defenses Stephen Checkoway University of Illinois at Chicago CS 487

Lecture 12 Malware Defenses Stephen Checkoway University of Illinois at Chicago CS 487

Lecture 12 Malware Defenses Stephen Checkoway University of Illinois at Chicago CS 487 Fall 2017 Slides based on Baileys ECE 422 Malware review How does the malware start running? Logic bomb? Trojan horse? Virus?

470 views • 27 slides

More recommend