Sophisticated Approaches to Penetrate Highly Secured Systems Claes - PowerPoint PPT Presentation

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Sophisticated Approaches to Penetrate Highly Secured Systems Claes Adam Wendelin April 20, 2017 Chair of Network Architectures and Services Department of Informatics Technical University of Munich

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Contents Background System penetration Quantum insert Malware detection Static analysis Dynamic analysis Heuristic analysis Kernel integrity protection Bibliography A. Wendelin – Malware Detection 2

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Incentives • Economic • £27 billion • $1.7-$4.2 million • Information A. Wendelin – Malware Detection 3

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Advanced Persistent Threats Figure 1: APT attack pyramid [2] A. Wendelin – Malware Detection 4

Chair of Network Architectures and Services Department of Informatics Technical University of Munich System penetration • Quantum Insert • How does it work? • What is required? • Detection? Figure 2: Quantum insert [3] A. Wendelin – Malware Detection 5

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Techniques to detect malware • Static analysis • Dynamic analysis • Heuristic analysis • Kernel integrity protection A. Wendelin – Malware Detection 6

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Static analysis • Analyze binary to determine intent • Syntactic signature detection • Semantic signature detection • Pack to hinder analysis A. Wendelin – Malware Detection 7

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Static analysis using syntactic signature detection • What does the malware look like • Hash functions and databases • Weak against code obfuscation • Randomizing order of 10 independent submodules leads to 10!=3628800 permutations • Polymorphic and metamorphic malware Figure 3: Instruction reordering [1] A. Wendelin – Malware Detection 8

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Dynamic analysis • Execute malware to observe it • Emulated or real-time • Syntactic signature detection • Polymorphic malware • Semantic signature detection A. Wendelin – Malware Detection 9

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Dynamic analysis with semantic signature detection • How does the malware behave • System calls • Behavior graphs • Robust against code obfuscations • Behavior is hard to randomize • Generalization of signatures Figure 4: Behavior graph [5] A. Wendelin – Malware Detection 10

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Heuristic analysis • Machinelearning and data mining • Automation of malware specifications • Feature vector • System calls • OpCode • Robustness? • False positive rates A. Wendelin – Malware Detection 11

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Kernel integrity protection • The kernel rules the computer • input-output • running processes • network • Control manipulation • Pointer tampering • Data manipulation • Inconsistent data structures A. Wendelin – Malware Detection 12

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Kernel integrity protection with shadow pointers • Verify access to function pointers Figure 5: Shadow pointers [7] A. Wendelin – Malware Detection 13

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Kernel integrity protection with signed drivers • Digitally sign drivers • Assumes that signed code is safe • What if signed code is not safe? • Windows and VirtualBox • Return-oriented rootkits A. Wendelin – Malware Detection 14

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Return-oriented rootkits Figure 6: Return-oriented attacks [4] A. Wendelin – Malware Detection 15

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Conclusion • Detecting malware is hard A. Wendelin – Malware Detection 16

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Conclusion • Detecting malware is hard • Detecting malware is getting harder A. Wendelin – Malware Detection 17

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Conclusion • Detecting malware is hard • Detecting malware is getting harder • Syntactic signature detection is broken A. Wendelin – Malware Detection 18

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Conclusion • Detecting malware is hard • Detecting malware is getting harder • Syntactic signature detection is broken • Heuristics and behavior detection are promising areas A. Wendelin – Malware Detection 19

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Conclusion • Detecting malware is hard • Detecting malware is getting harder • Syntactic signature detection is broken • Heuristics and behavior detection are promising areas • Keeping kernel’s integrity is important A. Wendelin – Malware Detection 20

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Questions? • Claes Adam Wendelin • claesadam.wendelin@in.tum.de A. Wendelin – Malware Detection 21

Chair of Network Architectures and Services Department of Informatics Technical University of Munich [1] E. Al Daoud, I. H. Jebril, and B. Zaqaibeh. Computer virus strategies and detection methods. Int. J. Open Problems Compt. Math , 1(2):12–20, 2008. [2] P . Giura and W. Wang. Using large scale distributed computing to unveil advanced persistent threats. Science J , 1(3):93–105, 2012. [3] L. Haagsma. Deep dive into quantum insert. Online at https://blog. foxit. com/2015/04/20/deep-dive-into-quantuminsert , 2015. [4] R. Hund, T. Holz, and F. C. Freiling. Return-oriented rootkits: Bypassing kernel code integrity protection mechanisms. In USENIX Security Symposium , pages 383–398, 2009. A. Wendelin – Malware Detection 22

Chair of Network Architectures and Services Department of Informatics Technical University of Munich [5] C. Kolbitsch, P . M. Comparetti, C. Kruegel, E. Kirda, X.-y. Zhou, and X. Wang. Effective and efficient malware detection at the end host. In USENIX security symposium , pages 351–366, 2009. [6] J. Postel and J. Reynolds. File Transfer Protocol (FTP), 1985. https://tools.ietf.org/html/rfc959. [7] Z. Wang, X. Jiang, W. Cui, and X. Wang. Countering persistent kernel rootkits through systematic hook discovery. In International Workshop on Recent Advances in Intrusion Detection , pages 21– 38. Springer, 2008. A. Wendelin – Malware Detection 23