Combating Snowshoe Spam with Fire Olivier van der Toorn - PowerPoint PPT Presentation

Combating Snowshoe Spam with Fire Olivier van der Toorn <o.i.vandertoorn@utwente.nl> November 13, 2018 University of Twente, Design and Analysis of Communication Systems ICT OPEN 2018

Overview Introduction Methodology Results Conclusions 1

Introduction

• Snowshoe Spam Background Info • Active DNS Measurements 2

• Snowshoe Spam Background Info • Active DNS Measurements 2

Background Info • Active DNS Measurements • Snowshoe Spam 2

• Sender Policy Framework (SPF) • DNS domain Starting Point • Snowshoe spam is hard to detect 3

• DNS domain Starting Point • Snowshoe spam is hard to detect • Sender Policy Framework (SPF) 3

Starting Point • Snowshoe spam is hard to detect • Sender Policy Framework (SPF) • DNS domain 3

Research Question How can we detect snowshoe spam through active DNS measurements? 4

Methodology

Methodology 5

• Queries more than 60% of registered domain names OpenINTEL • Active DNS Measurement Platform 6

OpenINTEL • Active DNS Measurement Platform • Queries more than 60% of registered domain names 6

• 37 Features • Long Tail Analysis Datasets & Features • Two types of datasets • Labeled • Unlabeled 7

• Long Tail Analysis Datasets & Features • Two types of datasets • Labeled • Unlabeled • 37 Features 7

Datasets & Features • Two types of datasets • Labeled • Unlabeled • 37 Features • Long Tail Analysis 7

Long Tail Analysis The long tail of the DNS 8

Long Tail Analysis long The tail of the DNS 8

• Ranked performance based on ‘precision’ metric Machine Learning • Trained and evaluated many classifier algorithms 9

Machine Learning • Trained and evaluated many classifier algorithms • Ranked performance based on ‘precision’ metric 9

Precision True Positives Precision = True Positives + False Positives 10

Machine Learning • Trained and evaluated many classifier algorithms • Ranked performance based on ‘precision’ metric • Selected AdaBoost Classifier as classifier of choice (110 false positives out of 10851 ham domains) 11

• Daily detections • Compared to other blacklists Realtime Blackhole List • DNS based way of hosting a blacklist 12

• Compared to other blacklists Realtime Blackhole List • DNS based way of hosting a blacklist • Daily detections 12

Realtime Blackhole List • DNS based way of hosting a blacklist • Daily detections • Compared to other blacklists 12

• Initially in evaluation mode SURF • SURFmailfilter 13

SURF • SURFmailfilter • Initially in evaluation mode 13

Results

Comparison training data 100% 16.6 11.2 80% CDF 60% blacklisted 40% benign 0 10 20 30 40 50 Number of A records 14

Comparison training data 100% 100% 16.6 77.0 98% 11.2 80% 96% CDF CDF 60% 94% blacklisted blacklisted 92% 40% benign benign 90% 0 10 20 30 40 50 0 20 40 60 80 100 Number of A records Number of MX records 15

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 28984 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 28984 1961 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 1144 28984 1961 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 1095 1144 28984 1961 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 968 1095 1144 28984 1961 16

Early Detection Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 928 968 1095 1144 28984 1961 16

Early Detection (update) 17

SURF Results daadzgam.com Domain names realdrippy.com coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates 18

SURF Results daadzgam.com Domain names realdrippy.com coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates 18

SURF Results daadzgam.com Domain names Detected realdrippy.com Blacklisted coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates 18

• 1080 emails • 447 (41.39%) emails with a score of five or higher SURF Results daadzgam.com Domain names Detected realdrippy.com Blacklisted coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates 19

SURF Results daadzgam.com Domain names Detected realdrippy.com Blacklisted coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates • 1080 emails • 447 (41.39%) emails with a score of five or higher 19

SURF Results daadzgam.com Domain names Detected realdrippy.com Blacklisted coachspoke.com stillscratch.com homerope.com quittradition.com 2017-05-24 2017-06-23 2017-07-23 Observation dates • 633 (58.61%) emails have a score below five • 52 unique domains in the body • of which 13 domains have never appeared in an email classified as spam • these 13 domains appeared in 31 emails (2.87%) 20

Additional email blocked 700 Emails marked as spam 600 500 400 300 200 100 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Additional score of the RBL 21

Additional email blocked 700 Emails marked as spam 600 500 335 400 320 300 200 120 100 22 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Additional score of the RBL 21

Additional email blocked 700 Emails marked as spam 554 600 497 441 500 352 335 400 320 300 200 120 100 22 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Additional score of the RBL 21

Additional email blocked 626 629 700 Emails marked as spam 554 600 497 441 500 352 335 400 320 300 200 120 100 22 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Additional score of the RBL 21

Additional email blocked (update) 22

Conclusions

• Early detection • Additional spam blocked Conclusions • Hard to detect spam is detectable 23

• Additional spam blocked Conclusions • Hard to detect spam is detectable • Early detection 23

Conclusions • Hard to detect spam is detectable • Early detection • Additional spam blocked 23

Questions 24