Melting the Snow Using Active DNS Measurements to Detect Snowshoe - PowerPoint PPT Presentation

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der Toorn February 1, 2018 University of Twente, Design and Analysis of Communication Systems

Introduction Olivier @lordievader:corellian.student.utwente.nl o.i.vandertoorn@utwente.nl 1

Introduction Olivier 2

Introduction Olivier DIARY DIARY DIARY DIARY 3

Introduction Olivier We hypothesize that the use of active DNS measurements is a good way to detect snowshoe spam domains. 4

Introduction Olivier How can we detect snowshoe spam domains through the use of active DNS measurements? 4

Introduction Olivier Are we able to automate the detection of snowshoe spam domains? 4

Introduction Olivier What are the advantages of this approach over other approaches? 4

Introduction Olivier What are the advantages of this approach over other approaches? How large is the time advantage of this approach? 4

Introduction Olivier What are the advantages of this approach over other approaches? How large is the time advantage of this approach? How much more spam is blocked because of this method? 4

Overview 5

Overview Black box 6

Overview Box of domains Black box 7

Overview Box of domains Black box Notepad 7

A Closer Look 8

A Closer Look Box of domains Black box Notepad 9

A Closer Look Box of domains Machine Notepad Learning 9

A Closer Look Box of domains Machine Notepad Learning 9

A Closer Look OpenINTEL Machine Notepad Learning 9

A Closer Look OpenINTEL Machine Notepad Learning 9

A Closer Look OpenINTEL Machine Realtime Learning Blackhole List (RBL) 9

A Closer Look OpenINTEL Machine Realtime Learning Blackhole List (RBL) 9

A Closer Look OpenINTEL Machine Realtime SURFmailfilter Learning Blackhole List (RBL) 9

OpenINTEL 10

OpenINTEL 11

OpenINTEL A dataset A labeled dataset OpenINTEL 12

OpenINTEL The long tail of the DNS 13

OpenINTEL long The tail of the DNS 13

OpenINTEL long The tail of the DNS 99% 98% 99.9% 97% 13

OpenINTEL 100% 100% 16.6 77.0 98% 11.2 80% 96% CDF CDF 60% 94% positives positives 92% 40% negatives negatives 90% 0 10 20 30 40 50 0 20 40 60 80 100 Number of A records Number of MX records 14

Machine Learning 15

Machine Learning 16

SVC 13449 1081 2339 8512 GaussianNB 13330 1200 2075 8776 RadiusNeighborsClassifier 13318 1212 2367 8484 12995 1535 2507 8344 BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 MultinomialNB 12179 2351 1397 9454 RandomForestClassifier 11156 3374 1488 9363 MLPClassifier 7273 7257 707 10144 6279 8251 695 10156 DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 4562 9968 676 10175 KNeighborsClassifier SGDClassifier 3599 10931 674 10177 Spam Ham TN FP FN TP Type Machine Learning 17

SVC 13449 1081 2339 8512 GaussianNB 13330 1200 2075 8776 RadiusNeighborsClassifier 13318 1212 2367 8484 Spam Ham TN FP FN TP Type Machine Learning 12995 1535 2507 8344 BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 12179 2351 1397 9454 MultinomialNB RandomForestClassifier 11156 3374 1488 9363 MLPClassifier 7273 7257 707 10144 6279 8251 695 10156 DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 4562 9968 676 10175 KNeighborsClassifier SGDClassifier 3599 10931 674 10177 17

SVC 13449 1081 2339 8512 GaussianNB 13330 1200 2075 8776 Spam Ham TN FP FN TP Type Machine Learning RadiusNeighborsClassifier 13318 1212 2367 8484 12995 1535 2507 8344 BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 12179 2351 1397 9454 MultinomialNB RandomForestClassifier 11156 3374 1488 9363 MLPClassifier 7273 7257 707 10144 6279 8251 695 10156 DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 4562 9968 676 10175 KNeighborsClassifier SGDClassifier 3599 10931 674 10177 17

SVC 13449 1081 2339 8512 Spam Ham Type TP FN FP TN Machine Learning GaussianNB 13330 1200 2075 8776 RadiusNeighborsClassifier 13318 1212 2367 8484 12995 1535 2507 8344 BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 12179 2351 1397 9454 MultinomialNB RandomForestClassifier 11156 3374 1488 9363 MLPClassifier 7273 7257 707 10144 6279 8251 695 10156 DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 4562 9968 676 10175 KNeighborsClassifier SGDClassifier 3599 10931 674 10177 17

Spam Ham Type TP FN FP TN Machine Learning SVC 13449 1081 2339 8512 GaussianNB 13330 1200 2075 8776 RadiusNeighborsClassifier 13318 1212 2367 8484 12995 1535 2507 8344 BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 12179 2351 1397 9454 MultinomialNB RandomForestClassifier 11156 3374 1488 9363 MLPClassifier 7273 7257 707 10144 6279 8251 695 10156 DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 4562 9968 676 10175 KNeighborsClassifier SGDClassifier 3599 10931 674 10177 17

Machine Learning True Positives Precision = True Positives + False Positives 18

Spam Ham Type TP FN FP TN Precision Machine Learning SVC 13449 1081 2339 8512 85.18% GaussianNB 13330 1200 2075 8776 86.53% RadiusNeighborsClassifier 13318 1212 2367 8484 84.90% 12995 1535 2507 8344 83.82% BernoulliNB GradientBoostingClassifier 12645 1885 9605 1246 56.83% 12179 2351 1397 9454 89.70% MultinomialNB RandomForestClassifier 11156 3374 1488 9363 88.23% MLPClassifier 7273 7257 707 10144 91.14% 6279 8251 695 10156 90.03% DecisionTreeClassifier AdaBoostClassifier 5971 8559 164 10687 97.32% 4562 9968 676 10175 87.09% KNeighborsClassifier SGDClassifier 3599 10931 674 10177 84.22% 19

AdaBoostClassifier Improved 6688 7842 110 10741 98.38% Spam Ham Type TP FN FP TN Precision Machine Learning AdaBoostClassifier 5971 8559 164 10687 97.32% MLPClassifier 7273 7257 707 10144 91.14% 6279 8251 695 10156 90.03% DecisionTreeClassifier MultinomialNB 12179 2351 1397 9454 89.70% 11156 3374 1488 9363 88.23% RandomForestClassifier KNeighborsClassifier 4562 9968 676 10175 87.09% GaussianNB 13330 1200 2075 8776 86.53% 13449 1081 2339 8512 85.18% SVC RadiusNeighborsClassifier 13318 1212 2367 8484 84.90% 3599 10931 674 10177 84.22% SGDClassifier BernoulliNB 12995 1535 2507 8344 83.82% GradientBoostingClassifier 12645 1885 9605 1246 56.83% 20

Spam Ham Type TP FN FP TN Precision Machine Learning AdaBoostClassifier Improved 6688 7842 110 10741 98.38% AdaBoostClassifier 5971 8559 164 10687 97.32% MLPClassifier 7273 7257 707 10144 91.14% 6279 8251 695 10156 90.03% DecisionTreeClassifier MultinomialNB 12179 2351 1397 9454 89.70% 11156 3374 1488 9363 88.23% RandomForestClassifier KNeighborsClassifier 4562 9968 676 10175 87.09% GaussianNB 13330 1200 2075 8776 86.53% 13449 1081 2339 8512 85.18% SVC RadiusNeighborsClassifier 13318 1212 2367 8484 84.90% 3599 10931 674 10177 84.22% SGDClassifier BernoulliNB 12995 1535 2507 8344 83.82% GradientBoostingClassifier 12645 1885 9605 1246 56.83% 20

Spam Ham Type TP FN FP TN Precision Machine Learning AdaBoostClassifier Improved 6688 7842 110 10741 98.38% AdaBoostClassifier 5971 8559 164 10687 97.32% MLPClassifier 7273 7257 707 10144 91.14% 6279 8251 695 10156 90.03% DecisionTreeClassifier MultinomialNB 12179 2351 1397 9454 89.70% 11156 3374 1488 9363 88.23% RandomForestClassifier KNeighborsClassifier 4562 9968 676 10175 87.09% GaussianNB 13330 1200 2075 8776 86.53% 13449 1081 2339 8512 85.18% SVC RadiusNeighborsClassifier 13318 1212 2367 8484 84.90% 3599 10931 674 10177 84.22% SGDClassifier BernoulliNB 12995 1535 2507 8344 83.82% GradientBoostingClassifier 12645 1885 9605 1246 56.83% 21

Realtime Blackhole List (RBL) 22

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 23

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 28984 23

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 28984 1961 23

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 1144 28984 1961 23

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 1095 1144 28984 1961 23

Realtime Blackhole List (RBL) Number of detected domains 100000 10000 1000 100 10 1 0 10 20 30 40 50 60 70 80 Detection in advance (days) 968 1095 1144 28984 1961 23