melting the snow detecting snowshoe spam domains using
play

Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS - PowerPoint PPT Presentation

Dec 22, 2023 •34 likes •784 views

Olivier van der Toorn <o.i.vandertoorn@utwente.nl> November 13, 2018 University of Twente, Design and Analysis of Communication Systems NOMS 2018 Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS Measurements

  1. Olivier van der Toorn <o.i.vandertoorn@utwente.nl> November 13, 2018 University of Twente, Design and Analysis of Communication Systems NOMS 2018 Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS Measurements

  2. Introduction

  3. 1

  4. 1

  5. 1

  6. 2 Snowshoe Spam Internet

  7. 2 • few hosts Snowshoe Spam Internet Spam

  8. 2 • few hosts Snowshoe Spam Internet Spam • many messages per host

  9. 2 • few hosts Snowshoe Spam Internet Spam Snowshoe Spam • many hosts • many messages per host

  10. 2 • few hosts Snowshoe Spam Internet Spam Snowshoe Spam • many hosts • many messages per host • few messages per host

  11. 3 Assumption

  12. 4 Assumption

  13. ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’

  14. Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 ip4:217.72.207.0/27 -all Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’

  15. Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all

  16. Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139;

  17. v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF

  18. 6 While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Hypothesis

  19. Snowshoe spam + SPF While snowshoe spammers are hard to detect, but still leave a trace in the DNS. 6 Hypothesis

  20. Snowshoe spam + SPF While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Many hosts + a DNS record for each host or a long SPF record 6 Hypothesis

  21. While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Snowshoe spam + SPF Many hosts + a DNS record for each host or a long SPF record 6 Hypothesis Domain with many records or long SPF records

  22. While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Snowshoe spam + SPF Many hosts + a DNS record for each host or a long SPF record Domain with many records or long SPF records Active DNS measurements are a good way to detect snowshoe spam domains. 6 Hypothesis

  23. Methodology

  24. OpenINTEL 7 (DNS data source) Overview

  25. 8 OpenINTEL (DNS data source) Machine Learning (processing) Overview

  26. 8 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) Overview

  27. 8 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Overview

  28. • Queries more than 60% of registered domain names (in total more than 206 million) • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background

  29. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  30. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  31. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  32. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  33. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  34. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  35. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  36. 11 OpenINTEL: Long Tail Analysis

  37. 12 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Machine Learning

  38. The performance of each classifier is compared based on the precision metric. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. Precision True Positives True Positives False Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms

  39. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. True Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms The performance of each classifier is compared based on the precision metric. Precision = True Positives + False Positives

  40. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. True Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms The performance of each classifier is compared based on the precision metric. Precision = True Positives + False Positives

  41. 14 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Realtime Blackhole List (RBL)

  42. 15 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) SURFnet

  43. We have selected the AdaBoost classifier as our classifier of choice, since it had the highest precision metric. The results of our daily detections are stored in an RBL. Active DNS measurements of more than 60% of registered domain names forms the source of our data. We filter out large domains via the Long Tail Analysis. We have evaluated the RBL in SURFmailfilter. 16 Methodology: Recap

  44. The results of our daily detections are stored in an RBL. Active DNS measurements of more than 60% of registered domain names forms the source of our data. We filter out large domains via the Long Tail Analysis. We have evaluated the RBL in SURFmailfilter. 16 Methodology: Recap We have selected the AdaBoost classifier as our classifier of choice, since it had the highest precision metric.

Recommend

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der Toorn February 1, 2018 University of Twente, Design and Analysis of Communication Systems Introduction Olivier

1.46k views • 90 slides
Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der Toorn November 13, 2018 University of Twente, Design and Analysis of Communication Systems Introduction Olivier

825 views • 71 slides
Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt;

Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt;

Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt; November 13, 2018 University of Twente, Design and Analysis of Communication Systems ICT OPEN 2018 Overview Introduction Methodology Results

1.05k views • 55 slides
Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why

Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why

Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why should you care? How to detect Spam? 2 What is Spam? What is Spam? All forms of malicious manipulation of user generated data so as to

943 views • 56 slides
Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a

Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a

Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a highly evolved form of information warfare. Fascinating socioeconomic study with many players - users, ISPs, spammers, technologists, legal

322 views • 17 slides
WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy,

WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy,

WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy, UC Berkeley (Thanks to Yahoo! Research for two internships and a fellowship!) Joint work with Olivier Chapelle and Carlos Castillo (Chato) from

514 views • 28 slides
Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 ,

Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 ,

Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 , Paolo Feraggina 2 , Aristides Gionis 1 1 Yahoo! Research Labs, Barcelona, Spain 2 University of Pisa, Italy motivation Query logs provide valuable

547 views • 16 slides
Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam

Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam

Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam is the friendly name given to unsolicited mail everyone receives in the mailbox. Comes from a Monty Python sketch, where in a caf everything

505 views • 13 slides
Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang

Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang

Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang Hao, Nadeem Ahmed Syed, Nick Feamster, Alexander G. Gray, Sven Krasser Motivation Spam: More than Just a Nuisance Spam: Ham: unsolicited bulk

523 views • 28 slides
Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All

Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All

Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All spam is spam but some spam is more spam than others Opinion spam similar to web spam or email spam in intent, but different in form / content Web

912 views • 38 slides
The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation

The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation

The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation When Congress returns from recess on December 8, 2003, it will finish its work on the CAN-SPAM Act of 2003 (Controlling the Assault of

323 views • 4 slides
Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to

Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to

Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to web spam Spam 221 Spamming PageRank Spam farm model Optimal farm structure Alliances of two farms Larger alliances Spam

876 views • 30 slides
Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and

Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and

Web Dynamics Web Spam Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and Information Systems Prof. Dr. G. Weikum MPII-Sp-0710-1/49 Agenda Web spam - Why and what? Web Spam - Spam taxonomy

669 views • 49 slides
Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information

Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information

Web Dynamics Web Spam Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information Systems Prof. Dr. G. Weikum MPII-Sp-0709-1/49 Agenda Web spam - Why and what? Web Spam - Spam taxonomy Overview

806 views • 49 slides
Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid.

Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid.

Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid. The most is in the North Pole. Sleet Rain that is partly frozen or mixed with snow is sleet. Sleet falls faster then snow.

486 views • 4 slides
Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency?

Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency?

Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency? Address priority areas sooner? Start earlier? Clear snow from driveways? 2018-19 Updates Added 8 miles of snow emergency route

103 views • 7 slides
A SNOW COLLEGE PROGRAM Snow College 150 College Ave. Ephraim, UT 84627 no more excuses! A SNOW

A SNOW COLLEGE PROGRAM Snow College 150 College Ave. Ephraim, UT 84627 no more excuses! A SNOW

A SNOW COLLEGE PROGRAM Snow College 150 College Ave. Ephraim, UT 84627 no more excuses! A SNOW COLLEGE PROGRAM for only $ 5 be active .00 Introduction Flyer be healthy a month be stronger be smarter be happy! fit bit flex (with option

164 views • 15 slides
MELTING OF NANOPARTICLES SUBRATA CHAKRABORTY RINI THOMAS SANDEEPAN MAITY 1 INTRODUCTION The

MELTING OF NANOPARTICLES SUBRATA CHAKRABORTY RINI THOMAS SANDEEPAN MAITY 1 INTRODUCTION The

MELTING OF NANOPARTICLES SUBRATA CHAKRABORTY RINI THOMAS SANDEEPAN MAITY 1 INTRODUCTION The bulk melting temperature is independent on its size. However nanoparticle melting temperature depends on its dimension, due to higher value of

298 views • 28 slides
12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic graphics 2.Detecting manipulated images Photo manipulation is the application of image editing techniques to photographs in order to create an

306 views • 13 slides
Nullification test collections for Web spam and SEO Timothy Jones (ANU) David Hawking

Nullification test collections for Web spam and SEO Timothy Jones (ANU) David Hawking

Nullification test collections for Web spam and SEO Timothy Jones (ANU) David Hawking (Funnelback) Ramesh Sankaranarayana (ANU) Nick Craswel (Microsoft Research) Detection Detecting problem content Nullification Preventing problem content

390 views • 14 slides
Web Spam Know Your Neighbors: Web Spam Detection using the Web Topology Presenter: Sadia Masood

Web Spam Know Your Neighbors: Web Spam Detection using the Web Topology Presenter: Sadia Masood

Seminar: Future Of Web Search University of Saarland Web Spam Know Your Neighbors: Web Spam Detection using the Web Topology Presenter: Sadia Masood Tutor : Klaus Berberich Date : 17-Jan-2008 The Agenda Focus of the First Paper

1.22k views • 53 slides
spam, ham and other food or how to distribute spam to 100k email addresses Who am I? Debian

spam, ham and other food or how to distribute spam to 100k email addresses Who am I? Debian

spam, ham and other food or how to distribute spam to 100k email addresses Who am I? Debian Developer since 2003 Listmaster Alioth Admin Maintainer of packages like iproute, icinga, nagios and a lot of other useless stu ff

219 views • 20 slides
Sulphur Melting An Overview 1 SANDVIK PROCESS SYSTEMS THE INDUSTRIAL PROCESSING COMPANY

Sulphur Melting An Overview 1 SANDVIK PROCESS SYSTEMS THE INDUSTRIAL PROCESSING COMPANY

Sulphur Melting An Overview 1 SANDVIK PROCESS SYSTEMS THE INDUSTRIAL PROCESSING COMPANY Agenda Sulphur Melting Strategy Before You Melt Sulphur Melting Technologies Footprint Comparison Sulphur Contamination

410 views • 17 slides
Machine Learning in Spam Filtering A Crash Course in ML Konstantin Tretyakov kt@ut.ee

Machine Learning in Spam Filtering A Crash Course in ML Konstantin Tretyakov kt@ut.ee

Machine Learning in Spam Filtering A Crash Course in ML Konstantin Tretyakov kt@ut.ee Institute of Computer Science, University of Tartu Overview Spam is Evil ML for Spam Filtering: General Idea, Problems. Some Algorithms Nave Bayesian

391 views • 35 slides

More recommend