in incorporating feedback in into tree based anomaly
play

In Incorporating Feedback in into Tree-based Anomaly Detection - PowerPoint PPT Presentation

Apr 10, 2024 β€’233 likes β€’756 views

In Incorporating Feedback in into Tree-based Anomaly Detection Shubhomoy Das, Weng-Keen Wong, Alan Fern, Thomas G. Dietterich and Md Amran Siddiqui School of EECS Anomaly Detection Goal: Identify rare or strange objects 2 Anomaly

  1. In Incorporating Feedback in into Tree-based Anomaly Detection Shubhomoy Das, Weng-Keen Wong, Alan Fern, Thomas G. Dietterich and Md Amran Siddiqui School of EECS

  2. Anomaly Detection β€’ Goal: Identify rare or strange objects 2

  3. Anomaly Detection β€’ Goal: Identify rare or strange objects 2

  4. Typical Investigation Anomaly Detector Ranking 𝑔(𝑦) 3

  5. Typical Investigation Anomaly Detector Ranking 𝑔(𝑦) 3

  6. Typical Investigation Anomaly Detector Ranking 𝑔(𝑦) 3

  7. Typical Investigation Anomaly Detector Ranking 𝑔(𝑦) . . . 3

  8. Typical Investigation Anomaly Detector Ranking 𝑔(𝑦) β€’ Major problem: Statistical anomalies don’t necessarily correspond to semantic anomalies . . . β€’ Need to deal with large number of false positives 3

  9. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) 4

  10. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Nominal 4

  11. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Nominal 4

  12. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) 4

  13. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Nominal 4

  14. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Nominal 4

  15. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Nominal . . . 4

  16. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Anomaly . . . 4

  17. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) Anomaly . . . 4

  18. Investigation with Feedback Anomaly Detector Ranking 𝑔(𝑦) β€’ Ranking is adaptive Anomaly . β€’ Reduces false positive . . 4

  19. Tree-based Anomaly Detection β€’ Isolation Forest β€’ HS-Trees β€’ RS-Forest β€’ RPAD β€’ Random Projection Forest β€’ … 5

  20. Isolation Forest Random feature and random split β‰₯ point < Shallow leaf indicates anomaly Deeper leaf indicates nominal 6

  21. Isolation Forest Random feature and random split β‰₯ point < Shallow leaf indicates anomaly Deeper leaf indicates nominal 6

  22. Isolation Forest Typically 100 trees in practice Random feature and random split β‰₯ point < Shallow leaf indicates anomaly Deeper leaf indicates nominal 6

  23. Weighted Representation of Trees < β‰₯ 𝑨(𝑦) = βˆ’1, 0, 0, βˆ’1, 0, 0, 0, βˆ’1, βˆ’1, … π‘ˆ (extremely sparse) β€’ Weights for isolation forest: π‘₯ = 1, 1, 1, 1, 1, 1, 1, 1, 1, … π‘ˆ π’š β€’ Different set of weights will result other tree based detectors 𝑑𝑑𝑝𝑠𝑓 𝑦 = π‘₯ π‘ˆ . 𝑨 𝑦 7

  24. Active Anomaly Discovery 𝑒 π‘Ÿ 𝜐 π‘₯ 𝑒 8

  25. Active Anomaly Discovery 𝑒 π‘Ÿ 𝜐 π‘₯ 𝑒 8

  26. Active Anomaly Discovery Nominal 𝑒 π‘Ÿ 𝜐 π‘₯ 𝑒 8

  27. Active Anomaly Discovery Nominal 𝑒 𝑒+1 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘₯ 𝑒 π‘₯ 𝑒+1 8

  28. Active Anomaly Discovery Nominal 𝑒 𝑒+1 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘₯ 𝑒 π‘₯ 𝑒+1 8

  29. Active Anomaly Discovery Anomaly Nominal 𝑒 𝑒+1 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘₯ 𝑒 π‘₯ 𝑒+1 8

  30. Active Anomaly Discovery Anomaly Nominal 𝑒 𝑒+1 𝑒+2 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘₯ 𝑒 π‘₯ 𝑒+1 π‘₯ 𝑒+2 8

  31. Active Anomaly Discovery Anomaly Nominal 𝑒 𝑒+1 𝑒+2 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘₯ 𝑒 π‘₯ 𝑒+1 π‘₯ 𝑒+2 8

  32. Active Anomaly Discovery Anomaly Nominal 𝑒 𝑒+1 𝑒+2 π‘Ÿ 𝜐 π‘Ÿ 𝜐 π‘Ÿ 𝜐 … π‘₯ 𝑒 π‘₯ 𝑒+1 π‘₯ 𝑒+2 8

  33. Result True anomalies Synthetic Dataset Baseline discovers 12 AAD discovers 23 anomalies in anomalies in 35 iterations 35 iterations 9

  34. Result 0 Feedback 10

  35. Result 0 Feedback 10 Feedback 10

  36. Result 20 Feedback 0 Feedback 10 Feedback 10

  37. Result 20 Feedback 0 Feedback 10 Feedback 25 Feedback 10

  38. Result 20 Feedback 0 Feedback 10 Feedback 25 Feedback 35 Feedback 10

  39. A closer look at the data with t-SNE 11

Recommend

Anomaly Detection of Trajectories Junier B. Oliva Anomaly Detection An anomaly (or outlier)

Anomaly Detection of Trajectories Junier B. Oliva Anomaly Detection An anomaly (or outlier)

Anomaly Detection of Trajectories Junier B. Oliva Anomaly Detection An anomaly (or outlier) in a dataset is an instance that is abnormal or unlikely based on the rest of the dataset If the instances in the dataset are labeled as

760 views β€’ 19 slides
Isolation trees Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Isolation

Isolation trees Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Isolation

DataCamp Anomaly Detection in R ANOMALY DETECTION IN R Isolation trees Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Isolation tree DataCamp Anomaly Detection in R Isolation tree plots DataCamp Anomaly Detection in R

631 views β€’ 19 slides
Detecting Attacks Anomaly-based Detection Signature-based Signature-based (Misuse)

Detecting Attacks Anomaly-based Detection Signature-based Signature-based (Misuse)

Detecting Attacks Anomaly-based Detection Signature-based Signature-based (Misuse) Detection Intrusion Detection Host-based Network-based Boriana Ditcheva and Lisa Fowler Active/Passive University of North Carolina at

401 views β€’ 12 slides
Levy Review Summary Presentation 2014 (Incorporating feedback received from presentations and

Levy Review Summary Presentation 2014 (Incorporating feedback received from presentations and

Levy Review Summary Presentation 2014 (Incorporating feedback received from presentations and discussions with Centres and Clubs as at 4 August 2014) Background 2007 AGM -75% levy increase due to national office financial position

518 views β€’ 23 slides
Travel Survey Reporting and Recommendations Final Report incorporating feedback from Key

Travel Survey Reporting and Recommendations Final Report incorporating feedback from Key

Cam and Dursley Station Travel Survey Reporting and Recommendations Final Report incorporating feedback from Key Stakeholder Presentation Meeting (2/2/15) The key background Cam and Dursley station is of strategic importance to the

760 views β€’ 62 slides
An Empirical Evaluation of Entropy- based Traffic Anomaly Detection George Nychis, Vyas Sekar,

An Empirical Evaluation of Entropy- based Traffic Anomaly Detection George Nychis, Vyas Sekar,

An Empirical Evaluation of Entropy- based Traffic Anomaly Detection George Nychis, Vyas Sekar, David Andersen, Hyong Kim, Hui Zhang Carnegie Mellon University Entropy-based Anomaly Detection Goal: detect abnormal behavior scan activity,

447 views β€’ 34 slides
Incorporating Grouping Information Into Bayesian Decision Tree Ensembles Junliang Du Antonio R.

Incorporating Grouping Information Into Bayesian Decision Tree Ensembles Junliang Du Antonio R.

Incorporating Grouping Information Into Bayesian Decision Tree Ensembles Junliang Du Antonio R. Linero 1 / 7 Grouping Structures Common scenarios: omics, with groups corresponding to groups of genes or groups of SNPs. 1 / 7 Additive Models

244 views β€’ 11 slides
Tree-based Methods Here we describe tree-based methods for regression and classification.

Tree-based Methods Here we describe tree-based methods for regression and classification.

Tree-based Methods Here we describe tree-based methods for regression and classification. These involve stratifying or segmenting the predictor space into a number of simple regions. Since the set of splitting rules used to segment the

785 views β€’ 51 slides
Tree-based Methods Here we describe tree-based methods for regression and classification.

Tree-based Methods Here we describe tree-based methods for regression and classification.

Tree-based Methods Here we describe tree-based methods for regression and classification. These involve stratifying or segmenting the predictor space into a number of simple regions. Since the set of splitting rules used to segment the

1.08k views β€’ 55 slides
Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection Jiong Zhang and

Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection Jiong Zhang and

Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection Jiong Zhang and Mohammad Zulkernine School of Computing Queens University, Kingston Ontario, Canada K7L 3N6 {zhang, mzulker} @cs.queensu.ca Abstract- Anomaly

284 views β€’ 6 slides
Outlook of Summer Rainfall Anomaly in Asia with Outlook of Summer Rainfall Anomaly in Asia with

Outlook of Summer Rainfall Anomaly in Asia with Outlook of Summer Rainfall Anomaly in Asia with

Sixteenth Session of the Forum on Regional Climate Monitoring, Assessment and Prediction for Asia (FOCRAII), May 7, 2020 ( ), y , Outlook of Summer Rainfall Anomaly in Asia with Outlook of Summer Rainfall Anomaly in Asia with SMART-based

457 views β€’ 16 slides
Session 12 Tree-based models: tree and rpart Two libraries The tree library is like the

Session 12 Tree-based models: tree and rpart Two libraries The tree library is like the

Session 12 Tree-based models: tree and rpart Two libraries The tree library is like the S-PLUS native library and implements the traditional S-PLUS tree technology The rpart library is due to Beth Atkinson and Terry Therneau of the Mayo

501 views β€’ 28 slides
PANACEA: AUTOMATING ATTACK CLASSIFICATION FOR ANOMALY-BASED NETWORK INTRUSION DETECTION SYSTEMS

PANACEA: AUTOMATING ATTACK CLASSIFICATION FOR ANOMALY-BASED NETWORK INTRUSION DETECTION SYSTEMS

PANACEA: AUTOMATING ATTACK CLASSIFICATION FOR ANOMALY-BASED NETWORK INTRUSION DETECTION SYSTEMS DAMIANO BOLZONI, SANDRO ETALLE AND PIETER HARTEL DISTRIBUTED AND EMBEDDED SECURITY GROUP TWENTE SECURITY LAB 10+ YEARS OF RESEARCH OVER ANOMALY

160 views β€’ 14 slides
1 Conceptual Example (III): Less formally (II) Or-tree-based Search n If all

1 Conceptual Example (III): Less formally (II) Or-tree-based Search n If all

2.4.3 Or-tree-based Search Formal Definitions (I) Or-tree-based Search Model A = ( S , T ): Basic Idea: n Prob set of problem descriptions If every solution is okay, represent the different n Altern Prob +

190 views β€’ 3 slides
What is an anomaly? Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Defining

What is an anomaly? Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Defining

DataCamp Anomaly Detection in R ANOMALY DETECTION IN R What is an anomaly? Alastair Rushworth Data Scientist DataCamp Anomaly Detection in R Defining the term anomaly Anomaly: a data point or collection of data points that do not follow the

870 views β€’ 21 slides
Incorporating Into the Earth Science Curriculum Project based learning -satellite/remote

Incorporating Into the Earth Science Curriculum Project based learning -satellite/remote

Incorporating Incorporating Into the Earth Science Curriculum Project based learning -satellite/remote sensing -ozone -plate tectonics Core Concepts - density - EMS - convection Resources Remote Sensing Project -Expansion of my

327 views β€’ 19 slides
15 Tree-based MT In this chapter, we will cover methods for sequence-to-sequence mapping that are

15 Tree-based MT In this chapter, we will cover methods for sequence-to-sequence mapping that are

15 Tree-based MT In this chapter, we will cover methods for sequence-to-sequence mapping that are based on tree structures. 15.1 Motivation for Tree-based Methods Translation Examples A Generalized Rule josei wa ringo wo tabeta NP 1 wa NP 2

648 views β€’ 15 slides
NEW ANOMALY INDUCED SECOND ORDER TRANSPORT F. PEA-BENTEZ IFT - UAM/CSIC BASED ON:

NEW ANOMALY INDUCED SECOND ORDER TRANSPORT F. PEA-BENTEZ IFT - UAM/CSIC BASED ON:

NEW ANOMALY INDUCED SECOND ORDER TRANSPORT F. PEA-BENTEZ IFT - UAM/CSIC BASED ON: 1304.5529 WITH EUGENIO MEGAS OUTLINE ANOMALIES AND HYDRODYNAMICS STRONGLY COUPLED MODEL FLUID GRAVITY CORRESPONDENCE RESULTS SUMMARY, ACTUAL AND

382 views β€’ 34 slides
CAS CS 460/660 Introduction to Database Systems Tree Based Indexing: B+-tree Slides from UC

CAS CS 460/660 Introduction to Database Systems Tree Based Indexing: B+-tree Slides from UC

CAS CS 460/660 Introduction to Database Systems Tree Based Indexing: B+-tree Slides from UC Berkeley 1.1 How to Build Tree-Structured Indexes Tree-structured indexing techniques support both range searches and equality searches . Two

668 views β€’ 36 slides
Incorporating Off-The- Shelf Components with Event-based Integration Jie Ren, Richard Taylor

Incorporating Off-The- Shelf Components with Event-based Integration Jie Ren, Richard Taylor

Incorporating Off-The- Shelf Components with Event-based Integration Jie Ren, Richard Taylor Institute for Software Research University of California, Irvine Outline Background Event-based Integration Microsoft Java Virtual

294 views β€’ 16 slides
A New SU(2) Anomaly Edward Witten PCTS, October 3, 2018 A familiar anomaly says that in four

A New SU(2) Anomaly Edward Witten PCTS, October 3, 2018 A familiar anomaly says that in four

A New SU(2) Anomaly Edward Witten PCTS, October 3, 2018 A familiar anomaly says that in four spacetime dimensions, an SU(2) gauge theory with a single multiplet of fermions that transform under SU(2) in the spin 1/2 representation is

820 views β€’ 43 slides
Objectives Classifications Feedback Based Stream Ciphers Linear Feedback Shift

Objectives Classifications Feedback Based Stream Ciphers Linear Feedback Shift

Stream Ciphers Debdeep Mukhopadhyay Assistant Professor Department of Computer Science and Engineering Indian Institute of Technology Kharagpur INDIA -721302 Objectives Classifications Feedback Based Stream Ciphers Linear Feedback

263 views β€’ 15 slides
Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional statistical models operate on sequences of words Many translation problems can be best explained by pointing to syntax reordering, e.g., verb

1.39k views β€’ 120 slides
Anomaly-mediated supersymmetry breaking demystified based on JHEP03(2009)123

Anomaly-mediated supersymmetry breaking demystified based on JHEP03(2009)123

Anomaly-mediated supersymmetry breaking demystified based on JHEP03(2009)123 (arXiv:0902.0464) Jae Yong Lee (KIAS) in collaboration with Dong-Won Jung (NCU) PHENO 2009 SYMPOSIUM 1 OUTLINE

439 views β€’ 26 slides

More recommend