mining large datasets case of mining graph data in the
play

Mining Large Datasets: Case of Mining Graph Data in the Cloud - PowerPoint PPT Presentation

Oct 12, 2022 •662 likes •1.36k views

Background Contributions Conclusion Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent dOrazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining

  1. Background Contributions Conclusion Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent d’Orazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 1 / 50

  2. Background Contributions Conclusion Context and motivations Application domains Computer networks, Social networks, Bioinformatics, Protein structure Chemoinformatics. Graph representation Chemical compound Data modeling. Identifying relationship patterns and rules. Social network Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 2 / 50

  3. Background Contributions Conclusion Context and motivations Mining graph data Graph mining aims to find patterns, hidden relations and behaviors in data. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 3 / 50

  4. Background Contributions Conclusion Context and motivations Mining graph data Graph mining aims to find patterns, hidden relations and behaviors in data. Mining graph goals Computing graph properties: Density, diameter, radius, ... Mining substructures from graph databases. Substructures: paths, trees, subgraphs. Frequent Subgraph Mining (FSM) task. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 3 / 50

  5. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  6. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  7. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . 3Vs of Big Data (Volume, Velocity and Variety). Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  8. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . 3Vs of Big Data (Volume, Velocity and Variety). Availability of cloud computing environments. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  9. Background Contributions Conclusion Context and motivations In this work We are interested to FSM from graph databases. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 5 / 50

  10. Background Contributions Conclusion Context and motivations In this work We are interested to FSM from graph databases. Frequent subgraph mining algorithms Various approaches of FSM. Existing approaches are mainly: Tested on centralized computing systems. Evaluated on relatively small databases. Few works for FSM in the cloud. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 5 / 50

  11. Background Contributions Conclusion Goals Questions Distributed FSM from large graph database. Data/computation distribution. Tuning cloud parameters. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 6 / 50

  12. Background Contributions Conclusion Outline 1 Background 2 Contributions 3 Conclusion Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 7 / 50

  13. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions 3 Conclusion Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 8 / 50

  14. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 9 / 50

  15. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Graph A graph is denoted as G = ( V , E ) where V is a set of nodes and E is a set of edges. Subgraph A graph G ′ = ( V ′ , E ′ ) is a subgraph of another graph G = ( V , E ) iff: V ′ ⊆ V , and E ′ ⊆ E ∩ ( V ′ × V ′ ). Density The density of a graph G = ( V , E ) is 2 ·| E | calculated by density ( G ) = ( | V |· ( | V |− 1)) . Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 10 / 50

  16. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 11 / 50

  17. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Cloud computing Large number of computers that are connected via Internet. Applications delivered as services. Hardware and system software delivered as services. Pay as you go. Cloud services can be rapidly and elastically provisioned. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 12 / 50

  18. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Service models Software as a Service (SaaS). Platform as a Service (PaaS), Infrastructure as a Service (IaaS), Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 13 / 50

  19. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 14 / 50

  20. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background MapReduce framework A framework for processing huge datasets. Large number of computers and task/node failures. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 15 / 50

  21. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background MapReduce framework A framework for processing huge datasets. Large number of computers and task/node failures. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 15 / 50

  22. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 16 / 50

  23. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background SPARK framework A general engine for large-scale data processing. Combine SQL, streaming, and complex analytics. It offers several high-level operators that make it easy to build parallel applications. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 17 / 50

  24. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background SHARK framework A distributed SQL query engine for Hadoop. Based on SPARK and uses the existing Hive client and metastore. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 18 / 50

  25. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 19 / 50

Recommend

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data mining is the use of efficient techniques for the analysis of very large collections of data and the extraction of useful and possibly unexpected

2.4k views • 94 slides
Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

What is Web Mining? Wh t i W b Mi i What is Web Mining? Wh t i W b Mi i ? ? Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques to automat cally d scover and extract nformat on automatically

774 views • 20 slides
Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v

Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v

Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v Big Data Preprocessing/Cleaning v Big Data Acquisition/Measurement J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,tp:// 1

484 views • 47 slides
Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes,

Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes,

Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2013/14 X.13- 1 Chapter X: Graph Mining 1. Introduction to Graph Mining 2. Centrality and Other Graph Properties 3.

576 views • 43 slides
DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is still no unique answer to this question. A tentative definition: Data mining is the use of efficient techniques for the analysis of very large

1.28k views • 62 slides
Web Mining Web Mining Web mining is the use of data mining techniques to automatically

Web Mining Web Mining Web mining is the use of data mining techniques to automatically

What is Web Mining? What is Web Mining? Web Mining Web Mining Web mining is the use of data mining techniques to automatically discover and extract information from Web documents/services (Etzioni, 1996, CACM 39(11)) Web mining aims to

566 views • 22 slides
Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets? Data Mining Gain competitive advantages by analyzing data that describes the life of our computerized society. Artificial Intelligence

878 views • 60 slides
GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan*

GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan*

Graph Mining and Graph Kernels GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan* ^University of Cambridge *IBM T. J. Watson Research Center August 24, 2008 | ACM SIG KDD, Las Vegas Graph Mining and Graph

1.28k views • 60 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

724 views • 54 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

936 views • 38 slides
Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis

Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis

Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis Li Xiong Slides credits: Jiawei Han and Micheline Kamber 1 April 2, 2008 Graph Mining and Social Network Analysis Graph mining Frequent

879 views • 62 slides
Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS Workshop Large-scale graph mining Many applications Friend suggestions Recommendation systems Security Advertising Benefits Big data available Rich

822 views • 71 slides
Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery

Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery

Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery Chlo-Agathe Azencott Machine Learning & Computational Biology Research Group MPIs Tbingen C.-A. Azencott Graph Mining for Chemoinformatics

654 views • 38 slides
Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken

Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken

Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2012/13 T II.Intro- 1 Topic II Intro: Graph Mining 1. Why Graphs? 2. What is Graph Mining 3. Graphs: Definitions 4. Centrality

712 views • 40 slides
CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets

CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets

CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets book. These slides have been modified for CS425. The original slides can be accessed at: www.mmds.org Graph data overview Problems with early

828 views • 80 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Data Mining in Bioinformatics Day 10: Graph Mining in Bioinformatics Karsten Borgwardt February

Data Mining in Bioinformatics Day 10: Graph Mining in Bioinformatics Karsten Borgwardt February

Data Mining in Bioinformatics Day 10: Graph Mining in Bioinformatics Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tbingen with permission from Xifeng Yan and Xianghong

885 views • 54 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data: Part I Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data: Part I Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data: Part I Instructor: Yizhou Sun yzsun@ccs.neu.edu November 12, 2013 Announcement Homework 4 will be out tonight Due on 12/2 Next class will be canceled I will still put the

1.06k views • 77 slides
Data Mining in Bioinformatics Day 9: Graph Mining in Chemoinformatics Chlo-Agathe Azencott

Data Mining in Bioinformatics Day 9: Graph Mining in Chemoinformatics Chlo-Agathe Azencott

Data Mining in Bioinformatics Day 9: Graph Mining in Chemoinformatics Chlo-Agathe Azencott & Karsten Borgwardt February 10 to February 21, 2014 Machine Learning & Computational Biology Research Group Max Planck Institutes Tbingen

477 views • 45 slides
GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten Borgwardt^ and Xifeng Yan*

GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten Borgwardt^ and Xifeng Yan*

Graph Mining and Graph Kernels GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten Borgwardt^ and Xifeng Yan* ^University of Cambridge *IBM T. J. Watson Research Center August 24, 2008 | ACM SIG KDD, Las Vegas Graph Mining and

1.16k views • 48 slides
Graph and Web Mining Motivation, Applications and Algorithms

Graph and Web Mining Motivation, Applications and Algorithms

Graph and Web Mining Motivation, Applications and Algorithms Prof. Ehud Gudes Department of Computer Science BenGurion University, Israel Course Outline Basic concepts of Data Mining and Association

730 views • 46 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com Tutorial on Machine Learning with R The Melbourne Data Science Week 2017 1 June 2017 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies .

733 views • 57 slides
Graph Mining Marco Serafini COMPSCI 532 Lecture 11 Classes of Graph Systems Graph

Graph Mining Marco Serafini COMPSCI 532 Lecture 11 Classes of Graph Systems Graph

Graph Mining Marco Serafini COMPSCI 532 Lecture 11 Classes of Graph Systems Graph Computation Think like a vertex Linear algebra Graph Search Find instances of path expressions Graph Mining Mine patterns of

436 views • 16 slides
THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing,

THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing,

DATA MINING THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing, mining, and post-processing Sampling, feature extraction and normalization Exploratory analysis of data basic statistics What is data

2.11k views • 139 slides

More recommend