not exactly approximate algorithms for big data
play

NOT EXACTLY! APPROXIMATE ALGORITHMS FOR BIG DATA FANGJIN YANG - PowerPoint PPT Presentation

Sep 24, 2022 •454 likes •1.11k views

NOT EXACTLY! APPROXIMATE ALGORITHMS FOR BIG DATA FANGJIN YANG DRUID COMMITTER METAMARKETS NELSON RAY QUANTITATIVE ANALYST GOOGLE OVERVIEW THE PROBLEM MANAGE DATA COST EFFICIENTLY THE DATA DEALING WITH EVENT STREAMS SIMPLIFYING

  1. NOT EXACTLY! APPROXIMATE ALGORITHMS FOR BIG DATA FANGJIN YANG · DRUID COMMITTER · METAMARKETS NELSON RAY · QUANTITATIVE ANALYST · GOOGLE

  2. OVERVIEW THE PROBLEM MANAGE DATA COST EFFICIENTLY THE DATA DEALING WITH EVENT STREAMS SIMPLIFYING STORAGE DATA SUMMARIZATION FINDING UNIQUES HYPERLOGLOG ESTIMATING DISTRIBUTION APPROXIMATE HISTOGRAMS

  3. THE PROBLEM

  4. Real-time Bidding Fangjin Yang & Nelson Ray 2014

  5. PROBLEMS ‣ Storing/processing billions of rows is expensive ‣ Reduce storage, improve performance ‣ Reduce storage by throwing away information ‣ Throwing away information reduces accuracy Fangjin Yang & Nelson Ray 2014

  6. THE DATA

  7. THE DATA Timestamp Bid Price 2013-10-28T02:13:43Z 1.19 2013-10-28T02:14:21Z 0.05 2013-10-28T02:55:32Z 1.04 2013-10-28T03:07:28Z 0.16 2013-10-28T03:13:43Z 1.03 2013-10-28T04:18:19Z 0.15 2013-10-28T05:36:34Z 0.01 2013-10-28T05:37:59Z 1.03 Fangjin Yang & Nelson Ray 2014

  8. DATA SUMMARIZATION Timestamp Bid Price Timestamp Revenue Number of Prices 2013-10-28T02:13:43Z 1.19 2013-10-28T02 2013-10-28T02:14:21Z 0.05 2.28 3 2013-10-28T02:55:32Z 1.04 2013-10-28T03:07:28Z 0.16 2013-10-28T03 1.19 2 2013-10-28T03:13:43Z 1.03 2013-10-28T04 0.15 1 2013-10-28T04:18:19Z 0.15 2013-10-28T05:36:34Z 0.01 2013-10-28T05 1.04 2 2013-10-28T05:37:59Z 1.03 Fangjin Yang & Nelson Ray 2014

  9. COMBINING SUMMARIZATIONS Timestamp Revenue Number of Prices Timestamp Revenue Number of Prices 2013-10-28T02 2.28 3 2013-10-28 2013-10-28T03 1.19 2 4.66 8 2013-10-28T04 0.15 1 2013-10-28T05 1.04 2 Fangjin Yang & Nelson Ray 2014

  10. Fangjin Yang & Nelson Ray 2014

  11. SUMMARIZATION SUMMARY ‣ Throw away information about individual events ‣ Drastically reduce storage and improve query speed • On average, 40x reduction in storage on with our own data ‣ We’ve lost info about individual prices ‣ Data summarization is not always trivial Fangjin Yang & Nelson Ray 2014

  12. CASE STUDY 1

  13. CASE STUDY 1 ‣ Problem: determine unique number of elements in a set ‣ Use case: measuring number of unique users DATA BIG DATA Fangjin Yang & Nelson Ray 2014

  14. EXACT SOLUTION ‣ Store every single username (in a Java HashSet) ‣ No loss of information, no accuracy tradeoff Fangjin Yang & Nelson Ray 2014

  15. HASHSET Timestamp Username Timestamp Usernames 2013-10-28T02:13:43Z user1 2013-10-28T02 2013-10-28T02:14:21Z user2 {user1, user2} 2013-10-28T02:55:32Z user1 2013-10-28T03:07:28Z user4 2013-10-28T03 {user4, user97} 2013-10-28T03:13:43Z user97 2013-10-28T04 {user2} 2013-10-28T04:18:19Z user2 2013-10-28T05:36:34Z user9834 {user9834, 2013-10-28T05 user97} 2013-10-28T05:37:59Z user97 Fangjin Yang & Nelson Ray 2014

  16. HASHSET Usernames Timestamp Usernames Timestamp 2013-10-28T02 {user1, user2} {user1, user2, 2013-10-28 2013-10-28T03 {user4, user97} user4, user97, user9834} 2013-10-28T04 {user2} {user9834, 2013-10-28T05 user97} Fangjin Yang & Nelson Ray 2014

  17. EXACT SOLUTION ‣ Storage/Computation: O(# uniques) ‣ We’re not throwing away any information about usernames ‣ Accuracy: 100% Fangjin Yang & Nelson Ray 2014

  18. INFEASIBLE STORAGE ‣ High cardinality user dimensions == infeasible storage • Storage cost for 10^9 unique elements == ~48GB of storage Fangjin Yang & Nelson Ray 2014

  19. CARDINALITY ESTIMATION ‣ Plenty of literature • Linear Counting • Count-Min Sketch • LogLog Fangjin Yang & Nelson Ray 2014

  20. HYPERLOGLOG ‣ Storage: 1.5 KB ( for cardinalities 10^9 and above) • 99.999997% decrease in storage size ‣ Computation: O(1) ( for cardinalities < ~10^10) ‣ Accuracy: 97% Fangjin Yang & Nelson Ray 2014

  21. HASH FUNCTIONS ‣ Maps value in one space (generally larger) to another value in another space (generally smaller) String 0001 HashFn Fangjin Yang & Nelson Ray 2014

  22. WHAT MAKES A GOOD HASH FUNCTION? ‣ Bits of output value are independent and have an equal probability of occurring (50%) String 50% Probability 0xxx HashFn String 50% Probability 1xxx HashFn Fangjin Yang & Nelson Ray 2014

  23. HASHING TWO STRINGS user1 0xxx HashFn user2 1xxx HashFn Fangjin Yang & Nelson Ray 2013

  24. THE NEXT BIT String 00xx 25% Probability HashFn String 10xx 25% Probability HashFn String 25% Probability 01xx HashFn String 25% Probability 11xx HashFn Fangjin Yang & Nelson Ray 2013

  25. HASHING 4 STRINGS user1 00xx HashFn user2 10xx HashFn user3 01xx HashFn user4 11xx HashFn Fangjin Yang & Nelson Ray 2013

  26. HYPERLOGLOG ‣ What about 001x? • If we hashed one string, 12.5% chance this could occur • If we hashed 8 strings, one of them should be this value ‣ What about 000001…x? • Extremely unlikely to occur if we only hashed one string Fangjin Yang & Nelson Ray 2013

  27. HYPERLOGLOG ‣ Looks at distribution of bits of hashed values ‣ Cares about the position of the left most ‘1’ bit ‣ 1000 -> position == 1 ‣ 0100 -> position == 2 ‣ 0011 -> position == 3 Fangjin Yang & Nelson Ray 2014

  28. HYPERLOGLOG ‣ Stores the max position of the left-most ‘1’ bit of hashed values ‣ User1 —> hash —> 1000 (position == 1) ‣ User2 —> hash —> 0100 (position == 2) ‣ User3 —> hash —> 0011 (position == 3) ‣ HLL will store position == 3 Fangjin Yang & Nelson Ray 2014

  29. HYPERLOGLOG Fangjin Yang & Nelson Ray 2014

  30. HYPERLOGLOG ACCURACY String 00xx HashFn String 10xx HashFn String 25% Probability 01xx HashFn String 11xx HashFn Fangjin Yang & Nelson Ray 2013

  31. HYPERLOGLOG ‣ If we fed the stream through a second hash function, we’d have a second independent estimate ‣ Adding more hash functions gives us more independent estimates that we can combine together for a lower variance estimate ‣ This is expensive because we have to hash the same data n times Fangjin Yang & Nelson Ray 2014

  32. HYPERLOGLOG ‣ Instead we can split the stream ‣ Estimate the cardinality of each sub-stream ‣ For each sub-stream ‣ Store the maximum over the positions of the leftmost '1' bit for hashed values of the sub-stream Fangjin Yang & Nelson Ray 2014

  33. HYPERLOGLOG Buckets -INF -INF -INF -INF Fangjin Yang & Nelson Ray 2014

  34. HYPERLOGLOG Buckets user1 01xxx...x 2 HashFn -INF -INF -INF Fangjin Yang & Nelson Ray 2014

  35. HYPERLOGLOG Buckets user1 01xxx...x 2 HashFn user4 01xxx...x 2 HashFn user12 01xxx...x 2 HashFn user7 1xxxx...x 1 HashFn Fangjin Yang & Nelson Ray 2014

  36. HYPERLOGLOG Buckets user6 001xx...x 2 -> 3 HashFn 2 2 1 Fangjin Yang & Nelson Ray 2014

  37. DETERMINING FINAL CARDINALITY Buckets 3 11.00 2 MATH 2 1 Fangjin Yang & Nelson Ray 2014

  38. HYPERLOGLOG Timestamp Buckets 2013-10-28T02 [3, 2, 2, 1] 2013-10-28T03 [1, 2, 1, 2] 2013-10-28T04 [2, 1, 4, 1] 2013-10-28T05 [2, 2, 3, 1] Fangjin Yang & Nelson Ray 2014

  39. HYPERLOGLOG Timestamp HLL Object 2013-10-28 [3, 2, 4, 2] Fangjin Yang & Nelson Ray 2014

  40. Fangjin Yang & Nelson Ray 2014

  41. RESULTS Fangjin Yang & Nelson Ray 2014

  42. CASE STUDY 2

  43. CASE STUDY 2 ‣ Problem: determine distribution of values ‣ Use case: quantiles and histograms ‣ Hourly truncation Fangjin Yang & Nelson Ray 2014

  44. THE DATA Timestamp Bid Price 2013-10-28T02:13:43Z 1.19 2013-10-28T02:14:21Z 0.05 2013-10-28T02:55:32Z 1.04 2013-10-28T03:07:28Z 0.16 2013-10-28T03:13:43Z 1.03 2013-10-28T04:18:19Z 0.15 2013-10-28T05:36:34Z 0.01 2013-10-28T05:37:59Z 1.03 Fangjin Yang & Nelson Ray 2014

  45. EXACT SOLUTION Bid Price Timestamp Timestamp Bid Prices 2013-10-28T02:13:43Z 1.19 2013-10-28T02 2013-10-28T02:14:21Z 0.05 [1.19, 0.05, 1.04] 2013-10-28T02:55:32Z 1.04 2013-10-28T03:07:28Z 0.16 2013-10-28T03 [0.16, 1.03] 2013-10-28T03:13:43Z 1.03 2013-10-28T04:18:19Z 0.15 2013-10-28T04 [0.15] 2013-10-28T05:36:34Z 0.01 2013-10-28T05 [0.01, 1.03] 2013-10-28T05:37:59Z 1.03 Fangjin Yang & Nelson Ray 2014

  46. EXACT SOLUTION Timestamp Bid Prices Timestamp Bid Prices 2013-10-28T02 [1.19, 0.05, 1.04] 2013-10-28 [1.19, 0.05, 1.04, 0.16, 2013-10-28T03 [0.16, 1.03] 1.03, 0.15, 0.01, 1.03] 2013-10-28T04 [0.15] 2013-10-28T05 [0.01, 1.03] Fangjin Yang & Nelson Ray 2014

  47. EXACT SOLUTION ‣ Arrays of values ‣ Storage: Linear ‣ Computation: Linear ‣ Accuracy: 100% ‣ Problem: Storing raw values can often be more expensive than storing the rest of the row. ‣ Solution: Store an approximate representation! Fangjin Yang & Nelson Ray 2014

  48. APPROXIMATE HISTOGRAMS ‣ “A Streaming Parallel Decision Tree Algorithm” ‣ Yael Ben-Haim & Elad Tom-Tov ‣ Storage: Sublinear/Linear ‣ Computation: Sublinear/Linear ‣ Accuracy: pretty good Fangjin Yang & Nelson Ray 2014

  49. RAW DATA • 40 Prices: 3.46, 5.37, 5.62, 5.87, 6.21, 6.79, 7.11, 7.36, 7.55, 7.64, 7.89, 7.9, 8.07, 8.44, 8.62, 8.78, 8.87, 9.03, 9.24, 9.36, 9.58, 9.59, 9.81, 10.31, 10.35, 10.39, 10.47, 10.77, 10.93, 11.04, 11.1, 13.1, 13.27, 13.29, 13.87, 14.29, 14.51, 14.9, 15.75, 17.07 Fangjin Yang & Nelson Ray 2013

  50. RAW DATA Fangjin Yang & Nelson Ray 2013

Recommend

Empirical Comparison of Approximate Inference Algorithms for Networked Data Prithviraj Sen Lise

Empirical Comparison of Approximate Inference Algorithms for Networked Data Prithviraj Sen Lise

Empirical Comparison of Approximate Inference Algorithms for Networked Data Prithviraj Sen Lise Getoor Department of Computer Science University of Maryland, College Park. Workshop on Open Problems in Statistical Relational Learning, 2006

326 views • 16 slides
Algorithms for Big Data (X) Chihao Zhang Shanghai Jiao Tong University Nov. 22, 2019 Algorithms

Algorithms for Big Data (X) Chihao Zhang Shanghai Jiao Tong University Nov. 22, 2019 Algorithms

Algorithms for Big Data (X) Chihao Zhang Shanghai Jiao Tong University Nov. 22, 2019 Algorithms for Big Data (X) 1/10 Today we will introduce a Monte-Carlo algorithm to approximate Matrix Multiplication . Algorithms for Big Data (X) . .

528 views • 32 slides
Approximate Search and Data Reduction Algorithms Research Questions Kyle Porter NTNU Gjvik

Approximate Search and Data Reduction Algorithms Research Questions Kyle Porter NTNU Gjvik

Approximate Search and Data Reduction Algorithms Research Questions Kyle Porter NTNU Gjvik Outline of Presentation Introduction: Problems General Goals Research Questions Brief theoretical/practical background

243 views • 21 slides
New Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with

New Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with

Background v = f g Procedures for min f g Additional Theoretical Results Experiments Summary New Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications Rishabh Iyer Jeff Bilmes

644 views • 37 slides
Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine

Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine

Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine Hamoudi, Patrick Rebentrost, Ansis Rosmanis, Miklos Santha arXiv: 1907.05378 1. Approximate Submodular Function Minimization 2. Quantum speed-up for

994 views • 98 slides
Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine

Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine

Quantum and Classical Algorithms for Approximate Submodular Function Minimization Yassine Hamoudi, Patrick Rebentrost, Ansis Rosmanis, Miklos Santha arXiv: 1907.05378 1. Approximate Submodular Function Minimization 2. Quantum speed-up for

1.54k views • 113 slides
Counting and sampling algorithms at low temperature New frontiers in approximate counting STOC

Counting and sampling algorithms at low temperature New frontiers in approximate counting STOC

Counting and sampling algorithms at low temperature New frontiers in approximate counting STOC 2020 Will Perkins (UIC) Algorithms and phase transitions When are phase transitions barriers to e ffi cient algorithms? What algorithmic

497 views • 30 slides
Approximate Implicitization and CAD- type intersection algorithms Tor Dokken and Vibeke Skytt

Approximate Implicitization and CAD- type intersection algorithms Tor Dokken and Vibeke Skytt

Approximate Implicitization and CAD- type intersection algorithms Tor Dokken and Vibeke Skytt SINTEF ICT, Department of Applied Mathematics This work is partly funded by the European Union through the projects: GAIA II, IST-2001-35512,

437 views • 28 slides
Approximate methods for scalable data mining Andrew Clegg Data Analytics &amp; Visualization

Approximate methods for scalable data mining Andrew Clegg Data Analytics &amp; Visualization

Approximate methods for scalable data mining Andrew Clegg Data Analytics &amp; Visualization Team Pearson Technology Twitter: @andrew_clegg Outline 1. Intro 2. What are approximate methods and why are they cool? 3. Set membership (finding

683 views • 41 slides
Approximate Search of Regular Expressions Using Bit-Parallel Algorithms Kristo Tammeoja Jaak Vilo

Approximate Search of Regular Expressions Using Bit-Parallel Algorithms Kristo Tammeoja Jaak Vilo

Approximate Search of Regular Expressions Using Bit-Parallel Algorithms Kristo Tammeoja Jaak Vilo Teooriapevad Ruges, 2007 Contents Regular expression (RE) syntax Glushkovs automaton Existing bit-parallel algorithms Exact

817 views • 52 slides
Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Approximate inference Approximate inference techniques Deterministic approximation Variational algorithms

1.96k views • 97 slides
Dynamic Inverse Problems: Schmitt Efficient Algorithms and Approximate Inverse Problems

Dynamic Inverse Problems: Schmitt Efficient Algorithms and Approximate Inverse Problems

Inverse Problems A.K. Louis, U. Dynamic Inverse Problems: Schmitt Efficient Algorithms and Approximate Inverse Problems Definitions Inverse Two Constraints Semi - discrete Case Examples A.K. Louis 1 U. Schmitt 2 Approximate Inverse 1

541 views • 29 slides
Probabilistic &amp; Unsupervised Learning Convex Algorithms in Approximate Inference Maneesh

Probabilistic &amp; Unsupervised Learning Convex Algorithms in Approximate Inference Maneesh

Probabilistic &amp; Unsupervised Learning Convex Algorithms in Approximate Inference Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1,

861 views • 41 slides
Projections for Approximate Policy Iteration Algorithms Riad Akrour , Joni Pajarinen, Gerhard

Projections for Approximate Policy Iteration Algorithms Riad Akrour , Joni Pajarinen, Gerhard

Projections for Approximate Policy Iteration Algorithms Riad Akrour , Joni Pajarinen, Gerhard Neumann, Jan Peters IAS, TU Darmstadt, Germany ICML19 Entropy Regularization in RL Widespread with actor-critic methods ICML19 Hard vs Soft

374 views • 6 slides
Statistical Algorithmic Profiling for Randomized Approximate Programs Keyur Joshi , Vimuth

Statistical Algorithmic Profiling for Randomized Approximate Programs Keyur Joshi , Vimuth

CCF-1629431 CCF-1703637 Statistical Algorithmic Profiling for Randomized Approximate Programs Keyur Joshi , Vimuth Fernando, Sasa Misailovic University of Illinois at Urbana-Champaign ICSE 2019 Randomized Approximate Algorithms Modern

609 views • 42 slides
Chapter 3: Top-k Query Processing and Indexing 3.1 Top-k Algorithms 3.2 Approximate Top-k Query

Chapter 3: Top-k Query Processing and Indexing 3.1 Top-k Algorithms 3.2 Approximate Top-k Query

Chapter 3: Top-k Query Processing and Indexing 3.1 Top-k Algorithms 3.2 Approximate Top-k Query Processing 3.3 Index Access Scheduling 3.4 Index Organization and Advanced Query Types 3-1 IRDM WS 2005 3.1 Top-k Query Processing with Scoring

592 views • 36 slides
Sparse Approximate Inverse and Hybrid Algorithms for Matrix Computations Vassil Alexandrov

Sparse Approximate Inverse and Hybrid Algorithms for Matrix Computations Vassil Alexandrov

www.bsc.es On Scalability Behaviour of Monte Carlo Sparse Approximate Inverse and Hybrid Algorithms for Matrix Computations Vassil Alexandrov (ICREA-BSC) SOFIA 2013, Overview About BSC HPC trends Monte Carlo History and Motivation

1.22k views • 49 slides
Algorithms and Data Structures: Overview Algorithms and data structures Data Abstraction,

Algorithms and Data Structures: Overview Algorithms and data structures Data Abstraction,

Algorithms and Data Structures: Overview Algorithms and data structures Data Abstraction, Ch. 3 Linked lists, Ch. 4 Recursion, Ch. 5 Stacks, Ch. 6 Queues, Ch. 7 Algorithm Efficiency and Sorting, Ch. 9 Trees, Ch. 10

729 views • 44 slides
Approximate Matrix Multiplication Lecture 21 April 11, 2019 Chandra (UIUC) CS498ABD 1 Spring

Approximate Matrix Multiplication Lecture 21 April 11, 2019 Chandra (UIUC) CS498ABD 1 Spring

CS 498ABD: Algorithms for Big Data, Spring 2019 Approximate Matrix Multiplication Lecture 21 April 11, 2019 Chandra (UIUC) CS498ABD 1 Spring 2019 1 / 34 Matrix data Lot of data can be viewed as defining a matrix. We have already seen

711 views • 47 slides
Sensors Approximate computing approximate edge detection Machine learning hidden

Sensors Approximate computing approximate edge detection Machine learning hidden

Uncertain&lt; T &gt; A First-Order Type for Uncertain Data James Bornholt Australian National University Todd Mytkowicz Microsoft Research Kathryn S. McKinley Microsoft Research Sensors Approximate computing

1.19k views • 97 slides
Character data Bioinformatics Algorithms (Fundamental Algorithms, module 2) Now the input data

Character data Bioinformatics Algorithms (Fundamental Algorithms, module 2) Now the input data

Character data Bioinformatics Algorithms (Fundamental Algorithms, module 2) Now the input data consists of states of characters for the given objects, Zsuzsanna Lipt ak e.g. morphological data, e.g. number of toes, reproductive method,

211 views • 4 slides
Approximate Graph Embeddings in the Cloud 2 5 3 AC B 2 2 2 2 D 0 3 1 Highlights of

Approximate Graph Embeddings in the Cloud 2 5 3 AC B 2 2 2 2 D 0 3 1 Highlights of

Approximate Graph Embeddings in the Cloud 2 5 3 AC B 2 2 2 2 D 0 3 1 Highlights of Algorithms 2018 Matthias Rost Technische Universitt Berlin, Internet Network Architectures Stefan Schmid Universitt Wien, Communication

329 views • 22 slides
Chapter 24 Approximate Max Cut CS 573: Algorithms, Fall 2013 November 19, 2013 24.1 Normal

Chapter 24 Approximate Max Cut CS 573: Algorithms, Fall 2013 November 19, 2013 24.1 Normal

Chapter 24 Approximate Max Cut CS 573: Algorithms, Fall 2013 November 19, 2013 24.1 Normal distribution 24.1.0.1 Normal distribution proof ) 2 ( ( x 2 ) 2 = x = exp d x 2 x 2 + y 2 ( ) Change of vars: x = r

360 views • 9 slides
Backward Analysis via Over-Approximate Abstraction and Under-Approximate Subtraction Alexey

Backward Analysis via Over-Approximate Abstraction and Under-Approximate Subtraction Alexey

Backward Analysis via Over-Approximate Abstraction and Under-Approximate Subtraction Alexey Bakhirkin 1 Josh Berdine 2 Nir Piterman 1 1 University of Leicester, Department of Computer Science 2 Microsoft Research Goal A backwards analysis

733 views • 23 slides

More recommend