Automated Debugging In Data Intensive Scalable Computing Systems - PowerPoint PPT Presentation

Automated Debugging In Data Intensive Scalable Computing Systems Muhammad Ali Gulzar 1 , Matteo Interlandi 3 , Xueyuan Han 2 , Mingda Li 1 , Tyson Condie 1 , and Miryung Kim 1 1 University of California, Los Angeles 2 Harvard University 3 Mircrosoft 1

Big Data Debugging in the Dark Develop locally Hope it works Run in cloud Bug! 1 2 3 4 Guesswork 5 Map Reduce 2

Motivating Example • Alice writes a Spark program that identifies, for each state in the US, the delta between the minimum and the maximum snowfall reading for each day of any year and for any particular year . Zip Code Date Snow Fall 99504 01/01/1994 245mm 99504 01/01/1993 85mm 90031 02/01/1991 0mm … … … 3

Problem Definition • Using a test function, a user can specify incorrect results TextFile FlatMap GroupByKey Map Output AK , 01/01 , 21251 AK, 01/01 , 304.8 AK , 01/01 , [304.8, 21336, 245, 85] 99504, 01/01/1992 , 1ft AK , 03/01, 114.5 AK, 1992 , 304.8 AK , 03/01 , [30.5 , 145] 99504, 03/01/1992 , 0.1ft AK , 1992 , 274.3 AK, 03/01 , 30.5 AK , 1992 , [304.8 , 30.5] 99504, 01/01/1993 , 70in AK , 1993 , 21251 AK, 1992 , 30.5 AK , 1993 , [21336, 145, 85] 99504, 03/01/1993 , 145mm AK , 1994 , 0 AK, 01/01 , 21336 AK , 1994 , [245] def test(key:String, delta: Float) : Boolean = { 99504, 01/01/1994 , 245mm CA , 02/01, 0 AK, 1993 , 21336 CA , 02/01 , [0] delta < 6000 99504, 01/01/1993 , 85mm CA , 1991 , 0 AK, 03/01 , 145 CA , 1991 , [0] } 90031, 02/01/1991 , 0mm AK, 1993 , 145 AK, 01/01 , 245 AK, 1994 , 245 …. …. Given a test function, the goal is to identify a minimum subset of the input that is able to reproduce the same test failure. 4

Existing Approach 1: Data Provenance for Spark TextFile FlatMap GroupByKey Map Output AK , 01/01 , 21251 AK, 01/01 , 304.8 AK , 01/01 , [304.8, 21336, 245, 85] 99504, 01/01/1992 , 1ft AK , 03/01, 114.5 AK, 1992 , 304.8 AK , 03/01 , [30.5 , 145] 99504, 03/01/1992 , 0.1ft AK , 1992 , 274.3 AK, 03/01 , 30.5 AK , 1992 , [304.8 , 30.5] 99504, 01/01/1993 , 70in AK , 1993 , 21251 AK, 1992 , 30.5 AK , 1993 , [21336, 145, 85] 99504, 03/01/1993 , 145mm AK , 1994 , 0 AK, 01/01 , 21336 AK , 1994 , [245] 99504, 01/01/1994 , 245mm CA , 02/01, 0 AK, 1993 , 21336 CA , 02/01 , [0] 99504, 01/01/1993 , 85mm CA , 1991 , 0 AK, 03/01 , 145 CA , 1991 , [0] 90031, 02/01/1991 , 0mm AK, 1993 , 145 AK, 01/01 , 245 AK, 1994 , 245 …. …. It over-approximates the scope of failure-inducing inputs i.e. records in the faulty key-group are all marked as faulty 5

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary search-like procedure on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output AK ,01/01 , 304.8 AK ,1992 , 304.8 AK ,03/01 , 30.5 99504, 01/01/1992 , 1ft AK ,1992 , 30.5 AK , 01/01 , 21251 AK ,01/01 , [304.8, 21336, 245, 85] 1 99504, 03/01/1992 , 0.1ft AK ,01/01 , 21336 AK , 03/01 , 114.5 AK ,03/01 , [30.5 , 145] 99504, 01/01/1993 , 70in AK ,1993 , 21336 AK , 1992 , 274.3 AK ,1992 , [304.8 , 30.5] 99504, 03/01/1993 , 145mm AK ,03/01 , 145 AK , 1993 , 21251 AK ,1993 , [21336, 145, 85] 99504, 01/01/1994, 245mm AK ,1993 , 145 AK , 1994 , 0 AK ,1994 , [245] 2 99504, 01/01/1993, 85mm AK ,01/01 , 245 CA , 02/01 , 0 CA ,02/01 , [0] 90031, 02/01/1991, 0mm AK ,1994 , 245 CA , 1991 , 0 CA ,1991 , [0] …. …. It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 6

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output AK ,01/01 , 304.8 AK , 01/01 , 21031 AK ,01/01 , [304.8, 21336] 99504, 01/01/1992 , 1ft AK ,1992 , 304.8 AK , 03/01 , 0 1 AK ,03/01 , [30.5] 99504, 03/01/1992 , 0.1ft AK ,03/01 , 30.5 AK , 1992 , 274.3 AK ,1992 , [304.8 , 30.5] 99504, 01/01/1993 , 70in 2 AK ,1992 , 30.5 AK , 1993 , 0 AK ,1993 , [21336] AK ,01/01 , 21336 AK ,1993 , 21336 Run 2 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 7

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output AK ,01/01 , 304.8 AK , 01/01 , 0 AK ,01/01 , [304.8] 99504, 01/01/1992 , 1ft AK ,1992 , 304.8 AK , 03/01 , 0 AK ,03/01 , [30.5] 99504, 03/01/1992 , 0.1ft AK ,03/01 , 30.5 AK , 1992 , 274.3 AK ,1992 , [304.8 , 30.5] 99504, 01/01/1993 , 70in AK ,1992 , 30.5 Run 3 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 8

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output 99504, 01/01/1992 , 1ft AK , 01/01 , 0 AK ,01/01 , 21336 AK ,01/01 , [21336] 99504, 03/01/1992 , 0.1ft AK , 1993 , 0 AK ,1993 , 21336 AK ,1993 , [21336] 99504, 01/01/1993 , 70in Run 4 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 9

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output 99504, 01/01/1992 , 1ft AK , 01/01 , 0 AK ,01/01 , 304.8 AK ,01/01 , [304.8] 99504, 03/01/1992 , 0.1ft AK , 1992 , 0 AK ,1992 , 304.8 AK ,1992 , [304.8] 99504, 01/01/1993 , 70in Run 5 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 10

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output 99504, 01/01/1992 , 1ft AK , 03/01 , 0 AK ,03/01 , 30.5 AK ,03/01 , [30.5] 99504, 03/01/1992 , 0.1ft AK , 1992 , 0 AK ,1992 , 30.5 AK ,1992 , [30.5] 99504, 01/01/1993 , 70in Run 6 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 11

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output 99504, 01/01/1992 , 1ft AK , 01/01 , 0 AK ,01/01 , 21336 AK ,01/01 , [21336] 99504, 03/01/1992 , 0.1ft AK , 1993 , 0 AK ,1993 , 21336 AK ,1993 , [21336] 99504, 01/01/1993 , 70in Run 7 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 12

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output AK , 01/01 , 0 AK ,03/01 , 30.5 AK ,01/01 , [21336] 99504, 01/01/1992 , 1ft AK , 03/01 , 0 AK ,1992 , 30.5 AK ,03/01 , [30.5] 99504, 03/01/1992 , 0.1ft AK , 1992 , 0 AK ,01/01 , 21336 AK ,1992 , [30.5] 99504, 01/01/1993 , 70in AK , 1993 , 0 AK ,1993 , [21336] AK ,1993 , 21336 Run 8 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 13

Existing Approach 2: Delta Debugging • Delta Debugging performs a systematic binary-like search on the input dataset using a test oracle function TextFile FlatMap GroupByKey Map Output AK ,01/01 , 304.8 AK , 01/01 , 21031 AK ,01/01 , [304.8, 21336] 99504, 01/01/1992 , 1ft AK ,1992 , 304.8 AK , 1992 , 0 AK ,1992 , [304.8] 99504, 03/01/1992 , 0.1ft AK ,01/01 , 21336 AK , 1993 , 0 AK ,1993 , [21336] 99504, 01/01/1993 , 70in AK ,1993 , 21336 Run 9 It does not prune input records known to be irrelevant because of the lack of semantic understanding of data-flow operators 14

Automated Debugging in DISC with BigSift Output: Minimum Fault-Inducing Input: A Spark Program, A Test Function Input Records Data Provenance + Delta Debugging Prioritizing Bitmap based Test Predicate Backward Test Pushdown Traces Memoization 15

Optimization 1: Test Predicate Pushdown • Observation: During backward tracing, data provenance traces through all partitions even though only a few partitions contain faulty intermediate data. Test Test Test Test Test Test Test Without Test Pushdown With Test Pushdown If applicable, BigSift pushes down the test function to test the output of combiners in order to isolate the faulty partitions. 16

Automated Debugging In Data Intensive Scalable Computing Systems - PowerPoint PPT Presentation

Automated Debugging In Data Intensive Scalable Computing Systems Muhammad Ali Gulzar 1 , Matteo Interlandi 3 , Xueyuan Han 2 , Mingda Li 1 , Tyson Condie 1 , and Miryung Kim 1 1 University of California, Los Angeles 2 Harvard University 3

Automated Debugging In Data Intensive Scalable Computing Systems Muhammad Ali Gulzar 1 Siman Wang

for Data Intensive Scalable Computing CAP3 Gene Assembly Program Compute intensive

MapReduce Data Intensive Computing Data-intensive computing is a class of parallel

Debugging Scalable Applications on the XT May 2nd 2009 Chris Gottbrath Director, Product

Limitlessly Scalable Storage for Capacity-Intensive Computing Meet Cloudian S3-compatible

Scalable Post-Mortem Debugging Abel Mathew CEO - Backtrace amathew@backtrace.io @nullisnt0

What is Advanced Research Computing? Data Supercomputing Computationally Mining Intensive

Data-Intensive Distributed Computing 451/651 (Fall 2020) Part 1: Introduction to Big Data Ali

Data-Intensive Distributed Computing 431/631 (Fall 2020) Part 1: Introduction to Big Data Ali

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2020) Part 7: Data Mining (2/4)

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2019) Part 6: Data Mining (3/4)

Data-Intensive Distributed Computing CS 431/631 451/651 (Winter 2019) Part 6: Data Mining (1/4)

Data-Intensive Distributed Computing 431/451/631/651 (Fall 2020) Part 1: MapReduce Algorithm

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2020) Part 9: Real-Time Data

Automated Debugging for Arbitrarily Long Executions Cristian Zamfir, Baris Kasikci, Johannes

Extreme Data-Intensive Scientific Computing Alex Szalay JHU Big Data in Science Data

Data-Intensive Distributed Computing CS 431/631 451/651 (Winter 2019) Part 6: Data Mining (4/4)

Data-Intensive Distributed Computing CS 431/631 451/651 (Winter 2019) Part 6: Data Mining (2/4)

Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 9: Real-Time Data

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2019) Part 9: Real-Time Data

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2019) Part 9: Real-Time Data

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2020) Part 6: Analyzing Relational

Data-Intensive Distributed Computing CS 431/631 451/651 (Fall 2020) Part 6: Analyzing Relational

Data-Intensive Distributed Computing CS 431/461 451/651 (Fall 2019) Part 2: From MapReduce to