Runaway complexity in Big Data And a plan to stop it Nathan Marz - PowerPoint PPT Presentation

Apr 25, 2023 •109 likes •1.33k views

Runaway complexity in Big Data And a plan to stop it Nathan Marz @nathanmarz 1 Agenda Common sources of complexity in data systems Design for a fundamentally better data system What is a data system? A system that manages the storage

Batch views are optimized for the queries they serve
Batch views • Batch-writable from MapReduce • Fast random reads • Examples: ElephantDB, Voldemort
Batch view database No random writes required!
Properties All Batch data view Function ElephantDB is only a few thousand lines of code Simple
Properties All Batch data view Function Scalable
Properties All Batch data view Function Highly available
Properties All Batch data view Function Can be heavily optimized (b/c no random writes)
Properties All Batch data view Function Normalized

Recommend

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal Thermal runaway runaway does not scare you, then you are 2 Thermal runaway Brand new to the stationary battery industry Ignorant of thermal

360 views • 20 slides

Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality

Complexity, Big Data Science, and Happiness Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality Symmetry Breaking Discrete Days, St. Michaels College, 2011 The Big Theory Revolution: Big Data

910 views • 74 slides

Data Streams & Communication Complexity Lecture 3: Communication Complexity and Lower Bounds

Data Streams & Communication Complexity Lecture 3: Communication Complexity and Lower Bounds Andrew McGregor, UMass Amherst 1/23 Basic Communication Complexity Three friends Alice, Bob, and Charlie each have some information x , y , z

1.11k views • 98 slides

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1,

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1, 2019 2 Runaway and Homeless Youth Programs October 1, 2019 3 Runaway and Homeless Youth (RHY) Definitions Runaway youth: a person under 18

525 views • 34 slides

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The case of the runaway trolley The case of the runaway trolley You are standing near a switch next a railroad track when you notice a runaway trolley

565 views • 34 slides

An Explanation for the High-Beta Runaway: the Non-Zonal Transition M.J. Pueschel many thanks to :

An Explanation for the High-Beta Runaway: the Non-Zonal Transition M.J. Pueschel many thanks to : P .W. Terry, F. Jenko, D.R. Hatch, W.M. Nevins, T. G orler, D. Told, A.E. White 55th Annual Meeting of the APS Division of Plasma Physics

305 views • 19 slides

Measurement and Simulation of Thermal Runaway Conditions in the LHC Interconnects L. Bottura and

Measurement and Simulation of Thermal Runaway Conditions in the LHC Interconnects L. Bottura and A. Verweij Based on work and many contributions from: L. Gaborit, P. Fessia, L. Fiscarelli, V. Inglese, G. Montenero, G. Peiro, H. Prin, C.

570 views • 45 slides

threshold conditions for runaway electron generation and suppression R. Granetz, A. DuBois, B.

An ITPA joint experiment to study threshold conditions for runaway electron generation and suppression R. Granetz, A. DuBois, B. Esposito, J. Kim, R. Koslowski, M. Lehnen, J. Martin-Solis ,C. Paz-Soldan, T.-N. Rhee, P. de Vries, J. Wesley, and L.

507 views • 28 slides

Implementation of a fluid model for the non-linear interaction between runaway electrons and

Summary Implementation of a fluid model for the non-linear interaction between runaway electrons and background plasma V. Bandaru 1 , M. Hoelzl 1 , G. Papp 1 , P. Aleynikov 2 , G. Huijsmans 3 1 Max-Planck-Institute for Plasma Physics, Garching,

555 views • 16 slides

Complexity of DLs RWTH Aachen 1 Germany Complexity of DLs: Overview of the Complexity of

Complexity of DLs RWTH Aachen 1 Germany Complexity of DLs: Overview of the Complexity of Concept Consistency P (co-)NP PSpace ExpTime NExpTime ALC reg ALUN (NP) ALCN add regular roles without , (wrt acyc. TBoxes) ALC u only A ALN

745 views • 16 slides

Lecture 7Limits on Climate Stability, Part 1 The runaway greenhouse/ Future evolution of the

41st Saas-Fee Course From Planets to Life 3-9 April 2011 Lecture 7Limits on Climate Stability, Part 1 The runaway greenhouse/ Future evolution of the Earth J. F. Kasting Venus 93-bar, CO 2 -rich atmosphere Practically no water

373 views • 26 slides

Novel Approaches for Mitigating Plasma Disruptions and Runaway Electrons in Tokamak ADITYA by R.

Novel Approaches for Mitigating Plasma Disruptions and Runaway Electrons in Tokamak ADITYA by R. L. Tanna Institute for Plasma Research, India (Contribution from ADITYA Team) 23-01-2015 25th IAEA/Fusion Engineering Conference (FEC)-2014,

554 views • 20 slides

Whose technology policy? Hegemony of Runaway Technology Policy??!! FATALISM +ve Grid

Power, Policy and Samkhya Philosophy Dipak Gyawali What technology policy? Whose technology policy? Hegemony of Runaway Technology Policy??!! FATALISM +ve Grid HIERARCHISM Asymmetrical Transactions: Risk Absorbing: Strong Ascribed

282 views • 4 slides

The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database

CS 5614: Database Management Systems 5 The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database companies are among the largest in the s/w industry Oracle, Informix, Sybase Other Market Forces: Microsoft,

688 views • 31 slides

A Comprehensive Scenario Agnostic Data LifeCycle Model for an Efficient Data Complexity

2016 IEEE 12th International Conference on eScience, Baltimore A Comprehensive Scenario Agnostic Data LifeCycle Model for an Efficient Data Complexity Management Amir Sinaeepourfard, Jordi Garcia , Xavier Masip-Bruin, Eva Marn-Tordera

335 views • 15 slides

Generating Networks with Sur- plus Complexity p. 1/17 Complexity, or amount of infor-

Generating Networks with Sur- plus Complexity p. 1/17 Complexity, or amount of information Note: Information = data + meaning Meaning B Meaning A Semantic Space B is more complex than A

497 views • 17 slides

} Input Correctness Output Rigorous, Unambiguous and Sufficiently Basic at each

Complexity & Analysis of Data Complexity & Analysis of Data Complexity & Analysis of Data Structures & Algorithms Structures & Algorithms Structures & Algorithms Piyush Kumar Piyush Kumar (Lecture 2: Alg (Lecture 2:

250 views • 13 slides

Massive runaway stars: probes for stellar physics and dynamics Mathieu Renzo Collaborators: E.

Massive runaway stars: probes for stellar physics and dynamics Mathieu Renzo Collaborators: E. Zapartas, S. E. de Mink, Y. G otberg, S. Justham, R. J. Farmer, R. G. Izzard, S. Toonen, D. J. Lennon, H. Sana, E. Laplace, S. N. Shore, F .

598 views • 42 slides

Thermal Runaway Propagation Testing of Lithium Battery Shipments for Safe Transportation J.

Thermal Runaway Propagation Testing of Lithium Battery Shipments for Safe Transportation J. Jeevarajan, Ph.D., Underwriters Laboratories Inc. December, 2018 Introduction and Background Lithium-ion batteries are being used in ground,

373 views • 18 slides

Building Connections with Group Home & Runaway Teens October 16, 2012 6-7 pm About the

Building Connections with Group Home & Runaway Teens October 16, 2012 6-7 pm About the Presenter: Hanna Duff Began volunteering with School on Wheels at Kids to Kids group home in February 2010 Earned elementary teaching

484 views • 16 slides

Communication Complexity BASICS Summer School 2015 Communication

Communication Complexity BASICS Summer School 2015 Communication Complexity of Relations Direct Sum Lower Bounds for Disjointness Asymmetric Communication Complexity and Data Structures d H ( x, y ) 0

1.7k views • 154 slides

The Effect of Thermal Management Material on Thermal Runaway Propagation Greg Wilk, Siddique

The Effect of Thermal Management Material on Thermal Runaway Propagation Greg Wilk, Siddique Khateeb, Stephen Wilke, Greg Albright, Said Al-Hallaj R&D Engineer, AllCell Technologies http://www.allcelltech.com Compact | Lightweight | Long

1.18k views • 21 slides

Today My Info : Timings for the class References Pre-Requisites Survey How you

449 views • 23 slides

Runtime Complexity CS 331: Data Structures and Algorithms Michael Lee <lee@iit.edu> So far,

Runtime Complexity CS 331: Data Structures and Algorithms Michael Lee <lee@iit.edu> So far, our runtime analysis has been based on empirical data i.e., runtimes obtained from actually running our algorithms This data is very sensitive

633 views • 49 slides