architectures that scale deep
play

Architectures that Scale Deep: Regaining Control in Deep Systems - PowerPoint PPT Presentation

Mar 13, 2023 •274 likes •951 views

Architectures that Scale Deep: Regaining Control in Deep Systems Ben Sigelman (@el_bhs, bhs@lightstep.com) Co-founder & CEO: LightStep Co-creator: OpenTracing, OpenTelemetry, Google Dapper, Google Monarch QCon SF, November 2019 Part I

  1. Architectures that Scale Deep: Regaining Control in Deep Systems Ben Sigelman (@el_bhs, bhs@lightstep.com) Co-founder & CEO: LightStep Co-creator: OpenTracing, OpenTelemetry, Google Dapper, Google Monarch QCon SF, November 2019

  2. Part I Scaling, and Deep Systems

  3. What is scale, anyway?

  4. Scaling wide

  5. Scaling wide

  6. Scaling wide

  7. Scaling wide

  8. Scaling wide

  9. Scaling deep

  10. Scaling deep

  11. Scaling deep

  12. Scaling deep

  13. Scaling deep

  14. How does this look for software?

  15. Software: Scaling wide

  16. Software: Scaling deep

  17. How do real-world systems look?

  18. Microservices at scale aren’t just wide systems , they’re deep systems

  19. Deep Systems Deep Systems Architectures with ≥ 4 layers of Architectures with ≥ 4 layers of independently operated services independently operated services (including external/cloud dependencies) (including external/cloud dependencies)

  20. What do deep systems sound like?

  21. What do deep systems sound like? “Don’t deploy on Fridays”

  22. What do deep systems sound like? “Where’s Chris?! I’m dealing with a P0 and they’re the only one who knows how to debug this.”

  23. What do deep systems sound like? “It can’t be our fault, our dashboard says we’re healthy”

  24. What do deep systems sound like? “Kafka is on fire”

  25. What do deep systems sound like? “I need 100% availability from your team. One hundred percent .”

  26. What do deep systems sound like? “I didn’t know I depended on that region”

  27. What do deep systems sound like? “That was on a dashboard but I can’t find it”

  28. What do deep systems sound like? Lots of challenges: - People-management - Security - Multi-tenancy - “Big-customer” success - Performance - Observability

  29. Part II Control Theory: TL;DR Edition

  30. Why do we care so much about observability , anyway?

  32. Inputs Outputs A System … and its state vector,

  33. Observability Inputs Outputs A System … and its state vector, How well can you infer internal state using only the outputs ?

  34. Controllability Inputs Outputs A System … and its state vector, How well can you control internal state using only the inputs ?

  35. Controllability is the dual of Observability

  36. Controllability is the dual of Observability

  37. Part III What Deep Systems Mean for Observability

  38. Pure Monoliths developers per service Deep Systems Architectural evolution # of services

  39. Stress (n): responsibility without control Stress what you can control what you are responsible for

  41. Observability: Shrink This Gap

  42. Mental models A System

  43. Managing Deep Systems Services must have SLOs (“Service Level Objectives”: latency, errors, etc) For effective service management, only three things matter: 0. Releasing service functionality 1. Gradually improving SLOs 2. Rapidly restoring SLOs In a deep system, we must control the entire “triangle” to maintain our SLOs

  44. There’s that word again… Controllability == Observability Controllability == Observability

  45. Observability: “The Conventional Wisdom” Observing microservices is hard Google and Facebook solved this (right???) They used Metrics, Logging, and Distributed Tracing… … So we should, too.

  46. 3 Pillars, 3 Experiences Metrics Logs Traces

  48. Three Pillars? Three Pillars? Two giant pipes… Metrics Without Traces: Cognitive Load ≈ O( depth 2 ) Logs

  49. Three Pillars? Three Pillars? Two giant pipes… Metrics Logs

  51. Two giant pipes… Metrics Without Traces: Cognitive Load ≈ O( depth 2 ) Logs

  53. Traces

  54. Traces provide Context

  55. Traces provide Context And context rules out invalid hypotheses

  56. Two giant pipes and a filter Metrics Context (from traces) Logs

  57. Context reduces cognitive load Relevant Metrics Context (from traces) Relevant Logs With Traces: Cognitive Load ≈ O( depth )

  58. Observability: Shrink This Gap

  60. Let’s Review

  61. Microservices don’t just scale wide, they scale deep Recognize deep systems

  62. Stress (n): responsibility without control Stress what you can control what you are responsible for

  63. “Controllability” (of SLOs) depends on observability

  64. “The Three Pillars of Observability” is a lousy metaphor … and traces are not sprinkles

  65. Tracing can reduce cognitive load from O( depth 2 ) to O( depth )

  66. Tracing is the backbone of simple observability in deep systems

  67. Thank You Play with LightStep, Feedback always for free, anytime: welcome: (no email address required!) twitter → @el_bhs lightstep.com/play the emails → bhs@lightstep.com

Recommend

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira and Ian Goodfellow A brief overview of other architectures Unsupervised Architectures Deep Belief Networks Autoencoders GANs

439 views • 25 slides
Multi-Scale and Multi-Physics Simulations on Present and Future Architectures

Multi-Scale and Multi-Physics Simulations on Present and Future Architectures

Multi-Scale and Multi-Physics Simulations on Present and Future Architectures www.uintah.utah.edu Martin Berzins 1. Background and motivation 2. Uintah Software and Multicore Scalability 3. Runtime Systems for Heterogeneous Architectures 4.

759 views • 27 slides
Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks

Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks

MLSLP 2012 Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks collaborations/discussions with many people) Introduction Deep neural net (modern multilayer perceptron) Hard to

330 views • 15 slides
MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket @objectrocket @kennygorman MongoDB at scale Designing for scale Techniques to ease pain Things to avoid What is scale? Scale;

539 views • 27 slides
Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on

Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on

Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on Density Inputs t=1, 1000 cells t=2, 650 cells t=3, 890 cells of varying physical attributes Representing inputs as densities (discretized in practice)

303 views • 7 slides
Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous

Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous

Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous Driving Scene understanding /Segmentation Applications that (probably) use DL WordLens Prisma Outline of todays talk Image Recognition Fun

559 views • 43 slides
DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets https://github.com/amznlabs/amazon-dsstne Scott Le Grand Senior Scientist Teza Technologies varelse2005@gmail.com Outline What's Deep Learning? Why GPUs? Deep Learning for

1.55k views • 76 slides
Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser

Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser

Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser , Michaela Blott 27 th March 2019 Outline Why FPGAs? Deep Learning: Challenges & Solutions FINN FPGAs to ACAPs Mega-Trend:

475 views • 31 slides
Structure preservation in (some) deep learning architectures Brynjulf Owren Department of

Structure preservation in (some) deep learning architectures Brynjulf Owren Department of

Structure preservation in (some) deep learning architectures Brynjulf Owren Department of Mathematical Sciences, NTNU, Trondheim, Norway LMS-Bath Symposium 2020 Joint work with: Martin Benning, Elena Celledoni, Matthias Ehrhardt, Christian

805 views • 31 slides
Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th,

Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th,

Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th, 2009, Montreal Main reference: Learning Deep Architectures for AI, Y. Bengio, to appear in Foundations and Trends in Machine Learning ,

1.09k views • 87 slides
Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep

Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep

Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep Learning Research at FAIR Deep Learning on GPUs Deep Learning at scale Emerging Trends Deep Learning Research at Facebook AI Research Image

1.18k views • 50 slides
Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with

Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with

Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with Machine Learning NeurIPS 2020 Thomas Falconer, UCL Energy Institute (Energy & AI Lab) Letif Mones, Invenia Labs Optimal Power Flow (OPF) Challenges

557 views • 14 slides
Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science

Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science

Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science University of Illinois at Urbana-Champaign http://iacoma.cs.uiuc.edu USC October 2016 Accelerated Progress in Transistor Integration Large

867 views • 76 slides
CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al

CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al

CS4501: Introduction to Computer Vision CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al 2014] The Problem: Classification Classify an image into 1000 possible classes: e.g. Abyssinian cat,

673 views • 36 slides
Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George

Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George

1 TTIC_ECP: Deep Epitomic CNNs and Explicit Scale/Position Search Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George Papandreou Iasonas Kokkinos Toyota Technological Institute Ecole Centrale Paris/INRIA

399 views • 23 slides
Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit,

Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit,

Deep learning for natural language processing Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit, LIF/CNRS 23 Feb 2017 Benoit Favre (AMU) DL4NLP: advanced architectures 23 Feb 2017 1 / 29 Deep

526 views • 29 slides
deep architectures for natural language processing Sergey I. Nikolenko 1,2 DataFest 4 Moscow,

deep architectures for natural language processing Sergey I. Nikolenko 1,2 DataFest 4 Moscow,

deep architectures for natural language processing Sergey I. Nikolenko 1,2 DataFest 4 Moscow, February 11, 2017 1 Laboratory for Internet Studies, NRU Higher School of Economics, St. Petersburg 2 Steklov Institute of Mathematics at St. Petersburg

847 views • 68 slides
Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho

Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho

IBM Research AI Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho (minsikcho@us.ibm.com) Deep Learning changing Our Life Automotive/transportation Medicine and Biology Security/public safety Consumer Web,

342 views • 19 slides
Guest Lectures: Bettina Speckmann, Cartography & Flow; Yang Wang, Architectures for Scale.

Guest Lectures: Bettina Speckmann, Cartography & Flow; Yang Wang, Architectures for Scale.

Guest Lectures: Bettina Speckmann, Cartography & Flow; Yang Wang, Architectures for Scale. Example Present: Biomechanical Motion; Proposals Expectations Tamara Munzner Department of Computer Science University of British Columbia CPSC

647 views • 28 slides
Deep In-memory Architectures for NAND Flash Memory Sujan K Gonugondla, Mingu Kang, Yongjune Kim,

Deep In-memory Architectures for NAND Flash Memory Sujan K Gonugondla, Mingu Kang, Yongjune Kim,

Deep In-memory Architectures for NAND Flash Memory Sujan K Gonugondla, Mingu Kang, Yongjune Kim, Naresh Shanbhag University of Illinois at Urbana-Champaign Mark Helm, Sean Eilert Micron Technology, Inc. 1 Machines are Beating Humans at

703 views • 19 slides
Learning Deep Architectures for AI Yoshua Bengio Dept. IRO, Universit e de Montr eal C.P.

Learning Deep Architectures for AI Yoshua Bengio Dept. IRO, Universit e de Montr eal C.P.

1 Learning Deep Architectures for AI Yoshua Bengio Dept. IRO, Universit e de Montr eal C.P. 6128, Montreal, Qc, H3C 3J7, Canada Yoshua.Bengio@umontreal.ca http://www.iro.umontreal.ca/ bengioy To appear in Foundations and Trends in

1.01k views • 71 slides
Parallel Processing of Large-Scale XML-Based Application Documents on Multi-core Architectures

Parallel Processing of Large-Scale XML-Based Application Documents on Multi-core Architectures

Introduction and Motivation Related Work Work Completed Parallel Processing of Large-Scale XML-Based Application Documents on Multi-core Architectures with PiXiMaL Michael R. Head Madhusudhan Govindaraju Department of Computer Science Grid

907 views • 36 slides
CSCE 496/896 Lecture 6: Architectures Stephen Scott Recurrent Architectures Introduction Basic

CSCE 496/896 Lecture 6: Architectures Stephen Scott Recurrent Architectures Introduction Basic

CSCE 496/896 Lecture 6: Recurrent CSCE 496/896 Lecture 6: Architectures Stephen Scott Recurrent Architectures Introduction Basic Idea I/O Mappings Stephen Scott Examples Training (Adapted from Vinod Variyam and Ian Goodfellow) Deep

378 views • 35 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides

More recommend