iot at scale
play

IoT at Scale or, "From Little Things, Big Data Grow" - PowerPoint PPT Presentation

Jan 23, 2023 •329 likes •1.08k views

Christopher Biggs IoT at Scale @unixbigot @accelerando_au IoT at Scale or, "From Little Things, Big Data Grow" Christopher Biggs, Accelerando Consulting @unixbigot @accelerando_au #YOWData Sept 2017 1 / 74 Christopher Biggs

  1. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au IoT at Scale or, "From Little Things, Big Data Grow" Christopher Biggs, Accelerando Consulting @unixbigot @accelerando_au #YOWData Sept 2017 1 / 74

  2. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Who am I? Christopher Biggs — @unixbigot — @accelerando_au ▷ Brisbane, Australia ▷ 20+ years in IT as developer, architect, manager ▷ Founder, Accelerando Consulting ▷ Accelerando is a "full service" consultancy - chips to cloud ▷ IoT, DevOps, Big † Data † Actual bigness may vary #YOWData Sept 2017 2 / 74

  3. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude From little Things, Big Data Grow "Internet of Things" (IoT) At Scale ▷ Why Even IoT? - my personal motivation ▷ Chips to Cloud - the journey from sensor to senses ▷ The Air Gap - connecting wireless devices to the Internet ▷ Data Tributaries - follow the data from droplet to ocean ▷ Making sense of it all - Managing and visualising IoT data #YOWData Sept 2017 3 / 74

  4. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Overture: Why Even IoT? #YOWData Sept 2017 4 / 74

  5. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture "Don't ever invite a vampire into your house, you silly boy. It renders you powerless." -- "Max", The Lost Boys #YOWData Sept 2017 5 / 74

  6. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture "The 'S' in IoT stands for 'Security'." -- Absolutely Everyone, The Internet #YOWData Sept 2017 6 / 74

  7. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture What is IoT For? #YOWData Sept 2017 7 / 74

  8. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture It's for not this

  9. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: Children #YOWData Sept 2017 9 / 74

  10. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: People of Colour #YOWData Sept 2017 10 / 74

  11. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: Women #YOWData Sept 2017 11 / 74

  12. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: Immigrants #YOWData Sept 2017 12 / 74

  13. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: Foreigners #YOWData Sept 2017 13 / 74

  14. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture People who shouldn't have to do crummy jobs: Robots #YOWData Sept 2017 14 / 74

  15. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Wait, what? "Aren't crummy jobs what robots are for ?" #YOWData Sept 2017 15 / 74

  16. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture "Do you want a war of extinction? Because that's how you get wars of extinction" --Malory Archer, nearly #YOWData Sept 2017 16 / 74

  17. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Arti�cial nonintelligence #YOWData Sept 2017 17 / 74

  18. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Doing all the things no intelligent being should have to be bothered by. #YOWData Sept 2017 18 / 74

  19. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Give me your tired-of-doing-that, your poorly done jobs, your harried masses yearning to be stress-free -- Emma Lazarus, kind of #YOWData Sept 2017 19 / 74

  20. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au IoT's Trinity #YOWData Sept 2017 20 / 74

  21. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Things Start with the simplest parts... #YOWData Sept 2017 21 / 74

  22. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Internet ...connect them together... #YOWData Sept 2017 22 / 74

  23. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Of (i.e. context) ...to form a synergystic whole. #YOWData Sept 2017 23 / 74

  24. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture My Three Laws of IoT ▷ Devices must cooperate for the bene�t of humans ▷ Devices must communicate, and obey instructions ▷ Devices must be as simple and reliable as possible #YOWData Sept 2017 24 / 74

  25. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture OK, end of setup -- "Ford Fairlane", (not just a car, but a really awful movie) #YOWData Sept 2017 25 / 74

  26. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Communications Crossing the Air Gap #YOWData Sept 2017 26 / 74

  27. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Fast, Cheap, Battery-ef�cient. Choose any two. #YOWData Sept 2017 27 / 74

  28. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications PoE (Power-over-Ethernet) Fast, no-battery, expensive wires. #YOWData Sept 2017 28 / 74

  29. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications EoP (Ethernet over Powerline) Cheaper wiring, not so fast. #YOWData Sept 2017 29 / 74

  30. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications PoDS (Plain ol' Dumb Serial) Please, no. #YOWData Sept 2017 30 / 74

  31. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications WiFi (Wireless Made-up-Acronym) #YOWData Sept 2017 31 / 74

  32. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Bluetooth (Don't chew on that Biro) #YOWData Sept 2017 32 / 74

  33. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications ETPH You know, telephones #YOWData Sept 2017 33 / 74

  34. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications LoRa Telephones from 1978, but without the wires. #YOWData Sept 2017 34 / 74

  35. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications IPoSUV #YOWData Sept 2017 35 / 74

  36. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Comms Recap ▷ If you have wires, use 'em ▷ PoDS is not IoT ▷ If you have the power budget for WiFi, great ▷ How about a bluetooth mesh? ▷ "Low-power" equals "really bloody slow" #YOWData Sept 2017 36 / 74

  37. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Structuring for Scalability #YOWData Sept 2017 37 / 74

  38. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Insert magic cloud here #YOWData Sept 2017 38 / 74

  39. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Nope #YOWData Sept 2017 39 / 74

  40. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Edge Computing aka "Not the cloud but we've spent so long plugging cloud we can't possibly admit that" #YOWData Sept 2017 40 / 74

  41. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Web API HTTP is awful on slow links HTTPs is aw-over�owing #YOWData Sept 2017 41 / 74

  42. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Message buses long-lived connection, lightweight messages #YOWData Sept 2017 42 / 74

  43. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Retro�t Websockets #YOWData Sept 2017 43 / 74

  44. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Build your own A bit of orchestration, a bit of docker, a bit of time #YOWData Sept 2017 44 / 74

  45. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Owning an architecture is a big responsibility Maybe you should start with a puppy. #YOWData Sept 2017 45 / 74

  46. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Don't build your own Oh, look, google, amazon and microsoft already did it #YOWData Sept 2017 46 / 74

  47. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Well, they announced it #YOWData Sept 2017 47 / 74

  48. Christopher Biggs — IoT at Scale @unixbigot @accelerando_au Prelude Overture Communications Scalability Microsoft Messaging in place, provisioning "coming soon" #YOWData Sept 2017 48 / 74

Recommend

Scale and level (approach) are different! Approach determines criteria for observation: scale

Scale and level (approach) are different! Approach determines criteria for observation: scale

Scale Phenomena occur at multiple scales Space and Time related Scale of measurement influences what we learn from observation Scale-dependence Levels of Organization criteria for observation Biosphere Community Region Interactions

135 views • 11 slides
New Physics at the TeV Scale? New Physics at the TeV Scale? A Supersymmetric and

New Physics at the TeV Scale? New Physics at the TeV Scale? A Supersymmetric and

FFD SEMINAR 2003 New Physics at the TeV Scale? New Physics at the TeV Scale? A Supersymmetric and Extra-dimensional talk Peter Skands Theoretical High Energy Physics New Physics at the TeV Scale, P . Skands p.1/47 Overview New

830 views • 72 slides
Scale Your Application Per erformance wit ith Web-Scale IT IT A Complimentary Webinar

Scale Your Application Per erformance wit ith Web-Scale IT IT A Complimentary Webinar

Scale Your Application Per erformance wit ith Web-Scale IT IT A Complimentary Webinar From healthsystemCIO.com Sponsored by Nutanix Your Line Will Be Silent Until Our Event Begins at 12:00 ET Thank You! Slide Deck:

270 views • 26 slides
Collaboration at Scale: The Innovation Competency 13-Feb-2019 Collaboration at Scale Designed

Collaboration at Scale: The Innovation Competency 13-Feb-2019 Collaboration at Scale Designed

Collaboration at Scale: The Innovation Competency 13-Feb-2019 Collaboration at Scale Designed for Scrum-centric organizations with more than 10 Scrum teams, the Collaboration at Scale webinar series provides focused, outcome-driven solutions

636 views • 28 slides
TeV-Scale Superpartners with an Unnatural Weak Scale Lawrence Hall University of California,

TeV-Scale Superpartners with an Unnatural Weak Scale Lawrence Hall University of California,

Galileo Galilei Institute Beyond the SM after LHC 8 July 2013 TeV-Scale Superpartners with an Unnatural Weak Scale Lawrence Hall University of California, Berkeley OR Multiverse SUSY Outline 1. High Scale SUSY Hall, Nomura 0910.2235

780 views • 57 slides
kohn kohn architecture 466 AMSTERDAM AVE. STOREFRONT PROPOSAL P-01 EXISTING PHOTOS 4 4 6 6 6

kohn kohn architecture 466 AMSTERDAM AVE. STOREFRONT PROPOSAL P-01 EXISTING PHOTOS 4 4 6 6 6

BROWNSTONE BELTCOURSE TAX PHOTO OF 466 AMSTERDAM AVE. EXISTING BUILDING (GROUND LEVEL) EXISTING STOREFRONT 1 2 3 SCALE : NTS SCALE : NTS SCALE : NTS CORNICE BRACKETS FIRE- BELT-COURSE ESCAPE 9 SCALE : NTS DESIGNATION PHOTO CAST

366 views • 5 slides
Scale Confusion What is Scale? Scale is the size of an object in relation to other objects. What

Scale Confusion What is Scale? Scale is the size of an object in relation to other objects. What

Scale Confusion What is Scale? Scale is the size of an object in relation to other objects. What Proportion Proportion refers to the relative size of parts of a whole (elements within an object).

313 views • 9 slides
FOCUS on the FLAME HSS: Hemispheric stroke scale MADRS: Montgomery Asberg depression rating

FOCUS on the FLAME HSS: Hemispheric stroke scale MADRS: Montgomery Asberg depression rating

Abbreviations Used Disclosures FMMS: Fugl-Meyer motor scale No financial conflicts to disclose FOCUS on the FLAME HSS: Hemispheric stroke scale MADRS: Montgomery Asberg depression rating scale mRS: Modified Rankin Scale

327 views • 8 slides
FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large Scale Solar Lead April 2017 CONTENTS 1. Introduction to CEFC 2. Investment trends 3. The future of large scale solar 4. Pathway to sustainable

664 views • 21 slides
Towards system-scale optimisation of HPC applications TADaaM : Topology-Aware System-Scale Data

Towards system-scale optimisation of HPC applications TADaaM : Topology-Aware System-Scale Data

Towards system-scale optimisation of HPC applications TADaaM : Topology-Aware System-Scale Data Management for High-Performance Computing Applications Emmanuel Jeannot October 2016 INTRODUCTION Optimize application execution at system-scale

336 views • 21 slides
Air gasification of biomass and polyethylene using a staged gasifier in lab scale and pilot scale

Air gasification of biomass and polyethylene using a staged gasifier in lab scale and pilot scale

6th International Conference on Sustainable Solid Waste Management Air gasification of biomass and polyethylene using a staged gasifier in lab scale and pilot scale Speaker: Yong-Seong Jeong Univ. of Seoul Contents 1. Introduction 2. Lab

720 views • 29 slides
From laboratory to industrial scale scale-up calculations of chemical processes for LCA Fabiano

From laboratory to industrial scale scale-up calculations of chemical processes for LCA Fabiano

From laboratory to industrial scale scale-up calculations of chemical processes for LCA Fabiano Piccinno 1,2 Roland Hischier 1 , Stefan Seeger 2 , Claudia Som 1 1 Empa, Technology and Society Lab, St. Gallen, Switzerland 2 University of Zurich,

367 views • 23 slides
What are Scale Ups? 20% growth by turnover/employees Ambition or potential to grow by 50%

What are Scale Ups? 20% growth by turnover/employees Ambition or potential to grow by 50%

What are Scale Ups? 20% growth by turnover/employees Ambition or potential to grow by 50% over next 3 years All sectors and geographies UK 3 rd for Start Ups but 13 th for Scale Ups 1% Growth in Scale Ups = 150bn Why Scale

338 views • 10 slides
Leonardo Lima FSC Webinar May 2018 Using Using our Scale our Scale for Good or Good G

Leonardo Lima FSC Webinar May 2018 Using Using our Scale our Scale for Good or Good G

Leonardo Lima FSC Webinar May 2018 Using Using our Scale our Scale for Good or Good G L O B A L Goals for 2022-2030 P R I O R I T I E S Commitmen Commitment t to to Bee Beef Pac acka kaging ging You outh th Famili

342 views • 13 slides
Southwest Anthony Henday Drive Noise Study 1 Introduction to Sound 2 Decibel Scale Noise is

Southwest Anthony Henday Drive Noise Study 1 Introduction to Sound 2 Decibel Scale Noise is

Southwest Anthony Henday Drive Noise Study 1 Introduction to Sound 2 Decibel Scale Noise is measured using a Decibel (dB) Scale The Decibel Scale is a base-10 logarithm scale (similar to Richter Scale) Change of 1-2 dB - threshold

409 views • 26 slides
MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket @objectrocket @kennygorman MongoDB at scale Designing for scale Techniques to ease pain Things to avoid What is scale? Scale;

539 views • 27 slides
liver ver sc scan ans s vi via a a re a recu current rrent multi ti-scale scale en enco

liver ver sc scan ans s vi via a a re a recu current rrent multi ti-scale scale en enco

Sp Spati tio-te tempo mporal ral mot otion on predic diction tion in free-bre breathing athing liver ver sc scan ans s vi via a a re a recu current rrent multi ti-scale scale en enco coder er dec ecod oder er Liset

673 views • 7 slides
emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network scale Regional scale

emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network scale Regional scale

Project Team: Gabriel Cira, AIA - Very Flat, Architecture Nick Lutsko - MIT EAPS, Atmospheric Dynamics Julia Hopkins - TU Delft & NEU, Coastal Engineering emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network

401 views • 5 slides
Hydro Lab Data Collection Ph Scale The pH scale measures how acidic or basic a substance is. The

Hydro Lab Data Collection Ph Scale The pH scale measures how acidic or basic a substance is. The

Hydro Lab Data Collection Ph Scale The pH scale measures how acidic or basic a substance is. The pH scale ranges from 0 to 14. A pH of 7 is neutral. A pH less than 7 is acidic. A pH greater than 7 is basic . Dissolved Oxygen The measure of how

356 views • 21 slides
Hint: Look at how much If you are given a each side has increased shape and a scale by.

Hint: Look at how much If you are given a each side has increased shape and a scale by.

Home Learnin ing: To solve ve Read the reminders below on finding the problems ems invol olvin ving the scale e scale factor before attempting the tasks on the factor. . next slide. Hint: Look at how much If you are given a each side

322 views • 9 slides
Survey: need for small-scale Estonian Latvian cooperation projects 1 Small-scale project u

Survey: need for small-scale Estonian Latvian cooperation projects 1 Small-scale project u

12/5/19 Survey: need for small-scale Estonian Latvian cooperation projects 1 Small-scale project u Budget up to 50 000 EUR u Simple application and reporting 2 1 12/5/19 Small-scale projects in EstLat Programme u 2004-2006: small

382 views • 5 slides
Introducing The Platform Marketer John Lee 1 Addressability at scale (AAS) Addressability At

Introducing The Platform Marketer John Lee 1 Addressability at scale (AAS) Addressability At

Introducing The Platform Marketer John Lee 1 Addressability at scale (AAS) Addressability At Scale Addressability at scale (AAS) is the application of data and analytics to drive to highly efficient individual level targeting and

640 views • 39 slides
INCORPORATING LARGE-SCALE CITIZEN INCORPORATING LARGE-SCALE CITIZEN DELIBERATION INTO

INCORPORATING LARGE-SCALE CITIZEN INCORPORATING LARGE-SCALE CITIZEN DELIBERATION INTO

INCORPORATING LARGE-SCALE CITIZEN INCORPORATING LARGE-SCALE CITIZEN DELIBERATION INTO ENVIRONMENTAL CONFLICT RESOLUTION America Speaks presentation to the 2008 ECR Conference Table Introductions Please form groups of 4-5 and give: Your name

848 views • 41 slides
Large-Scale Survey Interviewing Following Large Scale Survey Interviewing Following the 2008

Large-Scale Survey Interviewing Following Large Scale Survey Interviewing Following the 2008

Large-Scale Survey Interviewing Following Large Scale Survey Interviewing Following the 2008 WenChuan Earthquake Zhang Huafeng and Jon Pedersen International Blaise Users Conference (IBUC) Baltimore, USA , 1 Introduction Two household

712 views • 22 slides

More recommend