DRIVING INTELLIGENCE TO THE EDGE FEATURING AN END-TO-END OPEN - PowerPoint PPT Presentation

DRIVING INTELLIGENCE TO THE EDGE – FEATURING AN END-TO-END OPEN ARCHITECTURE FOR IoT Strata Data Conference, September 12, 2018 Dave Shuman, Cloudera, Industry Lead for Manufacturing and IoT Bryan Dean, Red Hat, Director Business Development for IoT Solutions 1

KEY FUNCTIONALITY FOR AN END-TO-END IoT ARCHITECTURE Device Management Intelligent Edge Processing Advanced Analytics Business & Application & Connectivity & Analytics & Machine Learning Integration Securely connect, Apply analytics at the edge with Centralize IoT data processing, Create and deliver cloud-native authenticate and manage machine learning and business analytics and machine learning applications. Integrate business disparate connected devices rules to enable local, low-latency to enable deep business insights applications and processes. that speak different protocols decision making and actionable intelligence End-to-End Security & Compliance Tools to enable end-to-end data security, compliance, authorization and authentication 2

WHY OPEN SOURCE FOR IoT “We believe the best way to support this complex environment is to base our commercial IoT platform, the Bosch IoT Suite, on open source 2.4 30* 250+ 130K components and open standards. These projects establish a horizontal million projects developers monthly open technology for IoT and provide lines of code visitors the technical breeding grounds for successful business ecosystems.” - Dr. Stefan Ferber, VP of Engineering, Bosch Software Innovations 3

ADDRESSING ENTERPRISE IoT NEEDS Operational Technology (OT) Information Technology (IT) Operational Technology (OT) Information Technology (IT) Data Management & Analytics ● Device Management ● Messaging & Integration ● Enterprise Data Mgmt. ● Industrial protocols ● Business Rules & CEP ● Persistent Data Storage ● OT Middleware ● Open Hybrid Platform-as-a-Service ● Big Data Processing & Analytics ● Intelligent gateways ● Enterprise Linux Platform ● Real-Time Analytics ● MQTT co-inventors ● IT security ● Machine Learning ● OT security ● Data Security & Compliance Enterprise IoT open source community 4

OPEN END-TO-END IoT ARCHITECTURE Integrating IoT operating technology, data management, analytics, and applications CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION Cloud-native Traditional DATA MANAGEMENT & apps apps ANALYTICS PLATFORM public, private, hybrid cloud • Modular, secure, end-to-end architecture • Streaming analytics and machine learning • Open, interoperable on hybrid cloud • Modern application development and agile integration 5

EDGE PROCESSING AND ANALYTICS Sensors, Data Integration, Edge Processing Advanced Analytics Application Development, Actuators, Routing, Device & Analytics & Machine Learning Deployment, Integration Command/Control Data Sources CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION DATA MANAGEMENT & ANALYTICS PLATFORM • Device connectivity • Data transformation • Intelligent routing • Edge applications • Edge analytics & real-time decisions 6

DATA INTEGRATION, ROUTING, DEVICE COMMAND/CONTROL Sensors, Data Integration, Edge Processing Advanced Analytics Application Development, Actuators, Routing, Device & Analytics & Machine Learning Deployment, Integration Command/Control Data Sources CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION Telemetry Data DATA MANAGEMENT & ANALYTICS PLATFORM • Device management, security, access control • Data aggregation • Event processing • Agile integration • Container-based application platform 7

ADVANCED ANALYTICS AND MACHINE LEARNING Sensors, Data Integration, Edge Processing Advanced Analytics Application Development, Actuators, Routing, Device Deployment, Integration & Analytics & Machine Learning Command/Control Data Sources CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION • Data ingest • Stream / batch processing • Secure data storage • Machine learning and real-time analytics DATA MANAGEMENT & ANALYTICS PLATFORM Telemetry Data Machine Learning Model 8

APPLICATION DEVELOPMENT, DEPLOYMENT, INTEGRATION Sensors, Data Integration, Edge Processing Advanced Analytics Application Development, Actuators, Routing, Device & Machine Learning Deployment, Integration & Analytics Command/Control Data Sources CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION Cloud-native Traditional DATA MANAGEMENT & apps apps ANALYTICS PLATFORM Application Data • Container-based application platform with Kubernetes orchestration • DevOps: automated build-test-deploy • Agile integration • Multi-cloud portability 9

OPEN END-TO-END IoT ARCHITECTURE CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION Cloud-native Traditional DATA MANAGEMENT & apps apps ANALYTICS PLATFORM • Device connectivity • Device management, • Data ingest • Container-based application platform • Data transformation security, access control • Stream / batch processing with Kubernetes orchestration • Intelligent routing • Data aggregation • Secure data storage • DevOps: automated build-test-deploy • Edge applications • Event processing • Machine learning and • Agile integration • Agile integration • Edge analytics & real-time • Multi-cloud portability real-time analytics decisions • Container-based platform 10

OPEN END-TO-END IoT ARCHITECTURE: PRODUCTS CONNECTED IoT IoT APPLICATION DEVELOPMENT, “THINGS” EDGE INTEGRATION HUB DELIVERY, & INTEGRATION Cloud-native Traditional DATA MANAGEMENT & apps apps ANALYTICS PLATFORM ENTERPRISE DATA HUB 11

END-TO-END ANALYTICS Data flow to derive deep business insights and actionable intelligence P r o t o c o l I n t e l l i g e n t A g g r e g a t i o n R o u t i n g T r a n s l a t i o n F i l t e r i n g APPLICATION DEVELOPMENT, CONNECTED IoT IoT DELIVERY, & INTEGRATION “THINGS” EDGE INTEGRATION HUB Telemetry Data Application Integration Cloud-native Traditional apps apps Application Telemetry Data Management Data D e e p d a t a Real-Time Data Data Ingest Storage Processing a n a l y s i s & Machine Real-Time Data i n s i g h t s Learning Security Analytics DATA MANAGEMENT & ANALYTICS PLATFORM 12

END-TO-END ANALYTICS Data flow to derive deep business insights and actionable intelligence P r o t o c o l I n t e l l i g e n t A g g r e g a t i o n R o u t i n g T r a n s l a t i o n F i l t e r i n g APPLICATION DEVELOPMENT, CONNECTED IoT IoT DELIVERY, & INTEGRATION “THINGS” EDGE INTEGRATION HUB Application Integration Cloud-native Traditional apps apps Application Telemetry Data Data M L M o d e l D e e p d a t a Real-Time Data Data Ingest Storage Processing a n a l y s i s & Actions Machine Machine Real-Time Data i n s i g h t s Learning Security Analytics Learning M L M o d e l P r e d i c t i o n / DATA MANAGEMENT & A l e r t ANALYTICS PLATFORM 13

PUTTING IT INTO PRODUCTION A trained model becomes a “downloadable brain” Connected IoT IoT Data Mgt. and Analytics Hub “Things” Edge Capture Images Report Result Download Model /home/cdsw/trained-model.h5 Predict Locally Red Hat Partner Confidential 14

REAL WORLD DEPLOYMENTS 15

END-TO-END IOT ARCHITECTURE - PoC Application Integration Applications Telemetry Telemetry Data Data Digital Twin IoT Integration Application IoT Edge Data Hub Telemetry Data MQTT over WiFi Command & Control Data Mgt. and Client XDK Analytics Telemetry Data Network: Red Zone Network: Green Zone 16

END-TO-END IOT ARCHITECTURE - PoC IoT Integration Hub Application IoT Edge Integration Apps PPM BI CDSW Device OT Middleware Connectivity Telemetry OT Middleware Telemetry Data Smart Data Services Digital Application Device Twin Data Management Machine Telemetry Learning Data MQTT over WiFi Administration Ingest Analyze Command & Business Logic Control Client XDK Process Store Platform-as-a-Service Telemetry Machine Learning Data Data Mgt. and Analytics Network: Red Zone Network: Green Zone 17

DATA PIPELINE & ARCHITECTURE • Data ingest • Stream / batch processing • Persistent data storage Sentry YARN • Machine learning and real-time analytics Spark IoT Calibrators Edge IoT Site Hub • Device connectivity • Device management, TCP/IP Kafka Spark Kudu • Data transformation Mqtt / OPC- security, and access control TCP/IP • Intelligent routing Kafka OPC- • Data aggregation UA • Business logic TCP/IP mqtt • Event processing • Edge analytics & real-time Impala • Integration services (API’s) UA decisions Store Analyze Integrate Unified Services Layer Data Science Workbench Model Deployment 18

Manifold Pressure 19