machine learning pipeline wit ith r
play

Machine Learning pipeline wit ith R Contributions to whiteboxing - PowerPoint PPT Presentation

May 02, 2023 •274 likes •858 views

Towards an Enterprise grade Machine Learning pipeline wit ith R Contributions to whiteboxing machine learning for interoperation with production environments Thomas.Strehl@at.ibm.com Thomas.Weinrich@at.ibm.com Rudolf.Pailer@at.ibm.com @20200120

  1. Towards an Enterprise grade Machine Learning pipeline wit ith R Contributions to whiteboxing machine learning for interoperation with production environments Thomas.Strehl@at.ibm.com Thomas.Weinrich@at.ibm.com Rudolf.Pailer@at.ibm.com @20200120

  2. REnterprise: Machine Learning with R in the Enterprise MLOps Whiteboxing ML for Interoperation with production environments • Enterprise Environment • Processes, Governance, Architecture, Security • Requirements, Develop, Test, Release Management, Rollout • Documentation, Incident Management • CRISP-DM + MLOps -> CrispML • Standard process for Data Mining related projects • DevOps Automation for Machine Learning • -> Service orientated Architecture for data preparation, training and scoring • Demo: rep-admin + rep-crispml • CrispML demo implementation on kubernetes • Automated ML pipeline for R Thomas.strehl@at.ibm.com Stand 20200120

  3. REnterprise: Classical Software Factory Lineup Development Environment Test Environment Production Environment Requirements Development Test Production Solution Design Governance Architecture Board Scrum, Standups Defect Management Incident Management Pipeline Jira + Confluence SCM: BitBucket Build: Jenkins Artifacts: Artifactory Deploy: Jenkins Test Test Management Unit: JUnit Service: SoapUI Functional: Tosca, Selenium Data Test Data Management Production specific samples semi-realistic Infrastructure Automation Config Management Release Management Monitoring: Dynatrace Logging: Splunk Thomas.strehl@at.ibm.com Stand 20200120

  4. REnterprise: Classical Machine Learning Pipeline CRISP-DM (Cross InduStry Process for Data Mining) • Business understanding • Data understanding • Data preparation • Modeling • Evaluation • Deployment • Devised in late 1990 • Used by around 45% of data projects Thomas.strehl@at.ibm.com Stand 20200120

  5. REnterprise: ML interaction with the Enterprise MLOps Interaction of Enterprise services with ML services • Data Preperation + Training • Mass Data: FileSystem, DataBase, DWH, DataLake , Data Platform, … • Scoring • Batch Scoring: e.g. Rscript, REST • Record Scoring: e.g REST • Governance • Reporting, Statistics, Performance • Documentation, Changes, Defects, Incidents Thomas.strehl@at.ibm.com Stand 20200120

  6. REnterprise: CRISP-DM generic Interfaces MLOps CrispML: Implementation of 9 methods for Training and Scoring • Training and Scoring • DataIngestion: Raw Data • DataPreparation: Algorithm independent • DataCuration: Algorithm specific • Training • ModelTraining: Algorithm, Hyperparameters • ModelReport : ML KPI’s • ModelPersist: Model Registry • Scoring • NewDataScore: persist each score with reference to metadata • NewDataLabel: import new ground truth • NewDataReport: verify new ground truth against persisted score Thomas.strehl@at.ibm.com Stand 20200120

  7. REnterprise: CrispML Components MLOps CrispML: Training and Scorings Servers and Clients Training - Area Scoring - Area CrispML-App Database CrispML-App plumber plumber trainer REST client Data Ingestion Data Ingestion Data Preperation Data Preperation Data Curation Data Curation Train Model Train Model Storage Persist Model Persist Model Score NewData Score NewData Labels NewData Labels NewData Verify NewData Verify NewData Thomas.strehl@at.ibm.com Stand 20200120

  8. REnterprise: CrispML Big Picture MLOps Orchestration: Run ML cycle including training and verification in place in each enviroment Deployment: Stage ML functionality across environmnets Complete pipeline subject to QA Fallback: Use model trained in QA GitLab All ML functionality in R package Development Environment QA Environment Production Environment R-Package Training Environment Training Environment Data Ingestion Data Ingestion Data Ingestion Data Ingestion Data Preperation Data Preperation Data Preperation Verify NewData Data Preperation Data Curation Data Curation Data Curation Train Model Train Model Train Model Labels NewData Data Curation Persist Model Persist Model Persist Model Scoring Environment Scoring Environment Score NewData Score NewData Train Model Score NewData Score NewData Labels NewData Labels NewData Labels NewData Persist Model Verify NewData Verify NewData Verify NewData Data: Access data pool shared across environments (optional GDPR filters) Thomas.strehl@at.ibm.com Stand 20200120

  9. REnterprise: CrispML implementation in R MLOps Crispml: Selfcontained, standalone, scalable REST Docker container • CrispML • All CRISP-DM methods designed for remote invocation • REST interface to ingestion, training, scoring • Plumber (other options: openCPU, rserver) • Admin Console • Lightweight demo implementation of remote control • Shiny GUI app • Challenge: no direct access to data, only via REST • Runtime Environment • Linux (any R platform), Kubernetes, … Thomas.strehl@at.ibm.com Stand 20200120

  10. REnterprise: Demo based on DevOps and Containers MLOps DevOps, Cloud • ‘Traditional’ platform • E.g. Bitbucket, Jira, Confluence, Jenkins, Artifactory, VMware • DevOps -> MLOps • Automatisierung der Pipeline: Gitlab, Tekton • Containerized • Docker, Kubernetes • IBM Cloud • Gitlab, Tekton, Kubernetes, logDNA, sysDIG, DB2 Thomas.strehl@at.ibm.com Stand 20200120

  11. Why DevOps – Traditional software delivery lifecycle Plan Develop Test Release Business Owner Require- Code Unit, UAT,.. Production Customer ments Failures due to Bottlenecks trying to Complex, manual, Poor visibility into inconsistent dev and deliver frequent processes for release dependencies across production releases to meet lack repeatability and releases, resources, environments market demands speed and teams

  12. Why DevOps – Transforming the software delivery lifecycle Plan Develop Test Release Business Owner Require- Code Unit, UAT,.. Production Customer ments Agility & Flexibility Standardization Fail fast & Fail early Automation

  13. DevOps: Continuous flow in Enterprise systems DevOps Practice Areas 3 DevOps dimensions 6 DevOps practice areas 4 DevOps software lifecycle DevOps Dimensions People Processes Technology

  14. DevOps Principles: Continuous everything Dashboard everything Automate everything Test everything Collaboration for speed Continuous monitoring Continuous Delivery Continuous testing • • • Collaborative steering • Visibility to the teams Continuous Integration Test automation • • • Collaborative Dev-Ops • Infra as Code • Feedback loops • Monitor and audit everything Continuous monitoring • Logging and monitoring •

  15. DevOps: Automation, automation, automation  If someone has to do the same thing more than once, it’s a candidate for automation  If something is hard, do it repeatedly  Develop and Test against production-like systems  Iterative and frequent deployments using repeatable and reliable processes  Continuously monitor and validate operational quality characteristics  Encourage a culture of experimentation and valuing team improvement • Minimizing business risk – fail small and fast • All DevOps principles also apply to MLOps -> CrispMl approach

  16. REnterprise: Containerizing: Docker + Kubernetes MLOps CrispML: • Docker • Lightweight variant of virtual server • Start from downloadable template and enhance along ‘ Dockerfile ’ • Persist as ‘image’ and instantiate as ‘container’ • Template images available e.g. for ‘ rshiny ’ and ‘plumber’ applications • Kubernetes • Orchestrator for containerized applications • Scaling, Loadbalancing , System Monitoring, Storage, Network, … Thomas.strehl@at.ibm.com Stand 20200120

  17. thomas.strehl@at.ibm.com @20200202 REnterprise: Scaling R on Kubernetes Scaling by future/promise Scaling by ingress alb for Monitoring of node Scaling by service within container different applications by Daemonsets across pods Kubernetes cluster Ingress ALB nginx Service AppX Service AppY Workers Nodes Nodes Pod 1 Pods Pod 1 sysDIG, logDNA Pods Storage Containers Containers Containers Containers plumber plumber-main plumber plumber-main sidecar sidecar plumber-child plumber-child plumber-child plumber-child Pod n Pod n sidecar sidecar Containers Containers plumber plumber Pod Pod sidecar sidecar DaemonSet DaemonSet Thomas.strehl@at.ibm.com Stand 20200120

  18. REnterprise: Performance Testing MLOps CrispML: loadimpact/k6 -> influxdb -> grafana • Many options • jmeter, Locust(python), grinder(java), Gatling(scala ), …………….. • loadimpact/k6 • java script, 3000+ stars on GitHub • Writes to influxdb, prebuilt Grafana dashboards, invoked as container Thomas.strehl@at.ibm.com Stand 20200120

  19. REnterprise: Database and Persistent storage MLOps CrispML: odbc, DBI, dbplyr -> DB2 (requires OS level driver) • Peformance sensitive • Throughput depends on network latency • R-Packages • pool: DB connection pool • dbplyr: Execute dataframe operations in DB CrispML: Kubernetes Persistent Volume Claim • Kubernetes file system, DWH, Data Lake, Data Platform • Persist results (models, parameters, …) • Persist state across instances of R processes on different pod/nodes Thomas.strehl@at.ibm.com Stand 20200120

Recommend

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

404 views • 26 slides
Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning Pipeline Learning Replicate Algorithm model Data Trained Serve Model Model Repeat entire pipeline Scaling Machine Learning Datasets and

890 views • 21 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni

AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni

AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni de Paula Lourenco, Jorge Piazentin Ono, Kyunghyun Cho, Claudio Silva, Juliana Freire International Workshop on Automatic Machine Learning, ICML,

421 views • 26 slides
AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni

AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni

AlphaD3M Machine Learning Pipeline Synthesis Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni de Paula Lourenco, Jorge Piazentin Ono, Kyunghyun Cho, Claudio Silva, Juliana Freire International Workshop on Automatic Machine Learning, ICML,

741 views • 28 slides
Spark: A Coding Joyride Visualizing and plotting data Create a machine-learning pipeline

Spark: A Coding Joyride Visualizing and plotting data Create a machine-learning pipeline

11/16/2015 Objectives Show Spark's ability to rapidly process Big Data Extracting information with RDDs Querying data using DataFrames Spark: A Coding Joyride Visualizing and plotting data Create a machine-learning pipeline

173 views • 6 slides
An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
A Machine Learning Pipeline for Drought Prediction Tommy Lees , Gabriel Tseng , Alex

A Machine Learning Pipeline for Drought Prediction Tommy Lees , Gabriel Tseng , Alex

A Machine Learning Pipeline for Drought Prediction Tommy Lees , Gabriel Tseng , Alex Hernandez-Garcia, Clement Atzberger, Simon Dadson, Steven Reece @tommylees112, @gabrieltseng tommylees112, gabrieltseng Agricultural drought is a significant

291 views • 11 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

564 views • 17 slides
Introduction to Machine Learning Linear Regression Prof. Andreas Krause Learning and Adaptive

Introduction to Machine Learning Linear Regression Prof. Andreas Krause Learning and Adaptive

Introduction to Machine Learning Linear Regression Prof. Andreas Krause Learning and Adaptive Systems (las.ethz.ch) Basic Supervised Learning Pipeline Training Data Test Data spam ? Learning Predic- ham Model method ?

566 views • 23 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

878 views • 29 slides
CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

1/22/19 CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine Learning? A Simple Task: Recognize Obama How do you program a computer to Recognize faces? Recommend movies? Decide which web

583 views • 5 slides
Machine Learning Problem Framework Presenters: Bikramjeet Singh(20752928) Priyansh Narang

Machine Learning Problem Framework Presenters: Bikramjeet Singh(20752928) Priyansh Narang

Machine Learning Problem Framework Presenters: Bikramjeet Singh(20752928) Priyansh Narang (20716980) Agenda Background research Brief introduction to Machine Learning ML Problem: Formulation ML Pipeline Questions

924 views • 39 slides
Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their performance We use machine learning in situations where it is very challenging (or impossible) to define the rules by hand: e.g. face detection

1.05k views • 28 slides
Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Introduction to Machine Learning 1 / 12 Outline 1 Nearest Neighbor Prediction 2 Complexity Considerations 3 The Voronoi Diagram 4 Overfitting

569 views • 12 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

JACOPO DI IORIO, MATTEO FONTANA, DANIELE OCCHIUTO, CLAUDIA VOLPETTI MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING STATISTICAL LEARNING Separate evolutions, but with shared methodologies MERGE

565 views • 55 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine learning refers to any system where the performance of a machine in performing a task improves by gaining more experience in performing that task .

396 views • 27 slides
MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How to separate the red class from the grey class? x 2 360 r x 1 Polar coordinates Data

1.04k views • 44 slides
Pipeline MACH IN E LEARN IN G W ITH P YS PARK Andrew Collier Data Scientist, Exegetic Analytics

Pipeline MACH IN E LEARN IN G W ITH P YS PARK Andrew Collier Data Scientist, Exegetic Analytics

Pipeline MACH IN E LEARN IN G W ITH P YS PARK Andrew Collier Data Scientist, Exegetic Analytics Leakage? Only for training data. For testing and training data. MACHINE LEARNING WITH PYSPARK A leaky model MACHINE LEARNING WITH PYSPARK A

816 views • 50 slides
Learning To Grasp Jake Varley Overview - What is a grasping pipeline? - A current grasping

Learning To Grasp Jake Varley Overview - What is a grasping pipeline? - A current grasping

Learning To Grasp Jake Varley Overview - What is a grasping pipeline? - A current grasping pipeline - Recent trends in related fields - A future grasping pipeline A Grasping Pipeline Data Driven Grasp Synthesis - a survey 2014

900 views • 44 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides

More recommend