ensemble learning with sagemaker and step functions
play

Ensemble Learning with Sagemaker and Step-Functions Dr. Benjamin - PowerPoint PPT Presentation

Dec 02, 2022 •175 likes •678 views

Ensemble Learning with Sagemaker and Step-Functions Dr. Benjamin Weigel | 09.09.2019 Hamburg, Germany Benjamin Weigel Data Engineer & Cloud Coordinator Europace AG https://www.europace.de/ There is manual efgort in obtaining a mortgage

  1. Ensemble Learning with Sagemaker and Step-Functions Dr. Benjamin Weigel | 09.09.2019 Hamburg, Germany

  2. Benjamin Weigel Data Engineer & Cloud Coordinator Europace AG https://www.europace.de/

  4. There is manual efgort in obtaining a mortgage

  5. Smart Document Classification

  6. Text Image Smart Model Model use output as input Document trained on trained on Classification OCR-extracted text page-bitmap Sequence Model trained on sequence information (i.e. “Page 1-4 is a contract”)

  7. Options to Build a Model Training-Pipeline

  8. AWS Sagemaker

  9. AWS Step-Functions - define a distributed workflow as series of steps - visual workflow - long running workflows (max 1yr) - 4000 transitions/month for free - after: 0.025 USD per 1000 transitions - can get expensive quickly

  10. AWS Step-Functions

  11. { "Comment" : "An example of the Amazon States Language." , "StartAt" : "FirstState" , "States" : { "FirstState" : { "Type": "Task", Amazon States "Resource" : "arn:aws:lambda:us-east-1:123456789012:function:..." , "Next": "ChoiceState" }, ... Language } } "ChoiceState": { "Type": "Choice", define a state-machine "Choices": [ ● { "Variable": "$.foo", JSON-based ● "NumericEquals": 1, "Next": "FirstMatchState" describe: }, ● { "Variable": "$.foo", a state and the ○ "NumericEquals": 2, "Next": "SecondMatchState" transition to the next } ], "Default": "DefaultState" error-conditions etc. ○ }

  12. Step Functions can control Sagemaker ● Transform and Training Jobs directly via these Resources: Step Functions & "arn:aws:states:::sagemaker:createTransformJob.sync" Sagemaker "arn:aws:states:::sagemaker:createTrainingJob.sync" easy peasy https://docs.aws.amazon.com/step-functions/latest/dg/connect-sagemaker.html

  13. "Image Model Training" : { "Type" : "Task", "Resource" : "arn:aws:states:::sagemaker:createTrainingJob.sync", "Parameters" : { "TrainingJobName" : "ImageModel", "AlgorithmSpecification" : { "TrainingImage" : "520713654638.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-mxnet:1.3-gpu-py3", "TrainingInputMode" : "File" }, "HyperParameters" : { sagemaker: "epochs" : "80", "batch_size" : "10", "conv_block_length" : "2", "cycle_length" : "10", createTrainingJob.sync "depth" : "5", "dropout" : "0.5", "max_lr" : "0.1", "min_lr" : "0.0001", ... "start_filter" : "4", "worker" : "4" ● configure job via Parameters }, "InputDataConfig.$" : "$.generated.image_model.InputDataConfig", "OutputDataConfig" : { section "S3OutputPath.$" : "$.generated.output_artifact_paths.image_model_prefix" }, "ResourceConfig" : { "InstanceCount" : 4, "InstanceType" : "ml.p2.xlarge", "VolumeSizeInGB" : 10 }, "RoleArn" : "arn:aws:iam::123456789012:role/sm-stepfunction-iam-role", "StoppingCondition" : { "MaxRuntimeInSeconds" : 172800 } } } https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html#API_CreateTrainingJob_RequestSyntax https://docs.aws.amazon.com/step-functions/latest/dg/connect-sagemaker.html

  14. The Good Photo by Joshua Ness on Unsplash

  15. Start simple { "StartAt" : "Train Text Model", "States" : { "Train Text Model" : { "Type" : "Task", "Resource" : "arn:aws:states:::sagemaker:createTrainingJob.sync", "Parameters" : { ... }, "End" : true } } }

  16. Expand from there { "StartAt" : "Fetch Preprocessed Data" , "States" : { "Fetch Preprocessed Data" : { "Type" : "Task", "Resource" : "arn:aws:states:::batch:submitJob.sync" , "Next" : "Train Text Model" , "Parameters" : { "JobName" : "FetchPreparedData" , "JobDefinition" : "arn:aws:batch: us-east-1:1234567890:job-definition/job:2 ", "JobQueue" : "arn:aws:batch:us-east-1:1234567890:job-queue/queue" , "Parameters" : { "DATA_INPUT_PATH.$" : "$.input_data" , "OUTPUT_PATH.$" : "$.ready_to_use_artifacts" } } }, "Train Text Model" : { ... } } }

  17. Retry if possible "Train Text Model" : { "Type" : "Task", "Resource" : "arn:aws:states:::sagemaker:createTrainingJob.sync", ... "Retry" : [ { "ErrorEquals" : [ "SageMaker.AmazonSageMakerException" ], "IntervalSeconds" : 1, "MaxAttempts" : 100, "BackoffRate" : 1.1 }, ... ] }

  18. If all else fails ... "Train Text Model" : { "Type" : "Task", ... "Catch" : [{ "ErrorEquals" : ["States.ALL" ], "Next" : "Notify Failure" }] }, "Notify Failure" : { "Type" : "Task", "Resource" : "arn:aws:states:::sns:publish" , "End" : true, "Parameters" : { "Subject" : "[ERROR] Model Training failed!" , "Message" : "Error during model training!" , "TopicArn" : "arn:aws:sns:*:123456789012:alerting_topic" , "MessageAttributes" : { ... } } }

  19. But there is a catch ...it’s a valid state after all

  20. Fail successfully! "Notify Failure" : { "Type" : "Task", "Resource" : "arn:aws:states:::sns:publish" , "Next" : "Fail", ... }, "Fail" : { "Type" : "Fail" }

  21. Text Image Model Model use output as input Add a few trained on trained on more models ... OCR-extracted text page-bitmap Sequence Model trained on sequence information (i.e. “Page 1-4 is a contract”)

  22. Use concurrency for time effjciency "Fetch Preprocessed Data" : { ... "Next" : "Base Model Training" }, "Base Model Training" : { "Type" : "Parallel" , "Next" : "Train Sequence Model" , "Branches" : [ { "StartAt" : "Train Image Model" , "States" : { "Train Image Model" : { ... "End" : true } } },{ "StartAt" : "Train Text Model" , "States" : { "Train Text Model" : { ... "End" : true }}}]},

  23. Beware of “silent” errors notification trigger won’t fire because there is no state defined for this scenario -> unexpected failure

  24. Everything should fail the same "Base Model Training" : { "Type" : "Parallel", "Next" : "Train Sequence Model" , "Branches" : [...], "Catch" : [ { "ErrorEquals" : [ "States.ALL" ], "Next" : "Notify Failure" } ] }

  25. Some jobs are long running and expensive then something fails and you have to debug (rerun) …

  26. Save time & money skip some steps... "States" : { "Skip Image Model Training?" : { "Type" : "Choice" , "Choices" : [ { "Variable" : "$.train_image_model" , "BooleanEquals" : false, "Next" : "Skip Fetch Preprocessing Artifacts" } ], "Default" : "Train Image Model" }, "Skip Fetch Preprocessing Artifacts" : { "Type" : "Pass", "End" : true }, "Train Image Model" : { ... "End" : true } }

  27. Rinse and repeat ...and add a little sprinkle on top

  28. Our Model Training Workflow - Lambda - Batch Job - Sagemaker - SNS - Choice - Pass (to skip steps) - Fail - Wait

  29. Our Model Training “Setup” Data (S3) Input Workflow - Input to setup state Step Function machine execution - define where the data is - (Hyper)Parameterization - Data & Models stored on S3 (each execution gets its own copy of the data) Models & Data (S3)

  30. The Bad Photo by Markus Spiske on Unsplash

Recommend

Ensemble Learning 4/10/17 Ensemble Learning Hypothesis Space: Supervised learning (data has

Ensemble Learning 4/10/17 Ensemble Learning Hypothesis Space: Supervised learning (data has

Ensemble Learning 4/10/17 Ensemble Learning Hypothesis Space: Supervised learning (data has labels) Classification (labels are discrete) Also regression, but the algorithms differ. The type of mapping that can be learned depends

377 views • 11 slides
Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble models) LEARNING PERFORMANCE REPRESENTATION DATA PROBLEM RAW DATA CLUSTERING EVALUATION FEATURES UCI datasets unigrams 20newsgroups SUPERVISED

926 views • 34 slides
One Year with Sagemaker Our experience of enhancing SaaS products based on custom Deep Learning

One Year with Sagemaker Our experience of enhancing SaaS products based on custom Deep Learning

One Year with Sagemaker Our experience of enhancing SaaS products based on custom Deep Learning Dennis Weyland econda GmbH | September 9, 2019 Community Day 2019 Sponsors Dennis Weyland Big Data & Artificial Intelligence Realtime

2.96k views • 27 slides
Ensemble Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November

Ensemble Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November

Ensemble Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 28, 2017 Prof. Michael Paul Ensemble Learning General idea: Combine multiple classifiers (usually with different strengths) to build a bigger,

2.05k views • 19 slides
Ensemble Kalman Filtering with One-Step-Ahead (OSA) smoothing: Application to State-Parameter

Ensemble Kalman Filtering with One-Step-Ahead (OSA) smoothing: Application to State-Parameter

Ensemble Kalman Filtering with One-Step-Ahead (OSA) smoothing: Application to State-Parameter Estimation and 1-Way Coupled Models Naila Raboudi, Boujemaa Ait-El-Fquih, Ibrahim Hoteit King Abdullah University of Science and Technology Earth

718 views • 46 slides
CSI5180. MachineLearningfor BioinformaticsApplications Ensemble Learning by Marcel Turcotte

CSI5180. MachineLearningfor BioinformaticsApplications Ensemble Learning by Marcel Turcotte

CSI5180. MachineLearningfor BioinformaticsApplications Ensemble Learning by Marcel Turcotte Version December 5, 2019 Preamble Preamble 2/50 Preamble Ensemble Learning In this lecture, we consider several meta learning algorithms all based

1.11k views • 92 slides
Class Imbalance Multiclass Problems General Idea Original D Training data .... Step 1:

Class Imbalance Multiclass Problems General Idea Original D Training data .... Step 1:

Ensemble Learning Class Imbalance Multiclass Problems General Idea Original D Training data .... Step 1: Create Multiple D 2 D 1 D t-1 D t Data Sets Step 2: Build Multiple C 1 C 2 C t -1 C t Classifiers Step 3: Combine C *

865 views • 35 slides
Ensemble Learning Machine Learning Introduction 2 In our daily life Asking different

Ensemble Learning Machine Learning Introduction 2 In our daily life Asking different

Machine Learning 1 Ensemble Learning Machine Learning Introduction 2 In our daily life Asking different doctors opinions before undergoing a major surgery Reading user reviews before purchasing a product There are countless

695 views • 22 slides
ensemble Learning Instructor: Prof. Shuai Huang Industrial and Systems Engineering University of

ensemble Learning Instructor: Prof. Shuai Huang Industrial and Systems Engineering University of

Lecture 17: AdaBoost and ensemble Learning Instructor: Prof. Shuai Huang Industrial and Systems Engineering University of Washington Random forest is one example Ensemble learning is very flexible. It is a hodgepodge of models, combined in

436 views • 5 slides
Pruning an ensemble of classifiers via reinforcement learning Authors : Ioannis Partalas,

Pruning an ensemble of classifiers via reinforcement learning Authors : Ioannis Partalas,

Pruning an ensemble of classifiers via reinforcement learning Authors : Ioannis Partalas, Grigorios Tsoumakas, Ioannis Vlahavas Journal : Neurocomputing 72 (2009) 1900-1909 Presentation : Jose Manuel Lopez Guede Introduction I Ensemble: a

399 views • 39 slides
BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging

BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging

Photo byUnsplash user @nathananderson BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging Random Forests Aykut Erdem // Hacettepe University // Fall 2019 Last time Decision Trees slide by David Sontag

681 views • 38 slides
CS480/680 Lecture 22: July 22, 2019 Ensemble Learning [RN] Sec. 18.10, [M] Sec. 16.2.5, [B]

CS480/680 Lecture 22: July 22, 2019 Ensemble Learning [RN] Sec. 18.10, [M] Sec. 16.2.5, [B]

CS480/680 Lecture 22: July 22, 2019 Ensemble Learning [RN] Sec. 18.10, [M] Sec. 16.2.5, [B] Chap. 14, [HTF] Chap 15-16, [D] Chap. 11 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Outline Ensemble Learning Bagging

1.72k views • 22 slides
Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL,

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL,

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL, University of Ulster 1 Ensemble learning Ensemble learning l Postulate multiple hypotheses to explain the data l Shortcomings of single model learning

681 views • 19 slides
An ensemble-based kernel le learning framework to handle data assimilation problems wit ith im

An ensemble-based kernel le learning framework to handle data assimilation problems wit ith im

An ensemble-based kernel le learning framework to handle data assimilation problems wit ith im imperfect forw rward simulators Xiaodong Luo, NORCE Outline Background and motivation An ensemble-based kernel algorithm for supervised

515 views • 35 slides
Pseudorandom Generators from Regular One-way Functions: New Constructions with Improved Parameters

Pseudorandom Generators from Regular One-way Functions: New Constructions with Improved Parameters

Pseudorandom Generators from Regular One-way Functions: New Constructions with Improved Parameters Yu Yu Joint work with Xiangxue Li and Jian Weng Asiacrypt 2013 One-way Functions One-way functions are an ensemble of functions ( ) n l

214 views • 20 slides
3. Preference Learning Techniques a. Learning Utility Functions b. Learning Preference

3. Preference Learning Techniques a. Learning Utility Functions b. Learning Preference

AGENDA 1. Preference Learning Tasks 2. Performance Assessment and Loss Functions 3. Preference Learning Techniques a. Learning Utility Functions b. Learning Preference Relations c. Structured Output Prediction d. Model-Based Preference

583 views • 39 slides
When Semi-Supervised Learning Meets Ensemble Learning Zhi-Hua Zhou http://cs.nju.edu.cn/zhouzh/

When Semi-Supervised Learning Meets Ensemble Learning Zhi-Hua Zhou http://cs.nju.edu.cn/zhouzh/

http:/ / lam da.nju.edu.cn When Semi-Supervised Learning Meets Ensemble Learning Zhi-Hua Zhou http://cs.nju.edu.cn/zhouzh/ Email: zhouzh@nju.edu.cn LAMDA Group National Key Laboratory for Novel Software Technology, Nanjing University, China

838 views • 64 slides
Pareto Optimal Streaming Unsupervised Ensemble Learning Soumya Basu University of Texas at

Pareto Optimal Streaming Unsupervised Ensemble Learning Soumya Basu University of Texas at

Pareto Optimal Streaming Unsupervised Ensemble Learning Soumya Basu University of Texas at Austin Steven Gutstein (ARL), Brent Lance (ARL), and Sanjay Shakkottai (UT Austin) Poster # 178 Streaming Unsupervised Ensemble Learning Po Poster #178

508 views • 3 slides
Overview of Decision Trees, Ensemble Methods and Reinforcement Learning CMSC 678 UMBC Outline

Overview of Decision Trees, Ensemble Methods and Reinforcement Learning CMSC 678 UMBC Outline

Overview of Decision Trees, Ensemble Methods and Reinforcement Learning CMSC 678 UMBC Outline Decision Trees Ensemble Methods Bagging Random Forests Reinforcement Learning Decision Trees 20 Questions: http://20q.net/ Goals: 1.

771 views • 62 slides
Online Learning with Pairwise Loss Functions Online Learning with Pairwise Loss Functions MLSIG

Online Learning with Pairwise Loss Functions Online Learning with Pairwise Loss Functions MLSIG

Online Learning with Pairwise Loss Functions Online Learning with Pairwise Loss Functions MLSIG Seminar Series, Dept. of CSA, IISc Joint work with B. Sriperumbudur, P. Jain, H. Karnick Purushottam Kar MLO Group, Microsoft Research India Outline

464 views • 28 slides
An Ensemble of Face Recognition Algorithms for Unsupervised Expansion of Training Data

An Ensemble of Face Recognition Algorithms for Unsupervised Expansion of Training Data

An Ensemble of Face Recognition Algorithms for Unsupervised Expansion of Training Data Anthony Clark anthonyclark@missouristate.edu Jeff Dale jdale@ieee.org Motivation Security Smart Surveillance Deep Learning Small Data Ensemble

617 views • 16 slides
(1, )-Evolution Strategy with Cumulative Step size Adaptation on linear cost functions

(1, )-Evolution Strategy with Cumulative Step size Adaptation on linear cost functions

(1, )-Evolution Strategy with Cumulative Step size Adaptation on linear cost functions Adrien Coutoux, Auger Anne, Hansen Nikolaus INRIA Saclay - Ile-de-France, Project team TAO (1, )-Evolution Strategy with Cumulative Step size

465 views • 23 slides
Derivative-Free Methods for Machine Learning Tasks Inverse Problem Formulations The Ensemble

Derivative-Free Methods for Machine Learning Tasks Inverse Problem Formulations The Ensemble

EnKF Kovachki & Stuart Derivative-Free Methods for Machine Learning Tasks Inverse Problem Formulations The Ensemble Kalman Filter Ensemble Kalman Filter Numerics Nikola B. Kovachki 1 Andrew M. Stuart 1 1 Computational and Mathematical

353 views • 32 slides
min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

a i a j a i , a j a j , a i min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A parallel compare-exchange operation. Processes P i and P j send their elements to each other. Process P i keeps min { a i ,

372 views • 22 slides

More recommend