ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION Henrik Brink - PowerPoint PPT Presentation

ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION Henrik Brink Machine Learning Engineering @ Wise.io / GE Digital brinkar

Agenda • From space to industrial machine learning • Challenges • Optimization dimensions • Infrastructure • Development and deployment • Solutions • The ML meta-algorithm • Containerization • Best engineering practices • Wrap-up and questions brinkar

From astronomy to industrial machine learning... brinkar

brinkar

brinkar

brinkar

brinkar

brinkar

brinkar

brinkar

Everyone ready...? brinkar

ONE DOES NOT SIMPLY DEPLOY ML INTO PRODUCTION brinkar

Wh What t to opti timi mize for when bui buildi ding ng ML ML systems? brinkar

Accuracy? Accu cy? brinkar

Implementatio Implemen ion c n cost? brinkar

Ru Runti time 1920 CPUs co cost? 280 GPUs brinkar

In Inter erpr pretabilit ability? brinkar

LOCAL INTERPRETABLE MODEL- AGNOSTIC EXPLANATIONS (LIME arxiv.org/abs/1602.04938 brinkar

Au Autom omati tion on vs vs aug augmen mentatio ion? n? brinkar

Wh What t to opti timi mize for when bui buildi ding ng ML ML systems? • Accuracy? • Implementation cost? • Runtime cost? • Interpretability? • Automation vs augmentation? brinkar

A A 5-st step ML meta-alg algorit ithm hm 1. Identify and define the problem 2. Collect and understand the data 3. Build and deploy a simple model that works end-to-end 4. Iterate to optimize an deploy improved models (inner loop) 5. Monitor and back-propagate changes to problem parameters (outer loop) brinkar

1. Identify and define the problem 5. Monitor and back-propagate changes to problem parameters brinkar

2. Collect and understand the data brinkar

2. Collect and understand the data brinkar

2. Collect and understand the data brinkar

3. Deploy simple model end-to-end 4. Iterate to optimize an deploy improved models Simplest possible model that solves the • problem End-to-end production deployment: • automated deployment, testing, logging, monitoring, feedback brinkar

Getting cl closer... brinkar

Mach chine learning infrastruct cture brinkar

Continuous integration Co brinkar

brinkar

3 r 3 reaso eason f n for using using c container ainers in s in mac machine lear hine learning ning... brinkar

1 P 1 Pac ackag aging ing brinkar

2 Inher 2 Inherit itanc ance tensorflow FROM tensorflow/tensorflow:latest-gpu # Your specialized modeling pipeline images text FROM my-special-pipeline # Your even more specialized pipeline segmentation classification sentiment brinkar

Immutabilit Immut ability brinkar

“Data should go through the exact same pipeline when making predictions, as when the model was built.” -- Good ML practitioner brinkar

Us Use gr great t framewor orks and service ces... ... brinkar

Op Open sou ource ce ML framewor orks brinkar

RISELab • Deploy models trained in your choice of framework to Clipper with a few lines of code by using an existing model container or Clipper writing your own • Easily update or add models to running applications • Use adversarial bandit algorithms to dynamically select best model for prediction at serving time • Set latency service level objectives for reliable query latencies • Run each model in a separate Docker container for simple cluster management and resource allocation • Deploy models running on CPUs, GPUs , or both in the same application brinkar

Ho Hosted M ed ML ser servic ices es brinkar

Mach chine learning infrastruct cture • Continuous integration • Containerization • Utilize ML platforms brinkar

Wrap up... Wr brinkar

Real-World Machine Learning manning.com/brink brinkar

nordic.ai meetup.com/datacph brinkar

Questi Qu tions? s? brinkar