apache systemml declarative large scale machine learning
play

Apache SystemML - Declarative Large-Scale Machine Learning Romeo - PowerPoint PPT Presentation

Oct 21, 2022 •378 likes •786 views

Apache SystemML - Declarative Large-Scale Machine Learning Romeo Kienzler (IBM Waston IoT) Berthold Reinwald (IBM Almaden Research Center) Frederick R. Reiss (IBM Almaden Research Center) Matthias Rieke (IBM Analytics) Swiss Data Science

  1. Apache SystemML - Declarative Large-Scale Machine Learning Romeo Kienzler (IBM Waston IoT) Berthold Reinwald (IBM Almaden Research Center) Frederick R. Reiss (IBM Almaden Research Center) Matthias Rieke (IBM Analytics) Swiss Data Science Conference 16 - ZHAW - Winterthur

  2. “High-level programming” –Assembler vs. Python?

  3. Why another lib? • Custom machine learning algorithms • Declarative ML • Transparent distribution on data-parallel framework • Scale-up • Scale-out • Cost-based optimiser generates low level execution plans

  4. Why on Spark? • Unification of SQL, Graph, Stream, ML • Common RDD structure • General DAG execution engine • lazy evaluation • distributed in-memory caching

  5. 2009-2010: Through 2007-2008: Multiple engagements with projects at IBM customers, we observe Research – Almaden 2009: We form a how data scientists involving machine dedicated team create ML solutions . learning on Hadoop. for scalable ML 2007 2008 2009 2010

  6. Research 2011 2013 2012 2014

  7. June 2015: IBM November 2015: June 2016: Announces open- SystemML enters Second Apache source SystemML Apache incubation release (0.10) September 2015: February 2016: Code available on First release (0.9) of Github Apache SystemML 2015 2016

  8. SystemML at Moved from Hadoop MapReduce to Spark SystemML supports both frameworks Exact same code 300X faster on 1/40 th as many nodes

  9. Systems Data Programmer Scientist R or Scala Python Results

  10. Alternating Least Squares j Products Customer i bought i product j. Customers

  11. Alternating Least Squares j Products Customer i bought i product j. Customers

  12. Alternating Least Squares j Products Customer i bought i product j. Customers

  13. Products Factor j Products Customer i bought i Customers Factor product j. Customers

  14. Products Factor j Products Customer i bought i Customers Factor product j. Customers

  15. Products Factor j Products Customer i bought i Customers Factor product j. Customers

  16. sparse matrix . × Multiply these two factors to Products Factor produce a less- j Products Customer i bought i Customers Factor product j. Customers

  17. sparse matrix . × Multiply these two factors to Products Factor produce a less- j Products Customer i bought i Customers Factor product j. Customers New nonzero values become product suggestions.

  19. U"="rand(nrow(X),"r,"min"="01.0,"max"="1.0);"" V"="rand(r,"ncol(X),"min"="01.0,"max"="1.0);"" while(i"<"mi)"{" """i"="i"+"1;"ii"="1;" """if"(is_U)" """"""G"="(W"*"(U"%*%"V"0"X))"%*%"t(V)"+"lambda"*"U;" """else" """"""G"="t(U)"%*%"(W"*"(U"%*%"V"0"X))"+"lambda"*"V;" """norm_G2"="sum(G"^"2);"norm_R2"="norm_G2;""""" """R"="0G;"S"="R;" """while(norm_R2">"10E09"*"norm_G2"&"ii"<="mii)"{" """""if"(is_U)"{" """""""HS"="(W"*"(S"%*%"V))"%*%"t(V)"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""U"="U"+"alpha"*"S;""" """""}"else"{" """""""HS"="t(U)"%*%"(W"*"(U"%*%"S))"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""V"="V"+"alpha"*"S;""" """""}" """""R"="R"0"alpha"*"HS;" """""old_norm_R2"="norm_R2;"norm_R2"="sum(R"^"2);" """""S"="R"+"(norm_R2"/"old_norm_R2)"*"S;" """""ii"="ii"+"1;" """}""" """is_U"="!"is_U;" }"

  20. U"="rand(nrow(X),"r,"min"="01.0,"max"="1.0);"" V"="rand(r,"ncol(X),"min"="01.0,"max"="1.0);"" while(i"<"mi)"{" """i"="i"+"1;"ii"="1;" """if"(is_U)" """"""G"="(W"*"(U"%*%"V"0"X))"%*%"t(V)"+"lambda"*"U;" """else" """"""G"="t(U)"%*%"(W"*"(U"%*%"V"0"X))"+"lambda"*"V;" """norm_G2"="sum(G"^"2);"norm_R2"="norm_G2;""""" """R"="0G;"S"="R;" """while(norm_R2">"10E09"*"norm_G2"&"ii"<="mii)"{" """""if"(is_U)"{" """""""HS"="(W"*"(S"%*%"V))"%*%"t(V)"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""U"="U"+"alpha"*"S;""" """""}"else"{" """""""HS"="t(U)"%*%"(W"*"(U"%*%"S))"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""V"="V"+"alpha"*"S;""" """""}" """""R"="R"0"alpha"*"HS;" """""old_norm_R2"="norm_R2;"norm_R2"="sum(R"^"2);" """""S"="R"+"(norm_R2"/"old_norm_R2)"*"S;" """""ii"="ii"+"1;" """}""" """is_U"="!"is_U;" }"

  21. U"="rand(nrow(X),"r,"min"="01.0,"max"="1.0);"" V"="rand(r,"ncol(X),"min"="01.0,"max"="1.0);"" while(i"<"mi)"{" """i"="i"+"1;"ii"="1;" """if"(is_U)" """"""G"="(W"*"(U"%*%"V"0"X))"%*%"t(V)"+"lambda"*"U;" """else" """"""G"="t(U)"%*%"(W"*"(U"%*%"V"0"X))"+"lambda"*"V;" """norm_G2"="sum(G"^"2);"norm_R2"="norm_G2;""""" """R"="0G;"S"="R;" """while(norm_R2">"10E09"*"norm_G2"&"ii"<="mii)"{" """""if"(is_U)"{" """""""HS"="(W"*"(S"%*%"V))"%*%"t(V)"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""U"="U"+"alpha"*"S;""" """""}"else"{" """""""HS"="t(U)"%*%"(W"*"(U"%*%"S))"+"lambda"*"S;" """""""alpha"="norm_R2"/"sum"(S"*"HS);" """""""V"="V"+"alpha"*"S;""" """""}" """""R"="R"0"alpha"*"HS;" """""old_norm_R2"="norm_R2;"norm_R2"="sum(R"^"2);" """""S"="R"+"(norm_R2"/"old_norm_R2)"*"S;" """""ii"="ii"+"1;" """}""" """is_U"="!"is_U;" }"

Recommend

Apache SystemML Declarative Machine Learning Luciano Resende IBM | Spark Technology Center IBM

Apache SystemML Declarative Machine Learning Luciano Resende IBM | Spark Technology Center IBM

Apache Big Data Seville 2016 Apache SystemML Declarative Machine Learning Luciano Resende IBM | Spark Technology Center IBM Spark Technology Center About Me Luciano Resende (lresende@apache.org) Architect and community liaison at IBM

1.18k views • 38 slides
SystemML: Declarative Machine Learning on Spark 05/03/19 Presented by: Juan Carrillo Candidate

SystemML: Declarative Machine Learning on Spark 05/03/19 Presented by: Juan Carrillo Candidate

SystemML: Declarative Machine Learning on Spark 05/03/19 Presented by: Juan Carrillo Candidate for MASc. in Computer Software Department of Electrical &amp; Computer Engineering University of Waterloo Agenda 1. Introduction 2. SystemML core

371 views • 20 slides
Large-Scale Machine Learning at Twitter 2 Large-Scale Machine Learning at Twitter Jimmy Lin and

Large-Scale Machine Learning at Twitter 2 Large-Scale Machine Learning at Twitter Jimmy Lin and

Large-Scale Machine Learning at Twitter 2 Large-Scale Machine Learning at Twitter Jimmy Lin and Alek Kolcz Twitter, Inc. 1 Image source:google.com/images Large-Scale Machine Learning at Twitter Outline Outline Is twitter big data?

582 views • 40 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine

TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine

TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine learning is everywhere This is in large part due to: 1. Invention of more sophisticated machine learning models 2. Availability of large

330 views • 22 slides
Large-Scale Machine Learning Shan-Hung Wu shwu@cs.nthu.edu.tw Department of Computer Science,

Large-Scale Machine Learning Shan-Hung Wu shwu@cs.nthu.edu.tw Department of Computer Science,

Large-Scale Machine Learning Shan-Hung Wu shwu@cs.nthu.edu.tw Department of Computer Science, National Tsing Hua University, Taiwan Machine Learning Shan-Hung Wu (CS, NTHU) Large-Scale ML Machine Learning 1 / 67 Outline When ML Meets Big

1.79k views • 133 slides
Mammoth Scale Machine Learning Speaker: Robin Anil, Apache Mahout PMC Member OSCON 10

Mammoth Scale Machine Learning Speaker: Robin Anil, Apache Mahout PMC Member OSCON 10

Mammoth Scale Machine Learning Speaker: Robin Anil, Apache Mahout PMC Member OSCON 10 Portland, OR July 2010 Quick Show of Hands Are you fascinated about ML? Have you used ML? Do you have Gigabytes

431 views • 42 slides
From Worst-Case to Realistic-Case Analysis for Large Scale Machine Learning Algorithms

From Worst-Case to Realistic-Case Analysis for Large Scale Machine Learning Algorithms

From Worst-Case to Realistic-Case Analysis for Large Scale Machine Learning Algorithms Maria-Florina Balcan, PI Avrim Blum, Co-PI Tom M Mitchell, Co-PI Students Travis Dick Nika Haghtalab Hongyang Zhang Motivation Machine learning

443 views • 22 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &amp;

925 views • 61 slides
Survey of Machine Learning Methods Pedro Rodriguez CU Boulder PhD Student in Large-Scale Machine

Survey of Machine Learning Methods Pedro Rodriguez CU Boulder PhD Student in Large-Scale Machine

Survey of Machine Learning Methods Pedro Rodriguez CU Boulder PhD Student in Large-Scale Machine Learning Overview Short theoretical review of each method Strong and weak points of each method Compare out of the box performance

420 views • 41 slides
Exploration of declarative languages applicability to development of large-scale data processing

Exploration of declarative languages applicability to development of large-scale data processing

Exploration of declarative languages applicability to development of large-scale data processing systems December 2016 Slavik Derevyanko, Anil Pacaci Declarative languages for distributed systems A research group at UC Berkeley lead by Prof.

593 views • 16 slides
Declara've Systems for Large Scale Machine Learning Markus

Declara've Systems for Large Scale Machine Learning Markus

Declara've Systems for Large Scale Machine Learning Markus Weimer, Tyson Condie, Raghu Ramakrishnan Cloud and Informa'on Services Laboratory MicrosoA

806 views • 61 slides
Big data management and predictive analytics for customer transactions Large Scale Distributed

Big data management and predictive analytics for customer transactions Large Scale Distributed

Big data management and predictive analytics for customer transactions Large Scale Distributed Machine Learning &amp; operations using Apache Spark and AWS BDA761 Big Data Mgmt in a Supercomputing environment MUDIT UPPAL Problem with Big

357 views • 15 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Parameter Learning Structure Learning Readings: KF 18.1, 18.3; Barber 9.5, 10.4 Dhruv Batra Virginia

351 views • 32 slides
Learning to Detect Faces A Large-Scale Application of Machine Learning ( This m aterial is not in

Learning to Detect Faces A Large-Scale Application of Machine Learning ( This m aterial is not in

Learning to Detect Faces A Large-Scale Application of Machine Learning ( This m aterial is not in the text: for further inform ation see the paper by P. Viola and M. Jones, I nternational Journal of Com puter Vision, 2 0 0 4 Viola-Jones Face

419 views • 26 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Structure Learning Readings: KF 18.4; Barber 9.5, 10.4 Dhruv Batra Virginia Tech Administrativia HW1

596 views • 30 slides
Large Scale Machine Learning with Stochastic Gradient Descent L eon Bottou leon@bottou.org

Large Scale Machine Learning with Stochastic Gradient Descent L eon Bottou leon@bottou.org

Large Scale Machine Learning with Stochastic Gradient Descent L eon Bottou leon@bottou.org Microsoft (since June) Summary i. Learning with Stochastic Gradient Descent. ii. The Tradeoffs of Large Scale Learning. iii. Asymptotic Analysis.

853 views • 37 slides
HiPS Hierarchical Parameter Synchronization in Large-Scale Distributed Machine Learning

HiPS Hierarchical Parameter Synchronization in Large-Scale Distributed Machine Learning

HiPS Hierarchical Parameter Synchronization in Large-Scale Distributed Machine Learning Jinkun Geng, Dan Li, Yang Cheng, Shuai Wang, and Junfeng Li 1 ACM SIGCOMM Workshop on NetAI Net AI for 2 Background Computation Communication

542 views • 33 slides
Large-scale machine learning and convex optimization Francis Bach INRIA - Ecole Normale Sup

Large-scale machine learning and convex optimization Francis Bach INRIA - Ecole Normale Sup

Large-scale machine learning and convex optimization Francis Bach INRIA - Ecole Normale Sup erieure, Paris, France Journ ees Statistiques du Sud - June 2014 Slides available at www.di.ens.fr/~fbach/gradsto_statsud.pdf Context Machine

1.43k views • 117 slides
Large-Scale Machine Learning I. Scalability issues Jean-Philippe Vert jean-philippe.vert@ {

Large-Scale Machine Learning I. Scalability issues Jean-Philippe Vert jean-philippe.vert@ {

Large-Scale Machine Learning I. Scalability issues Jean-Philippe Vert jean-philippe.vert@ { mines-paristech,curie,ens } .fr 1 / 76 Outline Introduction 1 Standard machine learning 2 Dimension reduction: PCA Clustering: k -means Regression:

1.25k views • 98 slides
Apache Flink Fast and Reliable Large-Scale Data Processing Fabian Hueske @fhueske 1 What is

Apache Flink Fast and Reliable Large-Scale Data Processing Fabian Hueske @fhueske 1 What is

Apache Flink Fast and Reliable Large-Scale Data Processing Fabian Hueske @fhueske 1 What is Apache Flink? Distributed Data Flow Processing System Focused on large-scale data analytics Real-time stream and batch processing Easy and

667 views • 39 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets: Inference Marginals, MPE, MAP Variable Elimination Readings: KF 9.1,9.2; Barber 5.1 Dhruv Batra Virginia Tech

667 views • 22 slides
Regularization Methods for Large Scale Machine Learning Alessandro Rudi University of Genova -

Regularization Methods for Large Scale Machine Learning Alessandro Rudi University of Genova -

Regularization Methods for Large Scale Machine Learning Alessandro Rudi University of Genova - Istituto Italiano di Tecnologia Massachusetts Institute of Technology lcsl.mit.edu In collaboration with: Raffaello Camoriano, Lorenzo Rosasco May

702 views • 46 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics: Bayes Nets: Representation/Semantics v-structures Probabilistic influence, Active Trails Readings: Barber 3.3; KF

322 views • 16 slides

More recommend