constraint solving meets machine learning and data mining
play

Constraint solving meets machine learning and data mining Algorithm - PowerPoint PPT Presentation

Aug 28, 2022 •39 likes •639 views

Constraint solving meets machine learning and data mining Algorithm portfolios Kustaa Kangas University of Helsinki, Finland November 8, 2012 K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012

  1. Constraint solving meets machine learning and data mining Algorithm portfolios Kustaa Kangas University of Helsinki, Finland November 8, 2012 K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 1 / 29

  2. Constraint problems Boolean satisifiability, integer linear programming etc. K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  3. Constraint problems Boolean satisifiability, integer linear programming etc. NP-hard K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  4. Constraint problems Boolean satisifiability, integer linear programming etc. NP-hard Many instances solvable with heuristic algorithms K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  5. Constraint problems Boolean satisifiability, integer linear programming etc. NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  6. Constraint problems Boolean satisifiability, integer linear programming etc. NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks Different algorithms are fast on different instances K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  7. Constraint problems Boolean satisifiability, integer linear programming etc. NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks Different algorithms are fast on different instances Typically no single best algorithm K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 2 / 29

  8. Algorithm selection Algorithm selection problem Given a problem instance, which algorithm should we run? Ideally: run the algorithm that’s fastest on the instance Problem: we cannot know without running the algorithms Traditional solution: run the average-case best algorithm Might be reasonably good Can be very bad on some instances Ignores algorithms that are good on some instances K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 3 / 29

  9. Algorithm portfolios Idea: Use several algorithms to improve expected performance. Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 4 / 29

  10. Algorithm portfolios Idea: Use several algorithms to improve expected performance. Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 4 / 29

  11. Algorithm portfolios Idea: Use several algorithms to improve expected performance. Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) Select one algorithm based on the instance K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 4 / 29

  12. Algorithm portfolios Idea: Use several algorithms to improve expected performance. Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) Select one algorithm based on the instance Anything from between K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 4 / 29

  13. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  14. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  15. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm ◮ explains how hard instances are and why K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  16. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm ◮ explains how hard instances are and why ◮ an approximate predictor of running time K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  17. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm ◮ explains how hard instances are and why ◮ an approximate predictor of running time ◮ predicts hardness based on instance features K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  18. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm ◮ explains how hard instances are and why ◮ an approximate predictor of running time ◮ predicts hardness based on instance features Selects the algorithm predicted to be fastest K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  19. SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm ◮ explains how hard instances are and why ◮ an approximate predictor of running time ◮ predicts hardness based on instance features Selects the algorithm predicted to be fastest Performed well in 2007 SAT Competition ◮ 1st place in 3 categories, one 2nd place and one 3rd place K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 5 / 29

  20. Empirical hardness models Where do empirical hardness models come from? K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 6 / 29

  21. Empirical hardness models Where do empirical hardness models come from? They must be learned from data: a set of algorithms 1 a set of training instances 2 a set of instance features 3 K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 6 / 29

  22. Empirical hardness models Where do empirical hardness models come from? They must be learned from data: a set of algorithms 1 a set of training instances 2 a set of instance features 3 We use machine learning to exploit correlations between features and algorithm performance K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 6 / 29

  23. Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance 1 instance 2 instance 3 instance 4 instance 5 instance 6 instance 7 instance 8 instance 9 instance 10 instance 11 instance 12 . . . . . . . . . . . . . . . . . . . . . . . . K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 7 / 29

  24. Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance 1 18 101 1222 6 1 16 instance 2 23 8124 241 2 1 8 instance 3 57 32 683 3 5 42 instance 4 17 435 153 4 1 10 instance 5 46 76 346 3 4 30 instance 6 57 32 327 2 11 12 instance 7 26 62149 2408 3 2 15 instance 8 70 226 498 3 4 30 instance 9 30 194 20060 5 2 25 instance 10 13 108 614 8 4 3 instance 11 36 307 556 2 5 11 instance 12 56 100 728 4 13 7 . . . . . . . . . . . . . . . . . . . . . . . . K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 8 / 29

  25. Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance 1 18 101 1222 6 1 16 15698 instance 2 23 8124 241 2 1 8 129 instance 3 57 32 683 3 5 42 14680 instance 4 17 435 153 4 1 10 14 instance 5 46 76 346 3 4 30 493 instance 6 57 32 327 2 11 12 7332 instance 7 26 62149 2408 3 2 15 31709 instance 8 70 226 498 3 4 30 214 instance 9 30 194 20060 5 2 25 131 instance 10 13 108 614 8 4 3 1 instance 11 36 307 556 2 5 11 2026 instance 12 56 100 728 4 13 7 60 . . . . . . . . . . . . . . . . . . . . . . . . K. Kangas (U. Helsinki) Constraint solving meetsmachine learning and data mining November 8, 2012 Algorithm portfolios 9 / 29

Recommend

A Constraint-Solving Approach to Faust Program Type Checking Constraint Programming Meets

A Constraint-Solving Approach to Faust Program Type Checking Constraint Programming Meets

A Constraint-Solving Approach to Faust Program Type Checking Constraint Programming Meets Verification 2014 Workhsop Imr Frotier de la Messelire 1 , Pierre Jouvelot 1 , Jean-Pierre Talpin 2 1 MINES ParisTech, PSL Research University 2 INRIA

620 views • 26 slides
Data Mining meets Football (soccer) Ulf Brefeld Knowledge Mining & Assessment TU Darmstadt

Data Mining meets Football (soccer) Ulf Brefeld Knowledge Mining & Assessment TU Darmstadt

Data Mining meets Football (soccer) Ulf Brefeld Knowledge Mining & Assessment TU Darmstadt / DIPF brefeld@cs.tu-darmstadt.de Data Mining meets Football (soccer) Ulf Brefeld Machine Learning Group Leuphana University of Lneburg

1.25k views • 63 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by I. H. Witten and E. Frank Implementation: Real machine learning schemes Decision trees From ID3 to C4.5 (pruning, numeric attributes, ...)

1.78k views • 130 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 6 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Implementation: Real machine learning schemes Decision trees From ID3 to C4.5 (pruning, numeric

1.87k views • 172 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 1 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 1 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 1 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Whats it all about? Data vs information Data mining and machine learning Structural

552 views • 42 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Ensemble learning Combining multiple models The basic idea Bagging Bias-variance

429 views • 27 slides
LIACS Data Mining course an introduction Course Textbook Data Mining Practical Machine Learning

LIACS Data Mining course an introduction Course Textbook Data Mining Practical Machine Learning

Arno Knobbe Joaquin Vanschoren LIACS Data Mining course an introduction Course Textbook Data Mining Practical Machine Learning Tools and Techniques second edition, Morgan Kaufmann, ISBN 0-12-088407-0 by Ian Witten and Eibe Frank Course

949 views • 51 slides
ONLINE MACHINE LEARNING AND DATA MINING EDO LIBERTY STANDARD MACHINE LEARNING SETTING =

ONLINE MACHINE LEARNING AND DATA MINING EDO LIBERTY STANDARD MACHINE LEARNING SETTING =

ONLINE MACHINE LEARNING AND DATA MINING EDO LIBERTY STANDARD MACHINE LEARNING SETTING = call = ? = crawl Training Test Data Data Train call Apply Model Label 2 STANDARD MACHINE LEARNING SETTING Predicting the future

746 views • 51 slides
Machine Learning: Der KDD-Prozess Knowledge Discovery in Databases Machine Learning Data-Mining

Machine Learning: Der KDD-Prozess Knowledge Discovery in Databases Machine Learning Data-Mining

Machine Learning: Der KDD-Prozess Knowledge Discovery in Databases Machine Learning Data-Mining Aufgaben Aufgabe Von Daten zum Wissen Wissensextraktion durch We are drowing in automatisches Erkennen von information,

1.01k views • 47 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by I. H. Witten and E. Frank Algorithms: The basic methods Inferring rudimentary rules Statistical modeling Constructing decision trees

1.26k views • 108 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 4 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Algorithms: The basic methods Inferring rudimentary rules Statistical modeling Constructing

1.13k views • 111 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 5 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 5 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 5 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Credibility: Evaluating whats been learned Issues: training, testing, tuning Predicting

1.06k views • 68 slides
CPSC 340: Machine Learning and Data Mining Data Exploration Summer 2020 This lecture roughly

CPSC 340: Machine Learning and Data Mining Data Exploration Summer 2020 This lecture roughly

CPSC 340: Machine Learning and Data Mining Data Exploration Summer 2020 This lecture roughly follow: http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap2_data.pdf Data Mining: Birds Eye View 1) Collect data. 2) Data mining! 3)

1.16k views • 77 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 3 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 3 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 3 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Input: Concepts, instances, attributes Terminology Whats a concept? Classification,

689 views • 35 slides
Lecture 1 Jan-Willem van de Meent What is Data Mining? Intersection of Disciplines Machine

Lecture 1 Jan-Willem van de Meent What is Data Mining? Intersection of Disciplines Machine

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 1 Jan-Willem van de Meent What is Data Mining? Intersection of Disciplines Machine Statistics Optimization Learning Applications Visualization Data

604 views • 36 slides
Big Data Meets Machine Learning Apache Spark MLlib 1 MLlib Spark MLlib Graphx

Big Data Meets Machine Learning Apache Spark MLlib 1 MLlib Spark MLlib Graphx

Big Data Meets Machine Learning Apache Spark MLlib 1 MLlib Spark MLlib Graphx Streaming Spark Dataframe Spark Core (RDD) 2 Machine Learning Algorithms Supervised learning Given a set of features and labels Builds a model that

590 views • 24 slides
Python for brain mining: (neuro)science with state of the art machine learning and data

Python for brain mining: (neuro)science with state of the art machine learning and data

Python for brain mining: (neuro)science with state of the art machine learning and data visualization Ga el Varoquaux 1. Data-driven science Brain mining 2. Data mining in Python Mayavi, scikit-learn, joblib 1 Brain mining Learning

675 views • 28 slides
CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map Machine Learning Approaches Supervised Semi-supervised Unsupervised Reinforcement Learning Learning Learning Learning Classification Regression

1.06k views • 57 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Data transformations Attribute selection Scheme-independent, scheme-specific Attribute

1.02k views • 47 slides
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by I. H. Witten and E. Frank Engineering the input and output Attribute selection Scheme-independent, scheme-specific Attribute

634 views • 59 slides
Machine Learning and Econometrics Hal Varian Jan 2014 Definitions Machine learning, data

Machine Learning and Econometrics Hal Varian Jan 2014 Definitions Machine learning, data

Machine Learning and Econometrics Hal Varian Jan 2014 Definitions Machine learning, data mining, predictive analytics, etc. all use data to predict some variable as a function of other variables. May or may not care about insight,

609 views • 44 slides
Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli

Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli

Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli Goharian, IIT/Georgetown Jiawei Han, Micheline Kamber, and Jian Pei, Data Mining 1 1 2 Classification: Basic Concepts n Classifica(on:

1.39k views • 126 slides
CPSC 340: Machine Learning and Data Mining Fundamentals of Learning Summer 2020 Last Time:

CPSC 340: Machine Learning and Data Mining Fundamentals of Learning Summer 2020 Last Time:

CPSC 340: Machine Learning and Data Mining Fundamentals of Learning Summer 2020 Last Time: Supervised Learning Notation Egg Milk Fish Wheat Shellfish Peanuts Sick? 0 0.7 0 0.3 0 0 1 0.3 0.7 0 0.6 0 0.01 1 0 0 0 0.8 0 0

779 views • 60 slides
Machine Learning and Data Mining Nearest neighbor methods Kalev Kask Supervised learning

Machine Learning and Data Mining Nearest neighbor methods Kalev Kask Supervised learning

+ Machine Learning and Data Mining Nearest neighbor methods Kalev Kask Supervised learning Notation Features x Targets y Predictions Parameters q Learning algorithm Change q Program (Learner) Improve performance

366 views • 21 slides

More recommend