The Gold Mine of the 21st Century Statistical Learning, Data Mining - PowerPoint PPT Presentation

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining The Gold Mine of the 21st Century Statistical Learning, Data Mining and Visualization February 24, 2014 Krzysztof Podgorski School of Economics and Management Lund University

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining Motto Nothing is more practical than a good theory. Vladimir Vapnik ∗ ∗ in Statistical Learning Theory . John Wiley, New York (1998)

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining?

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise.

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default identifying segments of a population likely to respond similarly to given events

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default identifying segments of a population likely to respond similarly to given events data mining tools sweep through databases to identify patterns in the buying activities to detect fraudulent transactions

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default identifying segments of a population likely to respond similarly to given events data mining tools sweep through databases to identify patterns in the buying activities to detect fraudulent transactions identifying anomalous data representing data entry error

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default identifying segments of a population likely to respond similarly to given events data mining tools sweep through databases to identify patterns in the buying activities to detect fraudulent transactions identifying anomalous data representing data entry error search for patterns in human genome to detect genetic conditioning of certain diseases

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How can business benefit from data mining? Automated prediction of trends that traditionally required extensive statistical analysis and specialized expertise. identify the targets most likely to maximize return on investment in future mailings forecasting bankruptcy and other forms of default identifying segments of a population likely to respond similarly to given events data mining tools sweep through databases to identify patterns in the buying activities to detect fraudulent transactions identifying anomalous data representing data entry error search for patterns in human genome to detect genetic conditioning of certain diseases A number companies in retail, finance, health care, manufacturing, transportation, and aerospace are already using data mining to take advantage of historical data.

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining Outline Concept of Statistical Learning 1 General Principles of Data Mining and Statistical Learning 2 Examples of Data Mining 3

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining What is statistical learning? Data mining – analysis of (often large) data to find re- Observe a phenomenon ✲ ✛ ✲ OBSERVE lationship to summarize in and collect data novel ways that are useful for the data owner ❄ Inference – identification of ❄ Propose a model of that the model that well de- ✲ ✛ ✲ MODEL phenomenon scribes the relations found in the data ❄ ❄ Prediction – making deci- Use the model to make ✲ ✛ ✲ sions with quantified uncer- PREDICT predictions tainty based on the model

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How statistical data mining different from statistics? Similarities

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How statistical data mining different from statistics? Similarities Statistical data mining in its broader meaning is identified as statistical learning which is a part of statistics since it is based on the same fundamental scheme of inference: Data → Model → Prediction

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How statistical data mining different from statistics? Similarities Statistical data mining in its broader meaning is identified as statistical learning which is a part of statistics since it is based on the same fundamental scheme of inference: Data → Model → Prediction Statistical data mining in its narrower meaning is a part of statistical learning that deals with searching for a possible model that maybe attached to the data

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How statistical data mining different from statistics? Similarities Differences Statistical data mining in its broader meaning is identified as statistical learning which is a part of statistics since it is based on the same fundamental scheme of inference: Data → Model → Prediction Statistical data mining in its narrower meaning is a part of statistical learning that deals with searching for a possible model that maybe attached to the data Statistical data mining is using statistical (uncertainty) modeling as its methodological foundation – this differs it from data mining as understood by a computer analyst

Concept of Statistical Learning General Principles of Data Mining and Statistical Learning Examples of Data Mining How statistical data mining different from statistics? Similarities Differences Statistical data mining in its broader Statistical data mining is typically meaning is identified as statistical dealing with much more complex learning which is a part of statistics data than the standard statistics since it is based on the same fundamental scheme of inference: Data → Model → Prediction Statistical data mining in its narrower meaning is a part of statistical learning that deals with searching for a possible model that maybe attached to the data Statistical data mining is using statistical (uncertainty) modeling as its methodological foundation – this differs it from data mining as understood by a computer analyst