1 CS 330 - Artificial Intelligence - Introduction II Instructor: Renzhi Cao Computer Science Department Pacific Lutheran University Fall 2017 Special appreciation to Ian Goodfellow, Joshua Bengio, Aaron Courville, Michael Nielsen, Andrew Ng, Katie Malone, Sebastian Thrun, Ethem Alpaydin, Christopher Bishop, Geoffrey Hinton.
Announcement 1. Gitlab 2. Think about teams 3. Read the book 4. Labs, and your computer availability
Introduction When machine learning starts? Inventors have long dreamed of creating machines that can learn. Desires date back to at least the time of ancient Greece. Inventors Pygmalion and statue he carved - Galatea Talos - a giant automaton made of bronze to protect Europa in Crete from pirates and invaders, by inventor Daedalus Inventor Hephaestus and the first human woman created by him - Pandora 3
Introduction When machine learning starts? People wonder whether such machines may become intelligent. Today, Artificial intelligence (AI) is a thriving field with many practical applications and active research topics. Machine learning in early days Used to solve problems that are intellectually difficult for human beings but relatively straightforward for computers, based on a list of formal and mathematical rules. Machine learning in now days The true challenge for machine learning is to solve tasks that are easy for people but difficult for machine to do. Recognizing spoken words, faces in images.
Introduction Why machine learning? • My experience at Silicon Valley. (13 w) It’s everywhere. You may not want to know how to make a • car, but it’s always good if you know something about it. A dream that one day the machine is as intelligent as • human. 5
Introduction Why machine learning? More importantly, the traditional program can not solve some problems, such as recognizing a three-dimensional object from a novel view in new lighting conditions in a cluttered scene. • What to write? Even you know what to • write, it’s too complicate. 6
Introduction Why machine learning? What rule to decide your credit card transaction is fraudulent? • There may be simple rules, but we need to combine a lot of weak rules • Rules will be changed, so your method needs to be updated all the time. 7
Introduction Machine learning approach Instead of writing program with all different rules, we collect examples that specify the correct output for a given input. A machine learning algorithm takes the examples and produces program that does the job The machine created program may look very different from typical • program. It may contains millions of numbers If we do it right, the machine created program works well for new • cases, and also the ones we trained on it If the data changes, the program can also be changed by training on • new data Massive amounts of computation are now cheaper than 8 paying someone to write-specific program.
Machine Learning in Our Daily Lives Postal Mail Spam Filtering Web Search Routing Vehicle Driver Movie Fraud Detection Recommendations Assistance Web Speech Social Networks Advertisements Recognition
Introduction Google search demo
Introduction Google translation demo
Introduction 12
Introduction 13
Introduction Self-driving car 14
Introduction Amazon recommendation 15
Art Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
Art Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
Art The Muse, Pablo Picasso, 1935 Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
Computational biology >T0759 HR9083A, Human, 109 residues MGHHHHHHSHMVVIHPDPGRELSPEEAHRAGLIDWNMFVKLRSQECDWEEISVKGPNGES SVIHDRKSGKKFSIEEALQSGRLTPAHYDRYVNKDMSIQELAVLVSGQK
Introduction Machine VS Human • IBM deep blue chess-playing system defeated world champion Garry Kasparov in 1997. 20
March 2016 AlphaGo 4 – Lee Sedol 1
The rules • Starts with an empty board. • Players take turns to place one stone on vacant point • Capture your opponent’s stones by completely surrounding them • Goal: Use your stones to form territories by surrounding vacant areas of the board https://www.britgo.org/intro/intro2.html
Jan. 2017 Baidu’s AI boss, Andrew Ng, pictured left with the host of ‘Super Brain’ and Baidu’s robot, Xiaodu
Jan. 2017 https://www.theguardian.com/technology/2017/jan/30/libratus-poker-artificial-intelligence-professional-human-players-competition
February 6, 2017 SWARM AI CORRECTLY PREDICTED THE OUTCOME OF SUPER BOWL LI, RIGHT DOWN TO THE FINAL SCORE The New England Patriots’ win over the Atlanta Falcons was nothing short of amazing. The Pats rallied back from a 25-point deficit to tie the game in the final minutes of regulation and secured the win with a decisive touchdown drive in overtime. Swarm AI (Combines swarming algorithms with human input) accurately predicted the outcome of the game, right down to the 34-28 win by the Patriots. http://www.digitaltrends.com/cool-tech/swarm-artificial-intelligence-super-bowl-patriots/
Introduction Machine learning VS AI • Interesting talk with students • Artificial Intelligence is the broader concept of machines being able to carry out tasks in a way that we would consider “smart”. • Machine Learning is a current application of AI based around the idea that we should really just be able to give machines access to data and let them learn for themselves. 29 http://www.forbes.com/sites/bernardmarr/2016/12/06/what-is-the-difference-between-artificial-intelligence-and-machine-learning/#2d597284687c
Deep learning Example: Shallow Example: Example: Example: autoencoders Logistic Knowledge MLPs regression bases Representation learning Machine learning AI
The MNIST Dataset Goodfellow, 2016
Introduction Historical Trends: Growing Datasets 10 9 Dataset size (number examples) Canadian Hansard 10 8 Sports-1M WMT 10 7 ImageNet10k 10 6 Public SVHN Criminals 10 5 ImageNet ILSVRC 2014 10 4 MNIST CIFAR-10 10 3 10 2 T vs. G vs. F Rotated T vs. C Iris 10 1 10 0 1900 1950 1985 2000 2015 Figure 1.8 Goodfellow, 2016
Biological neural network size from Wikipedia (2015). 33
Introduction Number of Neurons Number of neurons (logarithmic scale) 10 11 Human 10 10 17 20 10 9 16 19 Octopus 18 10 8 14 10 7 Frog 11 8 10 6 Bee 3 10 5 Ant 10 4 10 3 Leech 13 10 2 2 12 1 10 1 Roundworm 15 6 9 10 0 10 5 10 − 1 7 4 10 − 2 Sponge 1950 1985 2000 2015 2056 Figure 1.11 Goodfellow, 2016
Introduction Solving Object Recognition 0 . 30 ILSVRC classification error rate 0 . 25 0 . 20 0 . 15 0 . 10 0 . 05 0 . 00 2010 2011 2012 2013 2014 2015 Figure 1.12 Goodfellow, 2016
Introduction What is machine learning?
Introduction Machine learning definition Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed.
Introduction Machine learning definition Tom Mitchell provides a more modern definition: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” Example: playing checkers. E = the experience of playing many games of checkers T = the task of playing checkers. P = the probability that the program will win the next game.
Introduction Machine learning types • Supervised learning • Unsupervised learning • Reinforcement Learning
Introduction Supervised learning • Learn to predict an output when given an input vector UnSupervised learning Learn to select an action to minimize payoff • Reinforcement Learning Discover a good internal representation of the input •
Introduction Supervised learning • Each training case consists of an input vector x and a target output t. • Regression: the target output is a real number or a whole vector of real numbers. • Classification: the output is a class label.
Introduction Supervised learning We start by choosing a model-class: y = f( x , W ) • A model-class f is a way of using some numerical parameters W to map each input vector x into a predicted output y . Learning usually means adjusting the parameters to reduce the discrepancy between the target output t on each training case and the actual output y, which produced by the model. • For regression, we usually use the following as sensible measure of discrepancy: (y-t) 2 /2. • For classification, there are other measures that are generally more sensible.
850 Supervised Learning Regression: output is a continuous value Given “Right answers”
Breast Classification Output is discrete value (0 or 1, or 2, and etc.) 2.5
Recommend
More recommend
