Lecture 1: Introduction 1 5-Jan-15 Lecture 1 - Fei-Fei Li - PowerPoint PPT Presentation

Lecture 1: Introduction 1 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Welcome to CS231n 2 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Biology Biology Psychology Psychology Neuroscience Physics Physics Cognitive optics sciences Image graphics, algorithms, Computer Computer processing Computer Computer theory,… Science Science Vision ision systems, Speech, NLP architecture, … Robotics Information retrieval Engineering Engineering Machine learning Mathematics Mathematics 3 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Biology Biology Psychology Psychology Neuroscience Physics Physics Cognitive optics sciences Image graphics, algorithms, Computer Computer processing Computer Computer theory,… Science Science Vision ision systems, Speech, NLP architecture, … Robotics Information retrieval Engineering Engineering Machine learning Mathematics Mathematics 4 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Computer Vision courses @ Stanford • CS131 (fall, 2014, Prof. Fei-Fei Li): – Undergraduate introductory class • CS231a (this term, Prof. Silvio Savarese) – Core computer vision class for seniors, masters, and PhDs – Topics include image processing, cameras, 3D reconstruction, segmentation, object recognition, scene understanding • CS231n (this ter CS231n (this term, Pr m, Prof. Fei-Fei Li & Andr of. Fei-Fei Li & Andrej ej Karpathy Karpathy) ) – Neural network (aka “deep lear Neural network (aka “deep learning”) class on image ning”) class on image classification classification • CS231b (spring, 2015, Prof. Fei-Fei Li): Cutting Edge Computer Vision – Project-based advanced vision class to prepare students for CV research • CS231m (spring, 2015, Prof. Silvio Savarese): Mobile Vision – Computer vision and computational photography for mobile platform (e.g. Android) • And an assortment of CS331 and CS431 for advanced topics in computer vision 5 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Today’s agenda • A brief history of computer vision • CS231n overview 6 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

543million years, B.C. 7 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Camera Obscura Leonardo da Vinci 16 th Century, A.D. 8 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Hubel & Wiesel, 1959 9 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Block world Larry Roberts, 1963 10 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

11 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

David Marr, 1970s 12 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Stages of Visual Representation, David Marr, 1970s 13 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

• Generalized Cylinder • Pictorial Structure Brooks & Binford, 1979 Fischler and Elschlager, 1973 14 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

David Lowe, 1987 15 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Normalized Cut (Shi & Malik, 1997) 16 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Face Detection, Viola & Jones, 2001 17 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

“SIFT” & Object Recognition, David Lowe, 1999 18 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Spatial Pyramid Matching, Lazebnik, Schmid & Ponce, 2006 19 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Histogram of Gradients (HoG) Deformable Part Model Dalal & Triggs, 2005 Felzenswalb, McAllester, Ramanan, 2009 20 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

PASCAL V ASCAL Visual Object Challenge isual Object Challenge (20 object categories) (20 object categories) [Everingham et al. 2006-2012] 1 all aeroplane bicycle bird 0.9 boat bottle bus Average Precision car 0.8 cat chair cow diningtable 0.7 dog horse motorbike person 0.6 pottedplant sheep sofa train 0.5 tvmonitor 0.4 2009 2010 2011 2012 Challenge Year 21 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

www.image-­‑net.org ¡ 22K ¡categories ¡and ¡ 14M ¡ images ¡ • Animals ¡ • Plants ¡ • Structures ¡ • Person ¡ • Bird ¡ • Tree ¡ • ArHfact ¡ • Scenes ¡ • Fish ¡ • Flower ¡ • Tools ¡ • Indoor ¡ • Mammal ¡ • Food ¡ • Appliances ¡ • Geological ¡FormaHons ¡ • Invertebrate ¡ • Materials ¡ • Structures ¡ • Sport ¡AcHviHes ¡ ¡ ¡ Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009 22 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Steel ¡drum ¡ The Image Classification Challenge: 1,000 object classes 1,431,167 images Output: Output: Output: Output: Scale Scale ✔ ¡ ✗ ¡ T-shirt T-shirt Steel drum Giant panda Drumstick Drumstick Mud turtle Mud turtle Russakovsky et al. arXiv, 2014 23 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Steel ¡drum ¡ The Image Classification Challenge: 1,000 object classes 1,431,167 images 0.28 ¡ 0.26 ¡ 0.16 ¡ 0.12 ¡ 0.07 ¡ Russakovsky et al. arXiv, 2014 24 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Today’s agenda • A brief history of computer vision • CS231n overview 25 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

CS231n focuses on one of the most important problems of visual recognition – image classification 26 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

27 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

There is a number of visual recognition problems that are related to image classification, such as object detection, image captioning 28 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

• Object detection • Action classification • Image captioning • … 29 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Convolutional Neural Network (CNN) has become an important tool for object recognition 30 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Year 2010 ear 2010 Year 2012 ear 2012 Year 2014 ear 2014 NEC-UIUC SuperVision GoogLeNet VGG MSRA Dense ¡grid ¡descriptor: ¡ HOG, ¡LBP ¡ Coding: ¡local ¡coordinate, ¡ super-­‑vector ¡ Pooling, ¡SPM ¡ Linear ¡SVM ¡ ConvoluHon ¡ Pooling ¡ SoMmax ¡ Other ¡ [Lin CVPR 2011] [Krizhevsky NIPS 2012] [Szegedy arxiv 2014] [Simonyan arxiv 2014] [He arxiv 2014] 31 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Convolutional Neural Network (CNN) is not invented overnight 32 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

1998 LeCun et al. # of transistors # of pixels used in training 10 6 10 7 2012 Krizhevsky et al. # of transistors # of pixels used in training GPUs 10 14 10 9 33 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

The quest for visual intelligence goes far beyond object recognition… 34 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

35 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

36 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

PT = 500ms PT = 500ms Some kind of game or fight. Two groups of two men? The foregound pair looked like one was getting a fist in the face. Outdoors seemed like because i have an impression of grass and maybe lines on the grass? That would be why I think perhaps a game, rough game though, more like rugby than football because they pairs weren't in pads and helmets, though I did get the impression of similar clothing. maybe some trees? in the background. (Subject: SM) Fei-Fei, Iyer, Koch, Perona, JoV, 2007 37 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

38 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �

Computer V Computer Vision T ision Technology echnology Can Better Our Lives Can Better Our Lives 39 ¡ 5-­‑Jan-­‑15 ¡ Lecture 1 - � Fei-Fei Li & Andrej Karpathy � �