Biologically Inspired Machine Perception N i c h o l a s B u t k o - PowerPoint PPT Presentation

Biologically Inspired Machine Perception N i c h o l a s B u t k o , M a c h i n e P e r c e p t i o n L a b W I n t e r , 2 0 1 0

<Chapter 1> Artificial Intelligence vs. Natural Intelligence Borrowed Intelligence vs. Owned Intelligence Hard Things are Easy, Easy Things are Hard

Inspiration Early on, Artificial Intelligence grabbed hold of my imagination and wouldn’t let go. “The Age of Spiritual Machines” By 2020, computers will have more transistors than brains have neurons . That won’t be sufficient for computers to be intelligent: Can’t write a summary of a movie Can’t tie shoe-laces Can’t recognize humor AI is not limited by computing power , but by our understanding of “intelligence” A revolution in that understanding is required before we can create truly cognitive machines . I wanted to be part of that revolution.

First Steps Freshman year of undergrad: Volunteered in lab of AI prof in CSE. “Labeling” eyes and mouths. Thousands of images. Computer used this information to help figure out facial expression . One of the most successful paradigms in AI: “Supervised Learning” “Learn” about facial expressions from thousands of examples Use statistics , calculus , and linear algebra.

Computer Expression Recognition Toolbox “Supervised Learning” has been very successful; My own lab uses it extensively to develop sophisticated facial-expression recognizers. [Demo at end, if we have time]

Computer Expression Recognition Toolbox Widely Applicable: Driver Drowsiness Lie Detection Real/Fake Pain Autism Therapy Tutoring Smile Shutter Art Different from how humans learn: Nobody points out thousands of eyes and mouths to babies to help them learn about faces.

May, 11, 1997

What’s wrong?

Simple Is Hard Daniel Wolpert, “The Master Puppeteer” Crick Memorial Lecture, 2005 http://royalsociety.org/event.asp?id=3773

Why is simple hard? Artificial domains like chess have a clear, well defined structure. Natural domains like “seeing” are rife with ambiguity. Consider a simple problem like “how to look at something.” ? ? ?

Dealing with Ambiguities To know “how to look somewhere”, it is helpful to know “where did I look?” From many experiences of sending signals to your eye-muscle neurons, your brain can learn the relationship between actions and consequences. Whole Scene View 1 View 2 Difficulty No match Same object? (Which lightpost?) Same object type? (Lake or Cloud?) Same location? (Moving Target) Even the question “Where did I look?” is hard to answer! Lots of things could go wrong. Can we ever make explicit rules for all of them?

</Chapter 1> 1) Which of these is easiest for a computer program: Seeing, Doing your laundry, Playing Sudoku, Writing a Book Report, Laughing at funny jokes? 2) We gave four reasons that it’s tough to know where you’re looking. Can you remember them? What’s the main difficulty that unites them? 3) If you were going to use today’s state-of-the-art approaches to make an intelligent computer program that “Knows how to teach,” what is the first thing you should do?

<Chapter 2> The Computational Approach: Do we need feathers to fly? Define the Problem with a Generative Model Algebra to the Rescue: Finding all the rules.

How to study Natural Intelligence? Trying to understand perception by studying only neurons is like trying to understand bird flight by studying only feathers: it just cannot be done. In order to understand bird flight, we have to understand aerodynamics; only then do the structure of the feathers and the different shapes of bird’s wings make sense. --Marr, Vision , 1982 Study the “aerodynamics” of natural intelligence -- the underlying principles and objectives organizing behavior. Want a theory that’s not just about humans Flying is not about birds and feathers. Different organisms or systems may not have access to the type of actuators and sensors that humans have, but we still want to understand and build intelligent systems. Choose problems that will help us understand behavior in real life. E.g. “Learning how to look somewhere.”

Defining the Problem: A Generative Model A “Generative Model” is a tool to describe the structure of the t=1 t=2 t=3 problems organisms face. Sensory ( � ) You must describe how the things you can see relate to the Motor (a) things you want to know . {0.3, 0.2} {0.25, 0.5} {0.1, 0.1} + ∞ You must describe your � 1 Camera � image uncertainty about how things are � 1 ={x',y'} - ∞ + ∞ and how things will be . Where the - ∞ � � camera is = � 2 looking Probability theory tells us how World + ∞ � 2 appearance to make the best guess about a � 2 � ={x,y} - ∞ + ∞ how the things you want to know How the Motor - ∞ are and how they will be based on motors command work value everything you’ve seen before .

Finding all the rules A little probability theory: What you see right now. W here am I looking ? � �� � p ( τ t | τ 1: t − 1 , ψ 1: t , a 1: t ) = = p ( τ t | τ 1: t − 1 , a 1: t ) p ( ψ t | ψ 1: t − 1 , τ 1: t − 1 ) p ( ψ 1: t − 1 | τ 1: t − 1 ) p ( ψ 1: t | τ 1: t − 1 ) And a little algebra: g ( τ t ) = P redicted Motion Match � �� � ( τ t − C t α Kt ) T ( C t Σ α Kt C T t + Q α ) − 1 ( τ t − C t α Kt ) = − . 5 ( ψ xy − λ xy Kt ) 2 � � log( σ xy 2 t λ Kt + q 2 − . 5 − . 5 λ ) ( σ xy 2 λ Kt + q 2 λ ) � �� � xy xy � �� � Uncertainty P enalty Image Match Give us all the rules for making the best guess about where we are Everything you’ve seen so far looking.

Finding all the rules A little probability theory: What you see right now. W here am I looking ? � �� � p ( τ t | τ 1: t − 1 , ψ 1: t , a 1: t ) = = p ( τ t | τ 1: t − 1 , a 1: t ) p ( ψ t | ψ 1: t − 1 , τ 1: t − 1 ) p ( ψ 1: t − 1 | τ 1: t − 1 ) p ( ψ 1: t | τ 1: t − 1 ) Where you think you’re looking based on the neural signals sent to your eyes. And a little algebra: g ( τ t ) = P redicted Motion Match � �� � ( τ t − C t α Kt ) T ( C t Σ α Kt C T t + Q α ) − 1 ( τ t − C t α Kt ) = − . 5 ( ψ xy − λ xy Kt ) 2 � � log( σ xy 2 t λ Kt + q 2 − . 5 − . 5 λ ) ( σ xy 2 λ Kt + q 2 λ ) � �� � xy xy � �� � Uncertainty P enalty Image Match Give us all the rules for making the best guess about where we are Everything you’ve seen so far looking.

Finding all the rules A little probability theory: What you see right now. W here am I looking ? � �� � p ( τ t | τ 1: t − 1 , ψ 1: t , a 1: t ) = = p ( τ t | τ 1: t − 1 , a 1: t ) p ( ψ t | ψ 1: t − 1 , τ 1: t − 1 ) p ( ψ 1: t − 1 | τ 1: t − 1 ) p ( ψ 1: t | τ 1: t − 1 ) OK Match Good Match Possible Match And a little algebra: g ( τ t ) = P redicted Motion Match � �� � ( τ t − C t α Kt ) T ( C t Σ α Kt C T t + Q α ) − 1 ( τ t − C t α Kt ) = − . 5 ( ψ xy − λ xy Kt ) 2 � � log( σ xy 2 t λ Kt + q 2 − . 5 − . 5 λ ) ( σ xy 2 λ Kt + q 2 λ ) � �� � xy xy � �� � Uncertainty P enalty Image Match Give us all the rules for making the best guess about where we are Everything you’ve seen so far looking.

Finding all the rules A little probability theory: What you see right now. W here am I looking ? � �� � p ( τ t | τ 1: t − 1 , ψ 1: t , a 1: t ) = = p ( τ t | τ 1: t − 1 , a 1: t ) p ( ψ t | ψ 1: t − 1 , τ 1: t − 1 ) p ( ψ 1: t − 1 | τ 1: t − 1 ) p ( ψ 1: t | τ 1: t − 1 ) Avoid if possible And a little algebra: g ( τ t ) = P redicted Motion Match � �� � ( τ t − C t α Kt ) T ( C t Σ α Kt C T t + Q α ) − 1 ( τ t − C t α Kt ) = − . 5 ( ψ xy − λ xy Kt ) 2 � � log( σ xy 2 t λ Kt + q 2 − . 5 − . 5 λ ) ( σ xy 2 λ Kt + q 2 λ ) � �� � xy xy � �� � Uncertainty P enalty Image Match Give us all the rules for making the best guess about where we are Everything you’ve seen so far looking.

Finding all the rules A little probability theory: What you see right now. W here am I looking ? � �� � p ( τ t | τ 1: t − 1 , ψ 1: t , a 1: t ) = = p ( τ t | τ 1: t − 1 , a 1: t ) p ( ψ t | ψ 1: t − 1 , τ 1: t − 1 ) p ( ψ 1: t − 1 | τ 1: t − 1 ) p ( ψ 1: t | τ 1: t − 1 ) Best Guess! And a little algebra: g ( τ t ) = P redicted Motion Match � �� � ( τ t − C t α Kt ) T ( C t Σ α Kt C T t + Q α ) − 1 ( τ t − C t α Kt ) = − . 5 ( ψ xy − λ xy Kt ) 2 � � log( σ xy 2 t λ Kt + q 2 − . 5 − . 5 λ ) ( σ xy 2 λ Kt + q 2 λ ) � �� � xy xy � �� � Uncertainty P enalty Image Match Give us all the rules for making the best guess about where we are Everything you’ve seen so far looking.

Learning to Look Error on Desired Eye-Movement 30 20 10 0 50 100 150 200 250 300 350 Eye-Movements