Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Visual Servoing Henrik I. Christensen Robotics and Intelligent Machines @ GT College of Computing Georgia Institute of Technology Atlanta, GA hic@cc.gatech.edu February 14, 2008 H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 1 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Outline Introduction 1 Problem Structure 2 Servoing 3 Image Based Servoing 4 Smart Strategies 5 6 Summary H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 2 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Introduction Recap of camera model Overview of strategies to visual servoing The basic theory behind models for visual servoing A few examples to illustrate use of the system. H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 3 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Pin-Hole Model H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 4 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Pin-Hole Model The relations are then: x X = λ Z y Y = λ Z ⇒ λ X x = Z λ Y = y Z H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 5 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Pin-Hole Model Remember Homogeneous Coordinates? X Y � P = Z 1 Define the Perspective transform as 1 0 0 0 0 1 0 0 T = 0 0 1 0 1 0 0 0 λ H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 6 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary The general problem of servoing u ROBOT y* GENERATION PLANNING CONTROL TASK 3D WORLD x STATE ESTIMATION y H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 7 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Outline Introduction 1 Problem Structure 2 Servoing 3 Image Based Servoing 4 Smart Strategies 5 6 Summary H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 8 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Coordinate Systems PUMA 560 O CR O C O O G CL O O O R O W H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 9 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Basic Organization H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 10 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Camera - Robot Configurations VISUAL MEASUREMENTS number of cameras 1 2 >2 EYE−IN−HAND STAND−ALONE EYE−IN−HAND STAND−ALONE VM1 VM2 VM3 VM4 VM5 VM1 VM2 VM3 VM4 H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 11 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Outline Introduction 1 Problem Structure 2 Servoing 3 Image Based Servoing 4 Smart Strategies 5 6 Summary H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 12 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Approaches to servoing Position Based Servoing Image Based Servoing 2 1 2 D Servoing H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 13 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Position Based Servoing POSITION T d + V, Ω (CARTESIAN) ROBOT − BASED CONTROL Tc FEATURE 3D POSE CAMERA EXTRACTION ESTIMATION H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 14 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Position Servoing X X Tc Z Oc Z Y O Y Td X Z Od Y H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 15 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Image Based Servoing S + V, Ω d IMAGE ROBOT − BASED CONTROL S c FEATURE CAMERA EXTRACTION H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 16 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Image Servoing f 3 d f d 2 f 4 d 1 f d 3 X 4 f c f c 2 f c f c 1 Z Oc Y H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 17 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary 2.5D Servoing CONTROL LAW * pe + V POSITION ROBOT CAMERA − Ω u θ * + − ROTATION u θ PARTIAL FEATURE pe 3D POSE EXTRACTION ESTIMATION H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 18 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Outline Introduction 1 Problem Structure 2 Servoing 3 Image Based Servoing 4 Smart Strategies 5 Summary 6 H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 19 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Image Based Servoing Specify the task as an image based task Associate an error function ( e ) with the task Goal is achieved when e = 0 Derivation of an Image Jacobian to relate image changes to control of the robot H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 20 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Image Jacobian δ f 1 ( P ) δ f 1 ( P ) . . . δ P 1 δ P m . . ... � δ f � . . J ( P ) = = . . δ P δ f k ( P ) δ f k ( P ) . . . δ P 1 δ P m H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 21 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Point Motion The point motion can be written as C ˙ C Ω × C P + C V P = Or more detailed x ˙ = z ω y − y ω z + T x y ˙ = x ω z − z ω x + T y ˙ = y ω x − x ω y + T z z H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 22 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Point flow Given the perspective projection � u � x � = λ � v z y The image flow is then λ z ˙ x − x ˙ z ˙ = u z 2 λ z ˙ y − y ˙ z ˙ = v z 2 H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 23 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Image Jacobian V X V Y � ˙ � k u 1 − x 1 + x 2 � 0 � � 0 � u − xy − y V Z Z Z = 1 − y − 1 − y 2 ˙ 0 0 ω X v k v xy x Z Z ω Y ω Z H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 24 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary For k image points? 1 − x 1 1 + x 2 V X 0 − x 1 y 1 − y 1 u 1 ˙ Z 1 Z 1 1 1 − y 1 V Y − 1 − y 2 ˙ 0 x 1 y 1 x 1 v 1 1 Z 1 Z 1 V Z . . . . . . . . . . . . . . = A . . . . . . . ω X 1 − x k 1 + x 2 u k ˙ 0 − x k y k − y k ω Y Z k Z k k 1 − y k − 1 − y 2 v k ˙ 0 x k y k x k ω Z k Z k Z k H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 25 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Considerations Naive use of image based servoing might generate surprising motion of the camera Comparison: Approach Pros Cons PBVS Intuitive Calibration? IBVS Accuracy Singularities? H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 26 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Illustrative Examples H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 27 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Outline Introduction 1 Problem Structure 2 Servoing 3 Image Based Servoing 4 Smart Strategies 5 6 Summary H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 28 / 37
Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Smart Strategies Doing large scale servoing is really hard Doing limited recognition is much easier How about recognition based servoing? H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 29 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Model Based Servoing 3D CAMERA RENDER PROJECTION PARAMETERS MODEL MATRIX FITTING LOCATE 3D MODEL VISIBLE EDGES NORMAL MANUAL POSE FLOW INITIALIZATION AFFINE MOTION H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 30 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Initial Pose Estimation H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 31 / 37 Introduction Problem Structure Servoing Image Based Servoing Smart Strategies Summary Initialization H. I. Christensen (RIM@GT) Visual Servoing February 14, 2008 32 / 37
Recommend
More recommend
