ROBOTICS 01PEEQW Basilio Bona DAUIN Politecnico di Torino Mobile - PDF document

04/05/2015 ROBOTICS 01PEEQW Basilio Bona DAUIN – Politecnico di Torino Mobile & Service Robotics Sensors for Robotics – 4 di 23 1

04/05/2015 Vision � Vision is the most important sense in humans and is becoming important also in robotics � not expensive � rich of information � Vision includes three steps � Data recording and transformation in the retina � Data transmission through the optical nerves � Data elaboration by the brain ROBOTICS 01PEEQW - 2014/2015 3 Vision sensors: hardware CCD ( Coupled Charge Device, light-sensitive, discharging capacitors of 5 to 25 micron ) CMOS ( Complementary Metal Oxide Semiconductor technology ) ROBOTICS 01PEEQW - 2014/2015 4 di 23 2

04/05/2015 Artificial vision issues � Projection from a 3D world on a 2D plane: perspective projection (transformation matrices) � Discretization effects due to pixels (CCD or CMOS) � Misalignment errors (hardware) Pixel discretization Parallel lines Converging lines ROBOTICS 01PEEQW - 2014/2015 5 Camera models � Pinhole camera (aka perspective camera) ROBOTICS 01PEEQW - 2014/2015 6 di 23 3

04/05/2015 Pinhole camera image planes hole diameter A point images the image is reversed B point images Decreasing the image plane distance or the hole diameter makes the point images sharper Increasing the hole diameter makes the point images brighter Infinite depth-of-field Infinite depth-of-focus ROBOTICS 01PEEQW - 2014/2015 7 Camera models � Thin lens camera : the lens has a thickness d that is negligible compared to the radii of curvature of the lens surfaces R 1 , R 2 � Rays are refracted as they go through the lens (refraction index n )   1 1 1   ≈ − − > ( n 1) ; R 0 if convex Thin lens equation   i f R R   1 2 ROBOTICS 01PEEQW - 2014/2015 8 di 23 4

04/05/2015 Thin lens camera � Thin lens camera is reversible � Rays parallel to the optical axis pass through the focus and viceversa � Rays through the lens center are not refracted � There are two symmetrical foci � True lens shows aberration phenomena lens center optical axis f f ROBOTICS 01PEEQW - 2014/2015 9 Aberration Spherical ROBOTICS 01PEEQW - 2014/2015 10 di 23 5

04/05/2015 Image formation Thin lens approximation ≈ Pinhole camera Principal image plane π ′ Reversed image plane π F Optical axis π 3D object Focal Plane ROBOTICS 01PEEQW - 2014/2015 11 Image formation and equations Lens equation − ( p f ) f 1 1 1 = ⇒ = + ⇒ + = pq f p ( q ) − f ( q f ) p q f focal plane image plane real object field of view angle f f focal distance q p object distance image distance pf = q if p ≫ f then q ≈ f − ( p f ) the image plane is approx in the focal plane ROBOTICS 01PEEQW - 2014/2015 12 di 23 6

04/05/2015 Image formation   p   x   ⇒ =  P p p  y   p     z i f c   P x   i ⇒ =  i P p  y i i k x C     i c i c p x π π F P p z ROBOTICS 01PEEQW - 2014/2015 13 Transformations � Coordinate transformation between the world frame and the camera frame � Projection of 3D point coordinates onto 2D image plane coordinates � Coordinate transformation between possible choices of image coordinate frame ROBOTICS 01PEEQW - 2014/2015 14 di 23 7

04/05/2015 Transformations   c c R t Camera frame   = 0 0 c T   1 0 0 0   R   0 T 1 0     c   0 1 0 0   c =  T  i 0 0 0 f   R   0 0 0 0 1 World frame     T i pix R R π ′ i pix Image plane Optical Rescaling correction ROBOTICS 01PEEQW - 2014/2015 15 Reference frames f R 0 World frame P i i i ′ p p c i p i c R 0 C k i c c Optical axis O j j p R i i c c p c pix u Image plane P π π R v Focal plane F pix   →  → translation translation+scale R ←  R ←  R c i pix ROBOTICS 01PEEQW - 2014/2015 16 di 23 8

04/05/2015 Vector notation � in 3D   T ⇔ p =  x y x R    0 0 0 0 0   T ⇔ =  R p x y x    c c c c c   T ′ ′ ′ ′ ⇔ p =  x y x R    c c c c c � in 2D   T ⇔ p =  x y R    i i i i   T ⇔ =  R p u v  in pixel units   pix pix ROBOTICS 01PEEQW - 2014/2015 17 Perspective projection i c P p i z x k C c i c C i P p i x π π ′ π F All points give the same image P’ f P f p x p = ⇒ = − x i p f x − z p f x z i Usually the negative sign is avoided considering the reversed image plane ROBOTICS 01PEEQW - 2014/2015 18 di 23 9

04/05/2015 Camera projections � Perspective projection � Orthographic projection p x p if const x = i → x = f x p ≈ ⇒ x = α p i z i x p f p z z small compared large compared to the distance to the distance from the camera from the camera The pixel height of similar subject is different if the distance from the camera varies a lot. On the left the persons have different pixel height while on the right they have approximately similar heights, since their distance from the camera is high and does not vary much ROBOTICS 01PEEQW - 2014/2015 19 Perspective projection p i   p     f x p   x     p   x z i f     p ′ ′ ⇒ =  ⇒ =   = P p p P p f y y      c y c p p i     z   p f z p           z z   perspective projection     1 0 0 f 0 0 f     = ⇒ = = i p p p p p P p     0 1 0 0 f 0 i p c z i c c c         z arbitrary positive constant       f 0 0 f 0 1 0 0       λ = = = i p p p P p       0 f 0 0 f 0 1 0 i c c c c             ROBOTICS 01PEEQW - 2014/2015 20 di 23 10

04/05/2015 Perspective projection Homogeneous coordinates   T   T ɶ =  ɶ =  p p p p 1 p x y 1       c x y z i i i     x x f     = i ⇒ = i p p p f     i y z i y p         i i z Homogeneous ⇓ perspective/projection matrix   p       x x f 0 0 0       i p       y i i c p p ɶ = p y = 0 f 0 0 = Π p ɶ = Π T p ɶ       z i z i p c c c 0 0       z 1 0 0 1 0            1    ⇓ i c λ p = Π T p ɶ i c 0 0 ROBOTICS 01PEEQW - 2014/2015 21 Perspective projection Canonical projection matrix     this is the ideal case f 0 0 1 0 0 0       c c R t       Π i c =  0 0 T 0 f 0 0 1 0 0      c 0 0 T 1         0 0 1 0 0 1 0         ROBOTICS 01PEEQW - 2014/2015 22 di 23 11

04/05/2015 Perspective projection � PP is studied by projective geometry � PP preserves linearity ; lines in 3D correspond to lines in 2D and viceversa � PP does not preserve parallelism � The intersection points in 2D of parallel lines in 3D define vanishing points (points infinitely far away) ROBOTICS 01PEEQW - 2014/2015 23 Camera parameters � Intrinsic parameters: the parameters that link the pixel coordinates of an image point to the corresponding (metric) coordinates in the camera reference frames � Extrinsic parameters: the parameter that define the location and orientation of the camera reference frame with respect to a known world reference frame   W W R t   W = c c ⇒ T 6 parameters   c 0 T 1     � Camera calibration : the procedure to estimate these parameters ROBOTICS 01PEEQW - 2014/2015 24 di 23 12

04/05/2015 Camera intrinsic parameters pix n columns s s , in m x y s R u x pix i s pix x s y s c aspect ratio y pix i m rows i R i u v , v in pixel units j j u = 6 v = 8 ( , ) i   pix o   x =  c  pix o   y camera center in pixel units ROBOTICS 01PEEQW - 2014/2015 25 Camera intrinsic parameters � Focal length f � Transformation between pixel coordinates and camera coordinates � Geometric distortion introduced by the optical lens systems ROBOTICS 01PEEQW - 2014/2015 26 di 23 13