ROBOTICS 01PEEQW Basilio Bona DAUIN Politecnico di Torino Mobile - PowerPoint PPT Presentation

ROBOTICS 01PEEQW Basilio Bona DAUIN – Politecnico di Torino

Mobile & Service Robotics Sensors for Robotics – 4

Vision � Vision is the most important sense in humans and is becoming important also in robotics � not expensive � rich of information � Vision includes three steps � Data recording and transformation in the retina � Data transmission through the optical nerves � Data elaboration by the brain Basilio Bona 3 ROBOTICS 01PEEQW - 2015/2016

Vision sensors CCD ( Coupled Charge Device, light-sensitive, discharging capacitors ) CMOS ( Complementary Metal Oxide Semiconductor technology ) Basilio Bona 4 ROBOTICS 01PEEQW - 2015/2016

CCD sensors � A CCD sensor consists in a range of capacitors, each accumulating a charge proportional to the quantity of light that is hitting it. The charge in each capacitor is then turned into a numerical value by the camera inner system to produce a picture. � Advantages/Features of CCD Sensors � Conversion takes place in the chip without distortion � CCDs have very high uniformity → Good for HD quality images (not videos) � These sensors are more sensitive → Produce Better Images in Low Light � CCD sensors produce cleaner and less grainy Images → Low-noise images � CCD sensors has been produced for a longer period of time � Disadvantages of CCD Sensors � CCD sensors consume much more power � CCDs are interlaced � Inferior HD videos → Less pixel rates � CCDs are expensive as they require special manufacturing Basilio Bona 5 ROBOTICS 01PEEQW - 2015/2016

CMOS sensors � CMOS sensors can directly make the charge conversion on the generation photosite thanks to their pixel amplifier. Several transistors at each pixel amplify and move the charge using more traditional wires. Charge is then turned into a numericalcal value which correspond to the image. This characteristic give them the ability to avoid several transfers and to increase the processing speed � CMOS do not require any special manufacturing. Most of the digital cameras these days use CMOS Sensor as it reduces cost � Advantages/ Features Of CMOS Sensor � CMOS consumes less power (100 times less than CCD) � CMOS sensors are cheaper � Each pixel can be individually addressed. High reading rate � They produce better HD videos � Disadvantages Of CMOS Sensor � CMOS sensors are also more susceptible – sometimes images are grainy � CMOS sensors need more light for better image � Despite some disadvantages CMOS sensor is widely used in Mobile Phones, Tablets, PDAs and on most of the digital cameras. � CCD cameras produce better images but CMOS sensors are catching up fast with its low power consumption. Basilio Bona 6 ROBOTICS 01PEEQW - 2015/2016

Artificial vision issues � Projection from a 3D world on a 2D plane: perspective projection (transformation matrices) � Discretization effects due to pixels (CCD or CMOS) � Misalignment errors (hardware) Pixel discretization Parallel lines Converging lines Basilio Bona 7 ROBOTICS 01PEEQW - 2015/2016

Camera models � Pinhole camera (aka perspective camera) Basilio Bona 8 ROBOTICS 01PEEQW - 2015/2016

Pinhole camera image planes hole diameter A point images the image is reversed B point images Decreasing the image plane distance or the hole diameter makes the point images sharper Increasing the hole diameter makes the point images brighter Infinite depth-of-field Infinite depth-of-focus Basilio Bona 9 ROBOTICS 01PEEQW - 2015/2016

Camera models � Thin lens camera : the lens has a thickness d that is negligible compared to the radii of curvature of the lens surfaces R 1 , R 2 � Rays are refracted as they go through the lens (refraction index n )   1 1 1   ≈ − − > ( n 1) ; R 0 if convex Thin lens equation   i f R R   1 2 Basilio Bona 10 ROBOTICS 01PEEQW - 2015/2016

Thin lens camera � Thin lens camera is reversible � Rays parallel to the optical axis pass through the focus and viceversa � Rays through the lens center are not refracted � There are two symmetrical foci � True lens shows aberration phenomena lens center optical axis f f Basilio Bona 11 ROBOTICS 01PEEQW - 2015/2016

Aberration Spherical Basilio Bona 12 ROBOTICS 01PEEQW - 2015/2016

Image formation Thin lens approximation ≈ Pinhole camera Principal image plane π ′ Reversed image plane π F Optical axis π 3D object Focal Plane Basilio Bona 13 ROBOTICS 01PEEQW - 2015/2016

Image formation and equations Lens equation − ( p f ) f 1 1 1 = ⇒ = + ⇒ + = pq f p ( q ) − f ( q f ) p q f focal plane image plane real object field of view angle f f focal distance q p object distance image distance pf = q ≈ if p ≫ f then q f − ( p f ) the image plane is approx in the focal plane Basilio Bona 14 ROBOTICS 01PEEQW - 2015/2016

Image formation   p   x   ⇒ =  P p p  y   p     z i f c   P x   i i ⇒ =  P p  y i i k x C     i c i c p x π π F P p z Basilio Bona 15 ROBOTICS 01PEEQW - 2015/2016

Transformations � Coordinate transformation between the world frame and the camera frame � Projection of 3D point coordinates onto 2D image plane coordinates � Coordinate transformation between possible choices of image coordinate frame Basilio Bona 16 ROBOTICS 01PEEQW - 2015/2016

Transformations   c c R t Camera frame   = 0 0 c   T 1 0 0 0   R   0 0 T 1     c   0 1 0 0   c =  T  0 0 0 f i   R   0 0 0 0 1 World frame     i T pix R π ′ R i pix Image plane Optical Rescaling correction Basilio Bona 17 ROBOTICS 01PEEQW - 2015/2016

Reference frames f R 0 World frame P i i i ′ p p c i p i c 0 R C k i c c Optical axis O j j p R i i c c p c pix u Image plane P π π R v Focal plane F pix   →  → translation translation+scale R R R ←  ←  c i pix Basilio Bona 18 ROBOTICS 01PEEQW - 2015/2016

Vector notation � in 3D   T ⇔ =  p x y x R    0 0 0 0 0   T ⇔ =  p x y x R    c c c c c   T ′ ′ ′ ′ ⇔ =  p x y x R    c c c c c � in 2D   T ⇔ =  p x y R    i i i i   T ⇔ =  p R u v  in pixel units   pix pix Basilio Bona 19 ROBOTICS 01PEEQW - 2015/2016

Camera projections � Perspective projection � Orthographic projection p x p if const = → = x f ≈ ⇒ = α x i x p x p i z i x p f p z z small compared large compared to the distance to the distance from the camera from the camera The pixel height of similar subject is different if the distance from the camera varies a lot. On the left the persons have different pixel height while on the right they have approximately similar heights, since their distance from the camera is high and does not vary much Basilio Bona 20 ROBOTICS 01PEEQW - 2015/2016

Projections Basilio Bona 21 ROBOTICS 01PEEQW - 2015/2016

Perspective projection i c P p i z x k C i c c C i P p i x π π π ′ F All points give the same image P’ f P f p x p = ⇒ = − x i p f x − z p f x z i Usually the negative sign is avoided considering the reversed image plane Basilio Bona 22 ROBOTICS 01PEEQW - 2015/2016

Perspective projection p i   p     f x p   x     p x   i z f     p ′ ′   ⇒ =  ⇒ = = P p p P p f y y      c y c p i p     z   p z p f           z z   perspective projection     1 0 0 f 0 0 f     i = ⇒ = = p p p p p P p     0 1 0 0 f 0 i c z i c c c p         z arbitrary positive constant       f 0 0 f 0 1 0 0       i λ = = = p p p P p       0 f 0 0 f 0 1 0 i c c c c             Basilio Bona 23 ROBOTICS 01PEEQW - 2015/2016

Perspective projection Homogeneous coordinates   T   T ɶ =  ɶ =  p p p p 1 p x y 1       c x y z i i i     x x f     = i ⇒ = i p p p f     i y z i y p         i i z Homogeneous ⇓ perspective/projection matrix   p       x x f 0 0 0       p i       y i i c ɶ = = = Π ɶ = Π ɶ p p p y 0 f 0 0 p T p       p z i z i c c c 0 0       z 1 0 0 1 0            1    ⇓ λ = Π i c ɶ p T p i c 0 0 Basilio Bona 24 ROBOTICS 01PEEQW - 2015/2016