Multimedia Systems - Image & Video Capture Video An image is - PowerPoint PPT Presentation

Video - Basics September, 2000 Multimedia Systems - Image & Video Capture Video � An image is captured when a camera scans a scene � Colour => Red (R), Green (G) and Blue (B) array of digital samples � Density of samples (pixels) gives resolution � A video is captured when a camera scans a scene at Joemon Jose multiple time instants www.dcs.gla.ac.uk/~jj/teaching/demms4/ � Each sample is called a frame giving rise to a frame rate (frames/sec) measured in Hz Tuesday, 15 th January 2008 � TV (full motion video) is 25Hz � Mobile video telephony is 8-15 Hz … jerky 15/01/2008 video 2 Image Capture Image Data (RGB) Colour still image: � � 420 x 315 pixels, 8 bits/pixel = 387KB Red Green Blue 8 bits: 0-255 (R,G,B)=(153,102,204) (R,G,B)=(17,0,0) 15/01/2008 video 3 15/01/2008 video (R,G,B)=(204,153 205) 4

Video - Basics September, 2000 Video Technology: Human Visual Perception generating a colour � Mixing three primary colours in varying proportions, the frame buffer (2 D array of perception of different colours can be created phosphor dots 24 bit values) colour guns on display � Human eye build up of red � Cones to perceive colour � By exciting retina using different intensities of the three primary colours, the same colour may be perceived by the <128, 128, 255> green brain even if its unique wavelength is not present. What RGB value you 8 bits per colour blue see 15/01/2008 5 video 6 15/01/2008 video Human Information Colour processing � Identical colour combinations can cause different � Colour is a visual feature Presence and distributions of colour sensation under different conditions � colours induce sensations and which is immediately � Likewise two different colour can be perceived conveys meanings in the perceived identical … observer according to specific rules � the human eye & brain Representing colour on digital � Salient chromatic � Interpolation � images and reproducing properties are captured � Pictures and events that can still be identified as accurately on output devices separate are not at all straightforward � Colour interaction in the brain � Colour can add great Distances in colour space � Adaptation � should correspond to human value to an image � General-brightness adaptation perceptual distance � Lateral adaptation � Chromatic adaptation 8 7 15/01/2008 video 15/01/2008 video

Video - Basics September, 2000 Colour Space Representation of Colour Stimuli � To deal with colour we need to quantify it in some way � Points in three � Hardware-oriented dimensional space models � gives us the notion of colour space or domain � Calorimetric models � RGB, CMY, YIQ � User-oriented models � CIE Chromaticity diagram � Hierarchy of colour sets � HLS, HSV, HSB � Physiologically inspired � Perceivable by human beings models � Displayed on a monitor screen � CIE XYZ, RGB � Calculated and stored in a frame memory � Psychological models � HSV, 9 15/01/2008 video 10 15/01/2008 video Video Technology: Video Technology: representing colour Colour Models: RGB monochrome � RGB = Red Green Blue � � bilevel � directly modelled in device (i.e., corresponds to colour � one bit/pixel: 0 = black, 1 = white guns in display) � grey-scale � easy to implement � e.g., 8 bits/pixel = 256 intensities colour � � not based on visual (perceived) colours � value for each colour gun � no of bits gives colour range � not perceptually uniform � e.g., 24 bits = 8 bits for red, 8 bits for green, 8 bits for blue � colour depth 11 15/01/2008 video 12 15/01/2008 video

Video - Basics September, 2000 Video Technology: Video Technology: Colour Models: RGB Colour Space Colour Models: RGB Colour Space -,-,z Cyan Blue Blue (0,1,1) (0,0,1) Cyan -,y,- X,-,- Magenta Magenta White White (1,0,1) (1,1,1) Black Black Green (0,0,0) Green (0,1,0) Red Yellow Yellow Red (1,0,0) (1,1,0) 13 15/01/2008 video 14 15/01/2008 video Video Technology: Video Technology: Colour Models: RGB Colour Models: HSV � Colour is labeled as a relative weights of three primary � HSV = hue, saturation, value (intensity) colours, in an additive system using the primaries Red, � “painter’s model” Green, Blue � better model for representing colours as we see them � It is perceptually non-linear space (“I want a bright highly saturated apple green.”) � Equal distances in the space do not necessarily correspond to perceptually equal sensation � can be converted to/from RGB � Non-linear relationship between RGB values & the � like RGB, axes not perceptually uniform intensity produced in each phosphor dot, low intensity values produce small changes in response to screen � variant: HLS (hue, lightness, saturation) � It is not a good colour description system 15/01/2008 15 video 16 15/01/2008 video

Video - Basics September, 2000 Video Technology: Video Technology: Colour Models: HSV Colour Space Colour Models: HSV Non-linear transformation of RGB cube � Hue : quality by which we distinguish one family from others � V Chroma: quality by which we distinguish a strong colour from weak � ones Green Yellow Value: It is that quality by which we distinguish a light colour from � a dark one Cyan Red H corresponds to selecting a colour; S corresponds to selecting the � amount of white; selecting V corresponds to adding black Blue Perceptually non-linear Magenta � � Perceptual in the sense that we are using attributes that we normally think of � Attributes are not independent h variant: HLS (hue, lightness, saturation) s � 15/01/2008 17 video 18 15/01/2008 video Image Data (YUV) Video Technology: Colour Models: YUV � colour model used for TV signal transmission � Y represents luminance (intensity of monochrome signal) U (col. diff.) V (col. diff.) � U,V carry separate colour information (colour difference values) Y (luminance) RGB � Y = 0.2125R + 0.7154G + 0.0721B � U = B-Y, V = R-Y � typically, Y contributes most to signal bandwidth See: � Y=230 � [A.K. Jain, Fundamentals of Digital Image Processing , Prentice Hall, 1988] Y=127 15/01/2008 19 video

Video - Basics September, 2000 Video Technology: Video Technology: CIE Colour Specification System Colour Models: CMYK � Commission Internationale d’Éclairage � CMYK = cyan, magenta, yellow, black � colour labelling system � “printer’s model” � “XYZ” space � a subtractive model � international standard (1931) � set of practically available CMYK colours (“process colours”) are not equivalent to RGB set � based on colour matching functions determined by experiments with human subjects � gives uniform colour spaces � needs transformation into one of the other models 15/01/2008 21 video 15/01/2008 22 video Video Sequence Image & Video Capture Red Green Blue Consists of number of frames � � Images produced by digitising time-varying signal 8 bits: 0-255 generated by the sensors in a camera � Bit-mapped images Camera � � Circuitry Inside a Camera � Purely digital signal (data stream) is fed into a computer via a high speed interface Y (luminance) � IEEE 1394 (FireWire) U Computer � � Broadcast video is fed into a video capture card attached V Time to the computer 0(black), … ,255(white) t 1 (sec) t 2 (sec) t N (sec) � Video capture card- analogue signal is converted into a digital form 15/01/2008 video 23 24 15/01/2008 video

Video - Basics September, 2000 Pushing the hardware Video Data � Desktop PC � CIF (352 x 288), 8 bpp, � Consumers expectations are based on broadcast 30hz = 8.7 MB/sec television � 30 sec clip = 261 MB � Video to mobile device � Consumer equipment plays back at reduced frame rate � QCIF (176 x 144), 8 bpp, resulting in jittery- dropped frames 30 hz = 2.2 MB/sec � 30 sec clip = 65 MB � In order to accommodate low-end PCs considerable compromises over quality must be made � High Definition TV (HDTV) � 1280 x 720, 24 bpp, 50 hz = 0.4 GB/sec � 2.5 hour movie = 3.4 TB 15/01/2008 video 25 26 15/01/2008 video Persistence of vision Human Perception � If a sequence of still images is presented to our � What frame rate perceived as smooth? eyes at sufficiently high rate (frame rate~40 fps), � No identification of single frames if refresh we experience a continuous visual sensation rather frequency is high enough than perceiving individual images � Perception of 16 frames/s as continuous sequence � Depends on material � A lag in the eye’s response to visual stimuli which results in after images � More sensitive to low frequencies � If the consecutive images only differ by a small � More sensitive to changes in luminance and blue- amount, any changes from one to next will be orange axis perceived as movement of elements within images � Vision emphasizes edge detection � Film projector displays an image twice (24 fps becomes 48 fps) 15/01/2008 27 video 28 15/01/2008 video