T ODAY 1. Overview of the course 2. Introduction to image coding: - Purpose TSBK01 J ÖRGEN A HLBERG - History - How many samples/pixels/bits? I MAGE CODING AND DATA 3. A fundamental difference: Lossy vs lossless COMPRESSION ahlberg@isy.liu.se coding 4. Models for image & audio coding and the coding methods they imply • PhD in Image Coding ...and a little bit of speech & audio coding • Research Scientist at the Swedish Defence Research Agency (FOI) ... video coding is included as well • Co-founder of Visage Technologies • I have no office at the university, so don’t try to find me there! PART 1: Lectures Problem-solving classes/lessons O VERVIEW OF THE COURSE 1. Introduction 1. Entropy, Markov sources 2. Basic Information Theory 2. Source Coding 3. Source Coding Theory, Huffman Coding 3. Rate-Distortion, Scalar Quantization Course website 4. Arithmetic Coding, Lempel-Ziv Coding, 4. Vector Quantization Lossless Image Coding 5. Predictive Coding http://www.icg.isy.liu.se/courses/tsbk01 5. Coding of Analog Sources, Scalar Quantization 6. Transform Coding 6. Vector Quantization 7. Subband and Wavelet Coding 7. Predictive Coding 8. Miscellaneous 8. Transform Coding 9. Subband and Wavelet Coding Computer-aided classes/lessons 10.Video Coding Course components 1. Scalar and Vector Quantization 11.Speech and Audio Coding 2. To be determined 12.Fractal Coding. MPEG-4 Coding • 12 lectures Laborations • 8 problem-solving classes • 2 computer-aided classes Mandatory! • 2 laborations 1. To be determined. • A written exam 2. To be determined. PART 2: Exam I NTRODUCTION TO I MAGE C ODING Written exam Saturday December 20, 14-18. Mandatory. Literature Telecommunications 1. K. Sayood, Introduction to Data Compression . Purpose 2. Package containing Exercises, Laborations, and A compact digital representation of still or a Table & Formula collection. ! moving images. ! ! o i d u a r o Teachers Computer Image Image coding graphics processing English lectures: Jörgen Ahlberg, ahlberg@isy.liu.se Constraints Laborations, lessons: • Good image quality Harald Nautsch, harna@isy.liu.se • Robust to channel errors Peter Johansson, pejoh@isy.liu.se Digital signal • Real time performance processing • Cheap Examiner, Swedish lectures: Robert Forchheimer, robert@isy.liu.se
I MAGES : H OW MANY PIXELS ? H OW MANY BITS PER PIXEL ? 1152 128 176 2048 Bits per pixel Image type Examples 128 Search image for 144 QCIF archives 1 binary fax 4 simple computer PDA 352 256 graphics 288 256 8 grayscale telephoto, ultrasound 101 376 65 536 Ultrasound images palette colour computer graphics CIF 12 high contrast X-ray 512 720 512 576 5+6+5 = 16 “high colour” old digital photography HDTV 414 720 262 144 2 359 296 8 · 3 = 24 “true colour” digital photography “Classic” standard TV (digital studio (RGB) computer graphics format for image standard) processing systems 8 · 4 = 32 true colour + alpha computer graphics (RGBA) 14 · 4 multispectral remote sensing 14 · (many) hyperspectral remote sensing 640 480 320 240 307 200 76 800 VGA NTSC Low-end computer graphics X-ray, aerial/sattelite/consumer photos: > 1 Mpixels Lenna, 512x512, 8bpp Lenna, 128x128 Lenna, 512x512, 3bpp A UDIO : H OW MANY BITS PER SAMPLE B ITS PER SECOND Lenna, 512x512, 0.5 bpp (JPEG) AND PER SECOND ? Still images CD-quality Fax 2.4 – 14.4 kbit/s Telephoto 4.8 – 128 kbit/s • 16 bits per sample Teleradiology 64 – 128 kbit/s • 44100 samples per second • Two channels Moving images → 1.4 Mbit/s • Video telephony 4 – 128 kbit/s • Often used as reference (“non- Video conferencing 64 – 383 kbit/s compressed audio”). Be careful! Multimedia < 1.5 Mbits/s Digital TV 3 – 6 Mbits/s • Modern compression: 64 kbit/s with HDTV < 22 Mbits/s good quality Audio Digital telephony Digital speech 4 – 16 kbit/s • 8 bits per sample Music 64 – 256 kbits/s • 8000 samples per second • → 64 kbit/s • Modern compression: 4 kbit/s with good quality
E XAMPLE APPLICATION : F AX P ART 3: (T ELE FACSIMILE ) L OSSY VS LOSSLESS CODING Generation 1 (1966) in practise the same as perception-based • FM or AM modulation coding vs data compression • 6 minutes @ 96 lines/inch V IDEO : H OW MANY BITS PER SECOND ? • 4 minutes @ 64 lines/inch compare to coding of analog sources vs coding of digital sources Generation 2 Single sideband etc gives 3 (2) minutes. Fundamental difference! Generation 3 (1970) • Digital communication, runlength coding • <1 minute @ 4.8 kbit/s • Standardised 1980 Generation 4 (1981) • 16 times better quality • Colour L OSSLESS CODING L OSSY CODING Take some digital or analog data and represent it (Data compression, Entropy coding) using as few bits as possible in a way that you can reconstruct the original data as well as possible by Take some digital data, i.e., bits, and represent it some measure. using fewer bits in a way that you can reconstruct Determine what kind of distortion and how much the original data exactly . distortion you can accept. A C OMMON S CHEME Limited by the entropy of the data (source coding No limit in how much you can compress, as long as theorem). Based on information theory. a l n you can accept more distortion. g s i l a a l l i t a g n n i g d g s i i d s Some methods e Some methods g a l t o r t o a l g i t s n D i i A D • Transform coding • Huffman coding Sampling and Lossless Lossy coding • Wavelet coding Quantization coding • Lempel-Ziv coding • Predictive coding • Arithmetic coding • Psychoacoustic/visual coding Some applications • PNG Some applications • GIF • Still images: JPEG, JPEG2K, ... • Moving images: MPEG-1/2/4, • Zip WMV, ... • Lossless versions of JPEG, • Audio: MP3, WMA, RA, ... JPEG2K, WMA. • Speech: GSM (AMR), MPEG-4, ... P ART 4: A MODEL IMPLIES A CODING METHOD A DETERMINISTIC MODEL S IGNAL MODELS FOR The image constists of edges and IMAGE AND AUDIO CODING Object models surfaces: • Object-based coding Object models • Semantic coding • Illumination Deterministic models • Objects • Projections • Interpolation coding • Coding: Store the size and position of the edges. • Contour coding Signal models • Decoding: Interpolate the values between the edges. Statistical models • Deterministic models • Statistical models • Huffman coding ( Interpolative coding , L.D.Davidsson, 1967) • Predictive coding Perception models • Transform coding • Vector quantization • Time/space-frequency models • Psycho-acoustic/visual models Perception models • Time/space-frequency coding • Psycho-acoustic/visual masking
S TATISTICAL MODELS V ARIABLE LENGTH CODING a 00 b 01 Symbol probablities c 10 a 0.5 d 11 b 0.25 -> 2 bits per symbol c 0.125 P SYCHO - ACOUSTIC MODELS d 0.125 Huffman coding Strong tones will mask weaker ones Markov models a 1 Energy b 01 P 11 P 12 c 001 S 1 d 000 P 21 -> 1.75 bits per symbol P 13 P 22 S 2 P 31 P 23 Arithmetic coding S 3 P 32 P 33 Controlable code word lengths Frequency Universal coding Adaptive code word lengths P SYCHO - ACOUSTIC TIME - FREQUENCY V ARIOUS MODELS AND METHODS MODELS • 1D Markov models: predictive coding Better resolution in lower frequencis: Remap! • 2D Markov models: Transform coding • The font model: Dictionary coding P SYCHO - ACOUSTIC MODELS • Vision based models Hearing threshold Not available in this set of slides: Use dB 40 Robert’s. 30 20 10 0 2 4 6 8 10 12 kHz M ODEL - BASED CODING S UMMARY • Image & audio coding - Purpose, Semantic/object-based coding, coding through animation constraints, applications. • History of image transmission. • Pixels per image, samples per second, bits per pixel/sample, bits per second. • Lossy vs lossless coding Signal models → coding methods • • Deterministic models - Interpolation coding, contour coding • Statistical models - Predictive coding, Transform coding, Vector quantization • Perception models - Dual-mode coders, Masking, Time/ Space-Frequency coding • Object/scene models - Object-based/Semantic coding • Self-similarity models - Fractal coding, IFS
Recommend
More recommend
