MPEG- -4 Standards 4 Standards MPEG-4 Standards MPEG Fernando Pereira Fernando Pereira Fernando Pereira Klagenfurt, Austria, October 2008 Klagenfurt, Austria, October 2008 Audiovisual Compression: from Basics to Systems, Fernando Pereira
The Old Analogue Times: the TV Paradigm The Old Analogue Times: the TV Paradigm The Old Analogue Times: the TV Paradigm • Video data modeled as a sequence of pictures with a certain number of lines • One audio channel is added to the video signal • Video and audio have an analogue representation • User chooses among the available broadcast programmes Audiovisual Compression: from Basics to Systems, Fernando Pereira
Evolving Multimedia Context ... Evolving Multimedia Context ... • More information is in digital form, ... • More information is on-line, ... • More information is multimedia, … • Multimedia information now covers all bitrates and all networks • Applications & services become ‘multimedia’ … • Applications & services become ‘interactive’ … • Internet is growing … Audiovisual Compression: from Basics to Systems, Fernando Pereira
New Technologies, New Needs … New Technologies, New Needs … New Technologies, New Needs … • Having multimedia information available wherever you are, covering a wide range of access conditions • More freedom to interact with what is within the content • Reusing the multimedia content, combining elements of content in new ways • Hyperlinking from elements of the content • Finding and selecting the information you need • Identifying, managing and protecting rights on content • Common technology for many types of services, notably broadcasting, communications, retrieval Demands come from users, producers and providers ! Audiovisual Compression: from Basics to Systems, Fernando Pereira
We and the World around us … We and the World around us … We and the World around us … Audiovisual Compression: from Basics to Systems, Fernando Pereira
Towards owards t the he Real Real World World: : The The Object Object- -b based ased T Towards the Real World: The Object-based Representation Model Model Representation Model Representation • Audiovisual scene represented as a composition of objects • Integration of objects from different nature: A&V, natural and synthetic, text & graphics, animated faces, arbitrary and rectangular video shapes, generic 3D, speech and music, ... • Object-based hyperlinking, processing, coding and description • Interaction with objects and their descriptions is possible • Object-based content may be reused in different contexts • Object composition principle is independent of bitrate: from low bitrates to (virtually) lossless quality … Audiovisual Compression: from Basics to Systems, Fernando Pereira
Object- -based Content … based Content … Object-based Content … Object Sports results: Benfica - Sporting Stock information ... Sports results: Benfica - Sporting Stock information ... Audiovisual Compression: from Basics to Systems, Fernando Pereira
Conventional Audiovisual System Conventional Audiovisual System Conventional Audiovisual System sync & multiplexer demultiplexer compositor enc. dec. ... enc. dec. Audiovisual Compression: from Basics to Systems, Fernando Pereira
Object- -based Audiovisual System based Audiovisual System Object Object-based Audiovisual System sync & multiplexer demultiplexer Comp. Comp. Comp. compositor Comp. enc. Info Info dec. enc. dec. ... ... ... ... dec. enc. Audiovisual Compression: from Basics to Systems, Fernando Pereira
Object- -based Audiovisual System based Audiovisual System Object Object-based Audiovisual System AV objects AV objects AV objects AV objects AV objects AV objects coded coded uncoded uncoded coded coded sync & multiplexer dec. demultiplexer Comp. Comp. Comp. compositor Comp. enc. Info Info dec. enc. dec. ... ... ... ... dec. enc. Audiovisual Compression: from Basics to Systems, Fernando Pereira
Object- -based Audiovisual System based Audiovisual System Object Object-based Audiovisual System AV objects AV objects AV objects AV objects AV objects AV objects coded coded uncoded uncoded coded coded sync & multiplexer dec. demultiplexer Comp. Comp. Comp. compositor Comp. enc. Info Info dec. enc. dec. ... ... ... ... dec. enc. interaction interaction Audiovisual Compression: from Basics to Systems, Fernando Pereira
MPEG- -4: Object 4: Object- -Based Coding Standard Based Coding Standard MPEG-4: Object-Based Coding Standard MPEG • Adopts the object-based model giving a semantic value to the data structure • Integration of natural and synthetic content, both aural and visual • Object-based functionalities, e.g., re- using and manipulation capabilities • Powerful data model for interaction and personalisation • Exploitation of synergies, e.g., between Video Coding, Computer Vision and Computer Graphics Audiovisual Compression: from Basics to Systems, Fernando Pereira
MPEG- -4: Visual Coding Architecture 4: Visual Coding Architecture MPEG-4: Visual Coding Architecture MPEG Visual Visual Object 0 Object 0 Encoder Decoder Visual Visual Object 1 Object 1 Visual Compo- Demultiplexer Encoder Decoder Object Multiplexer sitor Segment . Visual Visual Object 2 Object 2 Encoder Decoder ... ... Visual Visual Object N Object N Encoder Decoder Composition Composition inform. inform. Audiovisual Compression: from Basics to Systems, Fernando Pereira
Basic MPEG- -4 Video Decoding 4 Video Decoding Basic MPEG Basic MPEG-4 Video Decoding video_object_layer_shape Previous Shape reconstructed decoding Coded shape VOP bitstream Demultiplexer Motion Motion compensation Decoded VOP decoding Coded motion bitstream VOP reconstruction Variable Inverse length scan Coded decoding texture bitstream Inverse Inverse Inverse AC/DC quantization DCT prediction Texture Decoding Texture Decoding Audiovisual Compression: from Basics to Systems, Fernando Pereira
Segmentation: a Limitation or not so Much ? Segmentation: a Limitation or not so Much ? Segmentation: a Limitation or not so Much ? Audiovisual Compression: from Basics to Systems, Fernando Pereira
The ‘Weather’ Girl ... The ‘Weather’ Girl ... The ‘Weather’ Girl ... Audiovisual Compression: from Basics to Systems, Fernando Pereira
Segmentation: the Problem that Sometimes does not Segmentation: the Problem that Sometimes does not Segmentation: the Problem that Sometimes does not Exist ... Exist ... Exist ... Audiovisual Compression: from Basics to Systems, Fernando Pereira
Segmentation: Automatic and Real- -Time ? Time ? Segmentation: Automatic and Real-Time ? Segmentation: Automatic and Real Audiovisual Compression: from Basics to Systems, Fernando Pereira
Synthetic Content: Facial Animation and More … Synthetic Content: Facial Animation and More … Synthetic Content: Facial Animation and More … Audiovisual Compression: from Basics to Systems, Fernando Pereira
The MPEG- -4 Tools (1) 4 Tools (1): : The The Codecs Codecs The MPEG-4 Tools (1): The Codecs The MPEG • Efficiently encode video data from very low bitrates, notably in view of low bitrate channels such as the telephone line or mobile environments, to very high quality conditions; • Efficiently encode music and speech data for a very wide bitrate range, notably from transparent music to very low bitrate speech; • Efficiently encode text and graphics ; • Efficiently encode time - c h anging 3D generic objects as well as some more specific 3D objects such as human faces and bodies; • Efficiently encode synthetically generated speech and music as well as 3D audio spaces; • Provide error resilience in the encoding layer for the various data types involved, notably in view of critical channel conditions; Audiovisual Compression: from Basics to Systems, Fernando Pereira
The MPEG- -4 Tools (2) 4 Tools (2): : Systems Tools Systems Tools The MPEG The MPEG-4 Tools (2): Systems Tools • Independently represent the various objects in the scene, notably visual objects, allowing to independently access, manipulate and re- use these objects; • Compose aural and visual, natural and synthetic, objects in one audiovisual scene; • Describe objects and events in the scene; • Provide hyperlinking and interaction capabilities; • Provide some means to protect audiovisual content so that only authorised users can consume it. Audiovisual Compression: from Basics to Systems, Fernando Pereira
MPEG- -4 Systems: the MP4 File Format 4 Systems: the MP4 File Format MPEG-4 Systems: the MP4 File Format MPEG The MP4 file format defines the storage of MPEG-4 audio, scenes and multimedia content using the ISO Base Media File Format. The MP4 file format is life-cycle file format—one in which the files can be used when capturing media, editing it, and combining it; when serving the media as a file download or as a stream; and when exchanging partial or complete presentations. Audiovisual Compression: from Basics to Systems, Fernando Pereira
Recommend
More recommend
