Layered Depth Images for Multi-View Coding Vincent Jantet - - PowerPoint PPT Presentation

▶
layered depth images for multi view coding
SMART_READER_LITE
LIVE PREVIEW

Layered Depth Images for Multi-View Coding Vincent Jantet - - PowerPoint PPT Presentation

Sep 15, 2023 119 likes •698 views

Layered Depth Images for Multi-View Coding Vincent Jantet ENS-Cachan, Antenne de Bretagne, Campus de Ker Lann, 35170 Bruz France INRIA Rennes, Bretagne Atlantique, Campus de Beaulieu, 35042 Rennes France Ph.D. Thesis defense, Rennes,

slide-1
SLIDE 1

Layered Depth Images for Multi-View Coding

Vincent Jantet

ENS-Cachan, Antenne de Bretagne, Campus de Ker Lann, 35170 Bruz – France INRIA Rennes, Bretagne Atlantique, Campus de Beaulieu, 35042 Rennes – France

Ph.D. Thesis defense, Rennes, 2012 Collaboration with IRISA, INSA and Brittany Region in Futurim@ge project

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 1 / 43

slide-2
SLIDE 2

Applicative context

Functionalities

3DTV: Depth feeling by stereo-vision simulation FVV: Live viewpoint selection

3DTV FVV

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 2 / 43

slide-3
SLIDE 3

3D video processing scheme

Real World Acquisition Representation Transmission Rendering Displaying Virtual World Server side Client side Multi-cam; Z-cam; . . . Z-Map; Mesh; . . . Projection; . . . TV Screen; Hologram; . . .

Each choice has an impact on following steps

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 3 / 43

slide-4
SLIDE 4

Thesis objectives

Intermediate representation

Compact Bit-Rate scalable

Virtual View rendering

Fast Accurate Acquisition Representation Transmission Rendering Displaying Server side Client side

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 4 / 43

slide-5
SLIDE 5

SoA: Rendering-optimized representations

Multi-View Videos [DTM96] Plenoptic Function (Light Ray) [AB91] Microfacet Billboarding [YSK+02] . . .

Multi-View Video Plenoptic Function Microfacet Billboarding

Advantages

Photo-realistic rendering

Limitations

Huge amount of data

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 5 / 43

slide-6
SLIDE 6

SoA: Transmission-optimized representations

2D plus depth video (2D+Z) [ISO07] Layered Depth Image (LDI) [SGHS98] Billboard Cloud [DDSD03] Polygon Mesh . . .

2D+Z LDI Billboard Cloud Polygon Mesh

Advantages

Compact representation

Limitations

Hard to construct from real scene

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 6 / 43

slide-7
SLIDE 7

Contributions

LDI representation

Compact representation (naturally remove correlations)

JPF rendering method

Point-based projection method which handle artifacts Acquisition Representation Transmission Rendering Displaying Server side Client side

MV+Depth JPF LDI MVC JPF

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 7 / 43

slide-8
SLIDE 8

Table of contents

1

View synthesis (JPF)

2

Layered Depth Image (LDI)

3

LDI-based multi-view compression

4

Conclusions

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 8 / 43

slide-9
SLIDE 9

Table of contents

1

View synthesis (JPF) Projection algorithm Joint Projection Filling (JPF) Rendering results

2

Layered Depth Image (LDI)

3

LDI-based multi-view compression

4

Conclusions

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 9 / 43

slide-10
SLIDE 10

View synthesis: Classical Warping algorithm

Warping Reference view Virtual View View synthesis methods use projection algorithm (warping)

Warping algorithm

Geometrical projection Input: Texture + Depth map

  • From reference View Point

+ Cameras parameters Output: Texture + Depth map

  • Seen from new View Point

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 10 / 43

slide-11
SLIDE 11

View Synthesis: Warping common artifacts

Disocclusions: Occluded areas which become visible Cracks: Small holes due to sampling Ghosting: Boundaries pixels with mixed foreground/background color

Disocclusions Cracks Ghosting

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 11 / 43

slide-12
SLIDE 12

View Synthesis: Classical scheme

Backward Projection Depth-aided Inpainting Backward Warping 3 Depth-aided Inpainting 5 Inpainting 4 Forward Warping 1 Filtering 2 Reference Projection Virtual 1 Forward Warping: Lose pixels connectivity

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 12 / 43

slide-13
SLIDE 13

View Synthesis: Classical scheme

Backward Projection Depth-aided Inpainting Backward Warping 3 Depth-aided Inpainting 5 Inpainting 4 Forward Warping 1 Filtering 2 Reference Projection Virtual 2 Filtering: Fills Cracks and avoids Ghosting

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 12 / 43

slide-14
SLIDE 14

View Synthesis: Classical scheme

Backward Projection Depth-aided Inpainting Backward Warping 3 Depth-aided Inpainting 5 Inpainting 4 Forward Warping 1 Filtering 2 Reference Projection Virtual 3 Backward Warping: Retrieves color from reference view

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 12 / 43

slide-15
SLIDE 15

View Synthesis: Classical scheme

Backward Projection Depth-aided Inpainting Backward Warping 3 Depth-aided Inpainting 5 Inpainting 4 Forward Warping 1 Filtering 2 Reference Projection Virtual 4 Depth Inpainting: Fills disocclusions with mixed FG/BG depth

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 12 / 43

slide-16
SLIDE 16

View Synthesis: Classical scheme

Backward Projection Depth-aided Inpainting Backward Warping 3 Depth-aided Inpainting 5 Inpainting 4 Forward Warping 1 Filtering 2 Reference Projection Virtual 5 Depth-aided Inpainting: Fills disocclusions with mixed FG/BG texture

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 12 / 43

slide-17
SLIDE 17

View Synthesis: Classical scheme

Limitations

Forward Warping: Lose connectivity Depth Inpainting: Can not retrieve structure Texture Inpainting: May fill BG with FB texture Errors are amplified along the process Need for an accurate virtual depth map synthesizing method Introducing a new Joint Projection Filling method

Forward Proj.

  • Dir. inpaint.

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 13 / 43

slide-18
SLIDE 18

JPF: Joint Projection Filling [Jantet et al., 3D Research]

McMillan [McM95]

Pixel scanning order to avoid the use of a zBuffer

Contribution

Also provides pixels connectivity information Reference view Virtual view Projection Process direction

Projection without zBuffer

BackGround pixels are projected before ForeGround pixels

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 14 / 43

slide-19
SLIDE 19

JPF: For rectified views

  • Ref. View

Process direction p q New View p′ q′ Consider p and q two pixels projected on p′ and q′

      

q′

x = p′ x + 1

No artifact

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 15 / 43

slide-20
SLIDE 20

JPF: For rectified views

  • Ref. View

Process direction p q New View Overlap p′ q′ Consider p and q two pixels projected on p′ and q′

      

q′

x = p′ x + 1

No artifact q′

x < p′ x + 1

Overlap

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 15 / 43

slide-21
SLIDE 21

JPF: For rectified views

  • Ref. View

Process direction p q New View Overlap Disocclusion p′ q′ Consider p and q two pixels projected on p′ and q′

      

q′

x = p′ x + 1

No artifact q′

x < p′ x + 1

Overlap q′

x > p′ x + 1

Disocclusion

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 15 / 43

slide-22
SLIDE 22

JPF Generalized: For non rectified views

Process direction Disocclusion Pq′ q′ p′ Pq′: the last pixel projected

  • n row q′

y

  • q′

x ≤ Pq′ x + 1

No artifact q′

x > Pq′ x + 1

Disocclusion

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 16 / 43

slide-23
SLIDE 23

JPF: Results

Process direction

Forward Proj. Navier-Strokes Directional inpaint. JPF Proj.

JPF method well synthesize sharp boundaries and thin fingers

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 17 / 43

slide-24
SLIDE 24

JPF: Conclusion

Advantages

One-step projection, without post-processing Handles cracks and disocclusions during the projection Fills disocclusions with background Preserves geometrical structures

Limitations

Hard to implement on GPU Introduce stretching artifacts if used for texture projection Should be used as a part of a full view synthesis method

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 18 / 43

slide-25
SLIDE 25

View Synthesis: Proposed scheme

Backward Projection. Depth-aided Inpainting Backward Warping Depth-Aided Inpainting JPF Reference Projection Virtual 1 2 3

JPF method replaces for:

Forward Projection Depth Filtering Depth Inpainting

Synthesized Depth used for:

Backward Warping Depth-Aided Inpainting

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 19 / 43

slide-26
SLIDE 26

View Synthesis rendering results

Disocclusions SoA inpaint.1 Daribo’s DAI2 JPF proj. Full-Z DAI 3 Inconsistent virtual depth map ⇒ Texture artifacts JPF synthesize correct depth map which helps DAI

1Navier-Strokes’s inpainting [BBS01] 2Daribo’s Depth Aided Inpainting [DP10] 3Full-Z Depth Aided Inpainting[Jantet et al., 2011a] (inspired from Daribo) Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 20 / 43

slide-27
SLIDE 27

Conclusions on Virtual view synthesis

From a single input video + depth:

JPF: Synthesize virtual depth map DAI: Recreate missing texture Realistic synthesized video, but: Introduces temporal flickering Incoherence between two synthesized views

From multi-view + depth:

Could retrieve real disocclusions textures ⇒ Introducing LDI

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 21 / 43

slide-28
SLIDE 28

Table of contents

1

View synthesis (JPF)

2

Layered Depth Image (LDI) Definition Classical LDI construction Incremental-LDI construction (I-LDI) Object-based classification (O-LDI)

3

LDI-based multi-view compression

4

Conclusions

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 22 / 43

slide-29
SLIDE 29

LDI: Layered Depth Image

Set of pixels, from a reference viewpoint, organized in layers

1st layer (visibles pixels) 2nd layer (occluded pixels) 3rd layer (...) ...

Advantages

Disocclusion: Could be filled by real texture Camera freedom: Virtual camera can move inside a large area Compactness: Eliminate some correlated pixels and reduce data size

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 23 / 43

slide-30
SLIDE 30

LDI: Classical construction scheme [SGHS98]

Every input views are warped

  • nto a reference viewpoint,

and then merged together

Merging policy

Eliminates duplicated pixels

. . .

Reference viewpoint Merging .

LDI View i View j

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 24 / 43

slide-31
SLIDE 31

Classical LDI limitations

Redundancies

Many pixels in many layers, partially empty Scattered pixels distribution Introducing Incremental LDI construction

Compression artifacts

Large depth discontinuities Motion in multi-layer Boundaries in multi-layer Uneasily compressed Introducing Object-based LDI representation

Scattered distribution Compressed depth map

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 25 / 43

slide-32
SLIDE 32

I-LDI: Incremental-LDI construction [Jantet et al., 3DTV]

Iterate for each input view

Use current I-LDI to synthesize one acquired viewpoint Compare with captured view to compute disocclusion texture Insert back textures into the I-LDI .

I-LDI

View synthesis

Viewpoint i

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 26 / 43

slide-33
SLIDE 33

I-LDI: Incremental-LDI construction [Jantet et al., 3DTV]

Iterate for each input view

Use current I-LDI to synthesize one acquired viewpoint Compare with captured view to compute disocclusion texture Insert back textures into the I-LDI .

I-LDI

View synthesis Disocclusions extraction

View i Viewpoint i

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 26 / 43

slide-34
SLIDE 34

I-LDI: Incremental-LDI construction [Jantet et al., 3DTV]

Iterate for each input view

Use current I-LDI to synthesize one acquired viewpoint Compare with captured view to compute disocclusion texture Insert back textures into the I-LDI .

I-LDI

View synthesis Disocclusions extraction Insertion

View i Viewpoint i

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 26 / 43

slide-35
SLIDE 35

I-LDI vs LDI Comparison.

LDI frames: many pixels in many layers, with scattered distribution

1st layer 2nd layer 3rd layer 4th layer ...

I-LDI frames: less pixels and less layers with compact distribution

1st layer 2nd layer 3rd layer 4th layer ...

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 27 / 43

slide-36
SLIDE 36

Classical LDI limitations

1st layer 2nd layer

Compression artifacts

Large depth discontinuities Motion in multi-layer Boundaries in multi-layer Uneasily compressed Introducing Object-based LDI

Compressed depth map Synthesized virtual view

Depth compression artifacts

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 28 / 43

slide-37
SLIDE 37

O-LDI: Object-based LDI [Jantet et al., ICIP]

Organizes pixels into layers to enhance depth continuity

Visible layer Occluded layer

Classical LDI depth layers

Foreground Background

Object-based LDI depth layers

Method based on a region growing algorithm

Region R initialized with pixels where ZFG and ZBG are already defined For each pixel q outside R: Extrapolate ZFG and ZBG Classify q

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 29 / 43

slide-38
SLIDE 38

O-LDI: Classification Initializing

Foreground Unclassified Background

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 30 / 43

slide-39
SLIDE 39

O-LDI: Classification Processing

Foreground Unclassified Background

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 31 / 43

slide-40
SLIDE 40

O-LDI: Classification Results

Foreground Unclassified Background

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 32 / 43

slide-41
SLIDE 41

O-LDI: Background inpainting

Background inpainting

Principe

Exemplar-based inpainting from Criminisi [CPT03] Robust and time-consuming method Preserves texture and structure

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 33 / 43

slide-42
SLIDE 42

O-LDI: Fast mesh-based rendering

Object-based LDI

Continuous layers can be rendered as meshes Foreground mesh is partially transparent

Meshes rendering

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 34 / 43

slide-43
SLIDE 43

O-LDI: Rendering results

Disocclusions Fast SoA inpainting O-LDI rendering

Online inpainting limitations

Fast inpainting, introduces: Artifacts Stretching Temporal flickering

O-LDI advantages

Robust offline inpainting Time coherent rendering Multi-view coherent rendering

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 35 / 43

slide-44
SLIDE 44

O-LDI Conclusions

O-LDI Advantages

Static background along time Compatible with fast mesh-based rendering Depth continuity improves rendering quality Remove unnecessary boundaries ⇒ Should improve compression

O-LDI Limitations

No backward compatibility with 2D decoding scheme

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 36 / 43

slide-45
SLIDE 45

Table of contents

1

View synthesis (JPF)

2

Layered Depth Image (LDI)

3

LDI-based multi-view compression

4

Conclusions

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 37 / 43

slide-46
SLIDE 46

MVD and LDI compression schemes

MVD compression (MVC)

V1 V3 V5 V7

Input views

V ′

1

V ′

3

V ′

5

V ′

7

Compressed views

MVC VSRS Rendering V ′′

6

Final view

LDI compression (MVC)

LDI4 MVC LDI′

4

DIBR V ′′

6

Input LDI Compression Compressed LDI Rendering Final view

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 38 / 43

slide-47
SLIDE 47

"Breakdancing" multi-view video

5 10 15 20 25 30 31 31.5 32 32.5 33 33.5 Bitrate (Mbit/s) PSNR (dB) 10 20 30 82 84 86 88 90 Bitrate (Mbit/s) SSIM (%)

MVC on V.1-3-5-7 — VSRS V.6 LDI from V.4-3-5 — Render V.6

MPEG (MVC/VSRS) LDI coded with MVC I-LDI coded with MVC "Breakdancing" MVD dataset O-LDI coded with MVC

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 39 / 43

slide-48
SLIDE 48

Table of contents

1

View synthesis (JPF)

2

Layered Depth Image (LDI)

3

LDI-based multi-view compression

4

Conclusions

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 40 / 43

slide-49
SLIDE 49

View synthesizing conclusions

JPF: Joint Projection Filling method

Projection with occlusion-compatible pixel scanning order Handles cracks Fills disocclusions with background Preserves geometrical structures

Virtual View Synthesis method with Full-Z Depth Aided Inpainting

First synthesizes virtual zMap to help synthesizing virtual view Preserves sharp boundaries Realistic disocclusions filling

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 41 / 43

slide-50
SLIDE 50

Intermediate representation conclusions

I-LDI: Incremental Layered Depth Image

Iterative LDI construction to avoid layers correlations Less layers Less pixels Compact distribution

O-LDI: Object-based Layered Depth Image

Pixels reorganisation to enhance depth continuity Static background No depth discontinuities ⇒ No compression artifacts Compatible mesh-based rendering

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 42 / 43

slide-51
SLIDE 51

Perspectives

Handle depth map inconsistencies

Non realistic depth maps drive down rendering quality

Improve temporal coherence

During LDI construction During views projection During Depth Aided Inpainting

Use more efficient compression scheme

Consider MPEG 3D-HEVC Explore dedicated Depth Map Compression schemes

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 43 / 43

slide-52
SLIDE 52

Publications

[Bosc et al., 2010] Bosc, E., Jantet, V., Morin, L., Pressigout, M., & Guillemot, C. (2010). Vidéo 3d: quel débit pour la profondeur? In CORESA. [Bosc et al., 2011] Bosc, E., Jantet, V., Pressigout, M., Morin, L., & Guillemot, C. (2011). Bit-rate allocation for multi-view video plus depth data. In 3DTV. [Jantet et al., 2011a] Jantet, V., Guillemot, C., & Morin, L. (2011a). Joint projection filling method for occlusion handling in depth-image-based rendering. 3D Research, 2, 1–13. [Jantet et al., 2011b] Jantet, V., Guillemot, C., & Morin, L. (2011b). Object-based layered depth images for improved virtual view synthesis in rate-constrained context. In ICIP. [Jantet et al., 2009] Jantet, V., Morin, L., & Guillemot, C. (2009). Incremental-ldi for multi-view coding. In 3DTV. [Jantet et al., 2010] Jantet, V., Morin, L., & Guillemot, C. (2010). Génération, compression et rendu de ldi. In CORESA. [Sourimant et al., 2009] Sourimant, G., Colleu, T., Jantet, V., & Morin, L. (2009). Recalage gps / sig / video, et synthèse de textures de bâtiments. In CORESA. [Sourimant et al., 2011] Sourimant, G., Colleu, T., Jantet, V., Morin, L., & Bouatouch, K. (2011). Toward automatic gis–video initial registration. Annals of Telecommunications, 67, 1–13.

[AB91] Edward H. Adelson and James R. Bergen. Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 44 / 43

slide-53
SLIDE 53

Multi-View Coding scheme (MVC)

On Multi-Views Video

. . . . . .

View i View j

Time

Prediction

Predictions: Temporal Spatial Inter-Views

Method

Inter-view prediction with motion vectors

Limitations

Only 25% size reduction for each additional view Motion vectors maladjusted to geometrical correlations

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 45 / 43

slide-54
SLIDE 54

Multi-View Coding scheme (MVC)

On Multi-Views Video

. . . . . .

View i View j

Time

Prediction

Predictions: Temporal Spatial Inter-Views

On LDI

. . . . . .

Layer 1 Layer n

Time

Predictions

Predictions: Temporal Spatial Inter-Layers

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 45 / 43

slide-55
SLIDE 55

Multi-View Coding scheme (MVC)

Method

First layer helps to predict

  • thers layers

Advantages

No geometric distortions Removes inter-layers correlations Static BG layer

On LDI

. . . . . .

Layer 1 Layer n

Time

Predictions

Predictions: Temporal Spatial Inter-Layers

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 45 / 43

slide-56
SLIDE 56

"Ballet" Multi-View Video dataset

10 20 30 40 30 31 32 33 Bitrate (Mbit/s) PSNR (dB) 10 20 30 40 85 86 87 88 89 Bitrate (Mbit/s) SSIM (%)

MVC on V.0-2-4-6 — VSRS V.5 LDI from V.4-2-6 — Render V.5

MPEG (MVC/VSRS) LDI coded with MVC I-LDI coded with MVC "Ballet" MVD dataset O-LDI coded with MVC

Vincent Jantet (ENS-Cachan – FR) Layered Depth Images for Multi-View Coding Ph.D. defense, 2012 46 / 43