2D-to-3D conversion: market overview, conversion workflow and basic - PowerPoint PPT Presentation

2D-to-3D conversion: market overview, conversion workflow and basic rules for good 3D Dr.-Ing. Sebastian Knorr (Director R&D and Knowledge Management, MD) - imcube labs GmbH, Berlin 15.03.20 11

Overview  Market overview - Cinema market - Home entertainment market  Conversion workflow - High quality conversion - Realtime conversion  Basic rules for good 3D - Conflicts of depth cues and binocular rivalry  Conclusion 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 1 11

Market overview – 3D movies 100 90 80 70 60 50 40 30 20 10 0 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 2 11

3D cinema screen penetration Source: PwC Report Nov. 2010 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 3 11

3D vs. 2D (turnover at box offices) 3D gross (in mill. $) % screens 3D Total gross (in mill. $) 3D % of gross 3D ratio x Chicken Little 17,3 1,5% 314,4 5,5% 3,67 Monster House 15,3 4,2% 140,2 10,9% 2,60 Meet the Robinsons 47,9 13,2% 169,3 28,3% 2,14 Monsters vs. Aliens 71,5 18,0% 161,6 44,0% 2,44 Bolt 60,0 - 180,0 33,0% - My Bloody Valentine 44,2 47,0% 51,5 85,0% 1,81 Coraline 51,9 - 75,2 69,0% - Avatar 1771,7 31,1% 2.684,4 66,0% 2,11 Average 260 19,2% 472,1 42,7% 2,46 Source: Screen Digest Example: Monsters vs. Aliens  Less than 20% of the screens were 3D capable  More than 40% of the revenues came from 3D  3D achieves around 2.5 times the turnover at box offices

2D and 3D box office in the US and Canada • in 2010: 20% of box office came from 3D • 8 of the top 20 grossing films were 3D! Source: MPAA 03/2010

3D @ home  In 2010 the major TV makers, including Samsung, LG, Sony and Panasonic, expand to more than 40 3D TV models  First 3D TV channels in 2010 from ESPN, DirecTV and Sky  3D Blu-ray players from all major home entertainment manufacturers  3D over IP: e.g. YouTube 3D Forecast of 3D TV unit shipments by regions

Worldwide 3D TV channels Currently: about 40 3D TV channels are existing or are announced for 2011 Source: PwC Report Nov. 2010 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 7 11

3D @ home Source: PwC 11/2010  According to market analysis organization PricewaterhouseCoopers up to a quarter of the US households will watch 3D TV at home in 2014  Research firm In-Stat predicts that one in five TVs purchased in the US in 2013 will be 3D sets. Sales are expected to grow by 231 % between 2010 and 2011

Is Enough 3D Content Available? • Currently: about 40 3D Blu-ray titles are available • chicken-and-egg problem • The lack of „A‟ -list 3D content has created a second-hand market: e.g. $400,- for Avatar in 12/2010 • 2D-to-3D conversion may reduce the enormous gap 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 9 11

Overview  Market overview - Cinema market - Home entertainment market  Conversion workflow - High quality conversion - Realtime conversion  Basic rules for good 3D - Conflicts of depth cues and binocular rivalry  Conclusion 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 10 11

Classification of conversion methods  JVC, Samsung, DDD, HDLogix, ...  bad quality => headache, sickness 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 11 11

Recovering of depth information ?  depth map  original image 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 12 11

Perception of depth  Monocular depth cues - Motion parallax, depth from motion, light and shading, relative size, familiar size, interposition, blur, perspective, arial perspective, texture gradient, accommodation -  11 monocular depth cues  Binocular depth cues - Convergence and stereopsis (retinal disparity) -  2 binocular depth cues 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 13 11

Automatic and realtime conversion  error-prone depth maps: conflicts of depth cues, pseudo 3D  visual discomfort  please: keep your hands off realtime conversion!!! 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 14 11

Workflow principles for high-quality 2D-to-3D conversion  Rotoscoping/ Object Segmentation  Depth Assignment/ Depth Grading (3D-Compositing)  Inpainting (Occlusion Filling)  Rendering 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 15 11

1) Rotoscoping/ object segmentation  Commercial Tools: Motor/ Mocha from Imagineer Systems, Adobe After Effects CS5 (Roto-Brush), Eyeon Fusion 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 16 11

2) Depth Assignment/ 3D-Object-Shaping to increase object roundness and to reduce cardboard-effects  simple and efficient 3D-object-shaping directly within the 2D-image  depth map to shift pixels according to their depth in a second stereoscopic view  ellipse tool  mesh-grid tool 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 17 11

3) Occlusion Handling - Inpainting original left view virtual right view ? disoccluded background foreground object 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 18 11

Interpolation vs. Patch-based Inpainting 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 19 11

Overview  Market overview - Cinema market - Home entertainment market  Conversion workflow - High quality conversion - Realtime conversion  Basic rules for good 3D - Conflicts of depth cues and binocular rivalry  Conclusion 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 20 11

Visual discomfort  Reasons for visual discomfort: - Conflicts of depth cues and binocular rivalry  When all cues line up, stereo is excellent  In case of a conflict of cues, stereo breaks down  Not all cues have to be “perfect”, but it helps 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 21 11

Conflicts of depth cues 22

Stereopsis vs. interposition Example: frame violation (or “framing”) - object has negative parallax (in front of the screen) - but is truncated by the image borders 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 23 11

Stereopsis vs. depth from motion 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 24 11

Binocular rivalry in 3D projects mostly occurs in:  3D live production 1. Vertical misalignment  3D live production 2. Luminance  3D live production 3. Reflections, Flares, Polarization  3D live production 4. Contamination  3D live production 5. Depth of field  3D live production 6. Pseudo 3D  realtime conversion 7. Partial pseudo 3D  3D live production 8. Synchronization  3D live production 9. Hyperconvergence  3D live production 10.Hyperdivergence  realtime conversion 11.Depth mismatch  conversion 12.Visual mismatch  all 13.Ghosting 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 25 11

Vertical misalignment & reflections, flares vertical misalignment: reflections, flares and polarization: 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 26 11

Hyperconvergence & hyperdivergence hyperconvergence hyperdivergence 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 27 11

Screen size e * D d perceived depth: e s

e * D d Screen size e s screen of half size: compression of the depth range screen of double size: expansion of the depth range  hyperdivergence!!!

3D-Screen-Size-Adjustment for perfect and comfortable 3D viewing conditions  Mobile 3DTV (small  3DTV (medium  3D cinema (large screen size) screen size) screen size) 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 30 11

From 2D to Stereo 3D to Multiview 3D for the next generation of 3D display technology 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 31 11

Conclusion - 3D Blu-ray is coming!  There is an enormous mass market potential for 3D, especially for the home entertainment market  Blu-ray currently offers the best way of delivering 3D to the home  3D is an additional and maybe the best driver for Blu-ray  BUT : 3D Blu-ray has to be positioned in the high quality sector to generate strong Blu-ray sales  Visual discomfort may dilute the overall excitement about 3D  An imcube product for the Blu-ray market will be available in the middle of 2011 15.03.20 2D-to-3D conversion: market overview, Conversion workflow and basic rules for good 3D 32 11