Collaborative View Synthesis for Interactive Multi-view Video - PowerPoint PPT Presentation

Collaborative View Synthesis for Interactive Multi-view Video Streaming Fei Chen, Jiangchuan Liu, Yuan Zhao , Edith Cheuk-Han Ngai

Outline • Background • System Description • View Synthesis Collaboration Strategy • View Synthesis Algorithm • Evaluation • Conclusion

Background

Background Character Multiview video enables users to enjoy the video from different viewpoints. Requirements for multiview streaming Rendering Quality: to reduce disparity of interview and smooth • the view sweeping process. Efficiency: to guarantee the availability of interactive • application Bandwidth Scalability: adaptive to available bandwidth of • users

System Description Methods • Multi view video coding • Scalable video coding • View Synthesis Collaboration Strategy Multiview Streaming Structure

View Synthesis Collaboration Shi- ¡synthesis Middle ¡synthesis Different strategies to generate visual views

View Synthesis Algorithm • DIBR: Depth image based rendering (a)Le- ¡Reference ¡View ¡(b)Right ¡Reference ¡View ¡(c)Synthesized ¡View ¡ (d)Original ¡View

View Synthesis Algorithm • S-DIBR: Shift depth image based rendering

S-DIBR • The shift value from the main reference view to the visual view is: fdn N S , where n 1 , 2 , ... = λ = 0 N 2 • The ¡shi- ¡value ¡from ¡the ¡visual ¡view ¡to ¡the ¡assistant ¡ reference ¡view ¡is: ¡ fd n N S ( 1 ), where n 1 , 2 , ... = λ − = 1 N 2 • Therefore ¡we ¡have ¡following ¡relaConship: ¡ A P P , P P ⎧ ⊂ = ⎪ i i 1 + ⎨ M P P , P P ⎪ ⊂ = ⎩ i 1 i +

View Synthesis Analysis Different ¡view ¡comparison

View Synthesis Analysis Synthesized ¡with ¡different ¡reference ¡views

Rendering Quality Rendering ¡quality ¡comparison ¡between ¡W-­‑DIBR(warping ¡DIBR) ¡and ¡ S-­‑DIBR(shi- ¡DIBR) • Similar rendering quality for 1 visual view • S-DIBR keeps the performance stable as the number of visual views increases

Efficiency • The computation latency: T N ( T T T T ) NT = + + + = + δ p m t h p • The ¡Cme ¡cost ¡in ¡pixel ¡projecCon ¡for ¡S-­‑DIBR: N T ( S ) T ∑ = p j , p j 1 = M A M A M A 2 K [( P P ) ( P P )...( P P )] = + + + + 1 1 2 2 N / 2 N / 2 j j N / 2 N / 2 M A 2 KP [ ] ∑∏ ∑∏ = α + α i i j 1 i 1 j 1 i 1 = = = = • And ¡we ¡have ¡the ¡computaCon ¡latency ¡reducCon: j j N / 2 M A ( ) ∑ ∏ ∏ α + α T ( D ) T ( S ) − i i j 1 i 1 i 1 1 = = = ϕ = = − T N

Efficiency • The computation latency reduction aggregates as the number of visual views increases computaCon ¡latency ¡comparison

Bandwidth Scalability scalable ¡rendering ¡with ¡different ¡bitrates

Conclusion • A collaborative view synthesis strategy for multiview streaming system • S-DIBR algorithm with rendering quality, efficiency and bandwidth scalability

Thanks Q & A