Edit Timelines & Efficient Streaming of Media Mangala Prabhu and Eric Reinecke
Agenda Part I: Trailers at Netflix ● Part II: Edit Intelligence In Pipelines, OpenTimeLineIO ●
Trailers at Netflix Mangala Prabhu, Compute and Storage Infrastructure @ Netflix
What do we in CSI? Managing cloud infrastructure for media processing ● Cloud compute efficiency ● Secure cloud storage of media ● Media transport layer ●
Traditional studios - storage on premises Netflix studios - storage in the cloud AWS storage Artists Local storage Artists Location1 Artists Location 2
High quality full Original trailer creation process length video Sound mixing High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video
High quality full Original trailer creation process length video Sound mixing High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Original trailer creation process length video Sound mixing 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Original trailer creation process length video 1 day Sound mixing 12+ days High Proxy trailer quality Conform in Edits in Adobe Proxy full length trailer Adobe Premiere Premiere video 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Original trailer creation process length video 25GB - 900 GB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Proxy trailer quality Conform in Edits in Adobe Proxy full length trailer Adobe Premiere Premiere video 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Original trailer creation process length video 25GB - 1TB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Editor’s pain point length video 25GB - 1TB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Editor’s pain point length video 25GB - 1TB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
How do we do it? Which parts of the movie do we really want? ● How to make this trailer length high quality video appear as a full length video? ●
How do we get the artist’s creative decisions? The artist decides on what goes into the trailer ● Adobe Premiere can export this decision into human readable format - an EDL ● file
EDL ( Edit decision list) EDLs have the timecodes from the proxy source that made it in the trailer ● and where it is placed in the trailer. Input Time codes Output Time codes
What bytes to download? EDL parser ● Gives expected time ranges ○ Movie metadata in DB ● fps - map time interval to frames ○ Index file - map frame to a byte range ○
Parts needed for trailer What bytes to download?
Parts needed for trailer What bytes to download? Non interesting bytes
Parts needed for trailer What bytes to download? Non interesting bytes Non interesting bytes Non frame - header info
Netflix tool - MezzFS (FUSE wrapper) Mounts cloud objects as local files ● Streams bytes from cloud storage to the user’s workstation ● Option to cache streamed bytes ● Streaming a cloud object from a byte offset ● Lets user set the context of “interesting bytes” versus ● “non-interesting bytes”
How to fake bytes? MezzFS (FUSE wrapper) ● User’s workstation MezzFS libfuse Userspace Kernel VFS FUSE
How to fake bytes? MezzFS (FUSE wrapper) ● User’s workstation MezzFS Seeking bytes for trailer…. libfuse Userspace Kernel VFS FUSE
How to fake bytes? MezzFS (FUSE wrapper) ● User’s workstation MezzFS stream of 0s Seeking non trailer bytes…. libfuse Userspace Kernel VFS FUSE
High quality full Original trailer creation process length video 25GB - 1TB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ●
High quality full Trailer creation process now length video 25GB - 1TB ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours download time ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ● EDL file
High quality trailer Trailer creation process now length video 25GB - 1TB tens of GBs ● 1 day 1-13 episodes / Film ● Sound mixing 2-10 hours order of minutes ● 12+ days High Low quality quality Conform in Edits in Adobe trailer Proxy full length trailer Adobe Premiere Premiere video 1 hour 300 - 500 MB ● 1-13 episodes / Film ● 2 mins - 5 mins download time ● EDL file
Original Model Current Model Norm Macdonald (1 min Trailer) Norm Macdonald(1 min Trailer) 6 episodes 268 GB / 2 hr 40 mins 6 episodes 11.26 GB / 4 mins ● ● Bordertown Recap(3 mins Recap) Bordertown Recap(3 mins Recap) 11 episodes 985 GB / 9 hrs 11 episodes 70 GB / 18 mins ● ●
Benefits of the approach Reduced download times ● No heavy disk space requirements ● Security advantage ● Lets creative folks to focus more on their creative work ● Headline
Edit Intelligence In Pipelines Eric Reinecke, Encoding Team @ Netflix
What do we do? Video and Audio encoding at scale ● VMAF Perceptual Video Quality Assessment ● Spearhead development of new codecs ● Media asset analysis and title metadata management ● Workflow tools for the asset creation pipeline ●
The Three Rewrites
Part II Agenda Timeline aware pipelines ● Some of the ways the edit has moved through pipelines ● How does OpenTimelineIO enable timeline-aware pipelines ●
What would my movie look like if I shipped it right now? Play Pitch ?
Where should I focus my energy? How many frames do I need to animate for this shot? What does the shot I’m reviewing look like in the context of all the other shots? How long is the movie running right now? What credits are shown at what time that I need to have translated? How many visual effects shots are we up to?
What are the “Interesting Bytes”? 15 Hours data transfer 22 minutes
All I have to do is get an EDL?
Option 1: CMX EDL CMX Editor CMX 3600 Keyboard
AUTO_ASSEMBLE_DIRECTIVE: ‘’ WAIT ’' /* Stop auto-assembly when the following edit is encountered. */ | ’’SKIP” /* Do not perform the following edit. */ | ‘’ BELL ’' / *Sound an audible indicator before performing the following edit.*/ ; EDL Format Auto-Assemble directives
TITLE: dissolve_test_2 FCM: NON-DROP FRAME 001 TST V C 01:00:04:05 01:00:04:10 01:00:00:00 01:00:00:05 * FROM CLIP NAME: clip_A 002 TST V C 01:00:04:10 01:00:04:10 01:00:00:05 01:00:00:05 002 TST V D 010 01:00:08:04 01:00:08:14 01:00:00:05 01:00:00:15 * BLEND, DISSOLVE * FROM CLIP NAME: clip_A * TO CLIP NAME: clip_B 003 TST V C 01:00:08:14 01:00:08:19 01:00:00:15 01:00:00:20 * FROM CLIP NAME: clip_B EDL sample
Credit: @GrahamFischer
Credit: @GrahamFischer
Option 2: Advanced Authoring Format (AAF)
Credit: AAF Association - AAF Edit Protocol
Option 3: Final Cut Pro XML Final Cut Pro 7 Final Cut Pro X
<xmeml version="4"> <sequence id="sequence-2"> <name>dissolve_test_2</name> (...) <media> <video> <track> <clipitem frameBlend="FALSE"> <name>clip_A</name> <file id="file-1"/> <duration>10</duration> <start>0</start> <end>-1</end> <in>86501</in> <out>86516</out> </clipitem> <transitionitem> <start>5</start> <end>15</end> (...)
The EDL Landscape
Recommend
More recommend
