Efcient Design Of Multi-ormat Video Decoders Dr Doug Ridge
Agenda • The Increasing Challenge Of Video Decoding • Video Decoder Implementaton Optons • Hardware Technology Comparison • Top-Level Video Decoder Design Consideratons • System Level Challenges • Designing A Robust Decoder • Verifcaton Methodology • Summary
The Increasing Challenge Of Video Decoding Video traffic and applications Video resolutions and frame Silicon area and power are becoming pervasive rates are quickly increasing – consumption are key cost UHD is 480Mpixels/sec factors Time to market pressures Cannot afford to miss market push SoC companies to window – no re-spins, must license IP trust IP supplier
Video Decoder Implementaton Optons • Fully SW-based decoder running on fast mult-core processors • Highly feeible and portable • Need very large, power hungry processors • Fully HW-based implementaton running in dedicated HW • Lowest cost, lowest power • Completely infeeible • A spectrum of mieed HW/SW architectures in between • Optmum point on spectrum driven by many variables • Target technology • Achievable clock rate • Formats to be supported • Resoluton and frame rate
Hardware Technology Comparison • FPGA implementaton • Clock rate of around 200MHz achievable • Many on-chip memory & DSP resources available • Hardware bug not normally catastrophic • Generally feable with an HDL change • SoC implementaton • Clock rate potentally >600MHz • More cycles to utlize and more opportunity for logic re-use • Hardware bug potentally catastrophic • SoC re-spin can incur huge tme and cost penaltes
TopiLevel Video Decoder Design Consideratons • Power & silicon area always important • Sample applicatons • Not just in mobile and low cost • VR headset applicatons e.g. VR headsets • Closed system allows more feeibility • Packaging and cooling costs signifcant in • Ultra low latency required all SoCs • Ultra low power required • Silicon area grows as a result of • Set top boe increased feeibility in the soluton • Mult-format and mult-stream • Mult-format needed • Mult-stream • High, mid and low-end potental • Image resoluton • Frame rate
System Level Challenges • Decoding video bitstream alone is not enough • Decoder design needs to consider real use cases • Use case eeamples • Mult-stream decoding with single decoder • Need to conteet switch between streams • Need ability to save lots of conteet • Need ability to switch frame store management setngs • Dynamic resoluton change handling • Low power modes • Disable blocks when not needed • Completely power down decoder when idle • Handle seek, fast forward & fast rewind operaton • Smooth fast forward needs faster than real-tme decode • All of these require sofware level control of the decoder
Designing A Robust Decoder • Robust to system integraton diferences • E.g. Memory system latencies • Robust to corrupted streams • Cannot hang under any circumstances • Needs to have good error concealment • Robust to non-compliant streams • Spec/standards ambiguites for eeample • Decoder architecture can help with robustness and feeibility • Dedicated HW for area/power reasons • SW control to be able to handle these aspects • These things must all be covered in the verifcaton methodology • Robust decoder comes from years of eeperience of practcal deployments
Verifcaton Methodology • Simulaton only methodology is not an opton for video codec verifcaton • FPGA prototyping or emulaton is necessary • Needs an automated regression system • In case of mult-format decoder, compleeity scales • Each format requires testng with thousands of streams • Each stream contains hundreds of frames • Test set can become huge • Range of test data • Standards compliance, commercial stress streams, corrupt streams, known issue streams
Example: CS8141 ‘Malone’ Video Decoder • Consideraton given for TSMC 28nm process due to target of many end customers • Tradeofs can be made to determine best process ft • Cost analysis • Factor in other IP in the system 40nm 28nm 16nm HEVC 4Kp30 HEVC 4Kp60 HEVC 4Kp120 HEVC 4Kp60 HEVC 4Kp120 (critcal path (critcal path block replicaton) block replicaton)
Example: CS8141 ‘Malone’ Video Decoder CPU • Mult-format, mult-stream video decoder APB Interrupt Control Registers • Supported formats • VP9 Profle 0, 2 @L5.1 Inverse Stream Transform Pre-Parser • Entropy H.265 HEVC MP@L5.1 MCX Decoders Dequant Spatial Meta Prediction Stream De-blocking • Data Merge H.264 AVC/MVC BP/MP/HP @L4.2 CABAC Parser Filters Queue CAVLC UVLC • VC-1 SP/MP/AP Huffman MV Motion Prediction Compensation • 32B W-Cache MPEG-2 MP/HL Re-Sample Filter • MPEG-4.2 SP/ASP • Memory access controller H.263 / Sorenson Spark 2D R-Cache On-chip Buffer • DivX 3.11 + GMC • China AVS-1 up to L6.1, AVS+ DTL-R DTL-W DTL-R2D DTL-W2D • To Display Real Media RV8/RV9/RV10 From Decode Decoded PES/ES Demux External DDR Meta Frames Video Memory System • ON2 / Google VP6 / VP8 Data Stream • BL JPEG / MJPEG • Technology is silicon proven in SoCs down • Performance to 16nm • VP9 & HEVC @ 4Kp60, AVC @ 4Kp30 • Other formats @ 1.5 - 2e HDp60 • JPEG ~80Mpieels/sec 4:2:0 • Optmized for area, but scalable to support higher rates
Summary • Architecture needs to be defned with capabilites of target technology in mind • Best architectures are result of end-applicaton eeperience • Architected at a system level rather than a functonal level • Building on eeistng architectures minimizes both tme-to- market and silicon area • Fleeible architecture allows trade-ofs to be made for target technology
More Informaton htp://www.amphionsemi.com +44 (0)2895 609 600 @AmphionSemi info@amphionsemi.com htp://www.linkedin.com/company/amphion-semiconductor-ltd-/
Recommend
More recommend
