Codec Chips Tribute to Prof. Goto Jinjia Zhou 1 , Dajiang Zhou 2 , - PowerPoint PPT Presentation

100x Evolution of Vid ideo Codec Chips Tribute to Prof. Goto Jinjia Zhou 1 , Dajiang Zhou 2 , Satoshi Goto 2 1 Hosei University, Tokyo, Japan 2 Waseda University, Kitakyushu, Japan

Prof. Goto has been my supervisor from 2008 to 2015. (M.S -> Ph.D -> PDF) 2

Prof. S. . Goto’s Vid ideo Coding Research Group ► One of the first Full-HD H.264 encoders, first to use SiS DRAM (VLSI’07 and JSSC’09) ► First 4kx2k@60fps H.264 decoder (VLSI’10) ► First 8kx4k H.264 decoder (ISSCC’12) ► First 8Kx4K H.264 (intra-frame) encoder (VLSI’12) ► First 8Kx4K H.264 ME encoder (VLSI’13 and JSSC’14) ► First 8Kx4K HEVC decoder (ISSCC’16 and JSSC’16) 3

Mass media 4

Video codec: encoder and decoder Compressed Source video video stream data Encoder Transmit. 100% ~1% Compression Video camera Channel/ ~1% Storage Restored video data stream Decoder Receiver 100% ~1% Decompression Display device 5

Applications of f video codec chips TV conference Surveillance Small frame delay Ultra-low power High compression High video quality Free-point view …. Automotive …… Enc./Dec. Chip Home entertainment Mobile/Portable 6 Source of the images: http://www.artesanosdecastillalamancha.org/wp-content/uploads/2015/06/28.png http://www.caradvice.com.au/67890/2011-brakes-camera-action-pedestrian-detection-automated-platooning/photos/

8K UHDTV and fr free-viewpoint TV 7680 pixels ≥120fps 25~30fps 7

Video coding standards Compression ratio ~200:1 ~50:1 ~100:1 ITU-T standards H.261 H.263 (+/++) Joint ITU-T & H.265 H.262 H.264 MPEG standards MPEG-2 MPEG-4 AVC HEVC MPEG standards MPEG-1 MPEG-4 1990 1995 2000 2005 2010 2013 8

Hig igh compression at hig igh complexity RAW H.264 (2003) HEVC (2013) 48000Mbps ~480Mbps ~240Mbps ► Recent powerful codecs address the huge video throughput in the communication channel ► Their high compression ratio, however, is at the expense of high complexity 9

Complexity of f vid ideo codecs (n (norm.) 1000 307.2 100 20 10 1 1 1080p/MPEG-2 4K/H.264 8K/HEVC 0.1 Complexity/pixel Throughput Overall complexity 10

Real-time 8K UHDTV codec systems NHK 8K codec (2007) NHK 8K encoder (2013) Our target: Single chip/chipset 11

Memory ry bandwidth issue ► Performance bottleneck  >50GBps BW required for decoding 8K UHDTV  >100GBps BW required for encoding 8K UHDTV ► Power consumption Memory  Majority of power traffic Codec consumed by DRAM traffic ► Fabrication cost DRAM  BW determines chip pin count 12

Data dependency issue ► Video codecs exploit all kinds of data dependencies to strengthen compression  Inter-frame prediction  Intra-frame prediction  Context-adaptive entropy coding (CABAC) ► Data dependencies restricts the degree of efficient parallelism/pipelining  Power and area issues  Performance issue 13

Challenges summarized Transform Entropy & Quant. Coding - Source Frm. Inv. Trans. & Decoder Data Inv. Quant. dependencies Computational complexity Deblocking Filter Intra Prediction Frame Output Motion Compensation Reference Frames Motion Estimation Memory bandwidth requirements 14

Our efforts to address the challenges Reduce memory access / Increase memory bandwidth Bus/interface optimization, 3DLSI System Processing order optimization Embedded compression Algorithm 2-D cache Architecture Reduce complexity Trade-off b/w time & quality Circuits Hardware friendliness, … Device Alleviate data dependencies Processing order optimization Evaluation Predictive execution, … 15

System integration ► FIFO vs RAM  FIFO: simple interface and flexibility  RAM: random accessibility for data reordering ► Proposed BIBO (Block-in-block- out) queues: combines benefits of the two <16>

Word write & block push BIBO queue Block Word ► Words in a block can be written in a random order ► A block can be pushed after all words are written ► Blocks follows first-in-first-out ► Blocks can be in a variable size <17>

Word read & block pull BIBO queue Merged ► Words in a block can be read in a random order ► A block can be pulled after all words are read ► Blocks can be pulled in different sizes as pushed <18>

Merge and split BIBO queue ► Block merging/splitting can be automated by BIBO given both word addressing and block scan follow a Z-scan order <19>

Merge and split Push BIBO queue <20>

Merge and split BIBO queue Pull <21>

Merge and split BIBO queue <22>

Im Implemented video codec chips fr from Goto’s La Lab. 23 Source: http://www.f.waseda.jp/goto/html/chip.html

Vid ideo decoder demo: 4K@FPGA, 8K@chip 24

Performance of f codec VLSI I chip ips Mpixel/s 3981 4000 144x 3000 1990 2000 1000 249 27.6 0 MIT Ours Ours NTU MIT NTT Ours ASSCC'08 ISSCC'12 VLSIC'12 VLSIC'13 ISSCC'13 VLSIC'15 ISSCC'16 H.264 decoder HEVC decoder H.264 encoder HEVC encoder 25

Thanks to all ll members who have contrib ibuted in in the vid ideo codec chip ip desig ign 26

Thank you! ji jinjia.z .zhou.35@hosei.a .ac.jp 27