Expanding the Boundaries of the AI Revolution: Changyong Ahn & - PowerPoint PPT Presentation

Expanding the Boundaries of the AI Revolution: Changyong Ahn & Nayoung Lee | March 2019

Outline 1

2

Machine Learning/Deep Learning Use Cases 3

Memory Challenges of Deep Learning Simple View Deep Neural Network Fundamental Concepts Weights x Input … Deep Neural Network Output Σ Weights x Input (Activation function, Compute) Weights x Input = Multiply & Accumulate sum … …… MEM Write Layer MEM Read Mem Memory ry size Top5 5 Er Error Year Ye CN CNN # of layers rs # of Pa Param rameters rs (MB MB) Ra Rate 1998 LeNet 8 60K 2012 AlexNet 7 60 million 240 15.3% 2014 GoogleNet 19 4 million 6.67% Source: Standford 2014 VGG Net 16 138 million 574 7.3% 2015 ResNet 50/152 519 3.6% 4

Memory Solution for ML/DL Systems Memory Sub system hierarchy change “In Package Memory” TSV Region <10ns LLC LLC IPM 50ns Interposer DRAM Substrate(“PCB”) DRAM 100ns – 1us Conventio Co ional DRAM IPM IPM Storage Class Memory Specific & high Fast Storage(SSD) Target Market/Price Broad & Cheap 50-100us Premium Fast Storage(SSD) Standardization JEDEC Semi Custom HDD HDD Qualification Period Relatively short Relatively long ~10ms Price Reliability / Key factors Competitiveness Performance * Source : SK hynix 1) In- Package Memory 2) SCM(“Storage Class Memory”) : 3DXP, PCRAM 5

6

HBM, What’s the difference? GDDR/DDR/LPDDR HBM  KGSD  FBGA  HBM in 2.5D SiP Mold DRAM Slice DRAM DRAM DRAM Slice Molding Molding PCB Substrate SoC DRAM Slice Side Side DRAM Slice DA ball TSV PHY PHY Interposer Directly soldered on PCB or used as a DI Substrate MM 7

HBM Advantages To Achieve 1TB Bandwidth …… More e Bandwid idth th High h Power r Efficien iciency Small ll Form m Factor tor Note: Advil is a registered trademark 40ea of 160ea of 4ea HBM2 in DDR4-3200 Module DDR4-3200 a single 50mm x 50mm Sip HBM BM2E HBM BM3 DDR DDR4 LPDD PDDR4(X (X) GDDR6 GDD HBM BM2 (JED EDEC) (TBD) BD) 3200Mbps 14Gbps 2.4Gbps >3.2Gbps Data rate 3200Mbps (up to 4266 (up to 16Gb 2.8Gbps (TBD) Mbps) ps) x16/ch Pin count x4/x8/x16 x16/x32 x1024 x1024 x1024 (2ch per die) Bandwidth 5.4GB/s 12.8(17)GB/s 56GB/s 307GB/s 358GB/s >500GB/s 8GB/16GB/ Density 8Gb/16Gb/2 4Gb/8Gb 8Gb/16Gb 4GB/8GB 8GB/16GB 24GB (per package) 4Gb/32Gb (TBD) 8

9

HBM Architecture HBM2 core ore die suppor orts ts 4 ps pseudo o channel nels s or 2 c chann nnels els Each ch channe nel l consis sists ts of 2 P Pseudo o Channe nels ls. . Only y BL4 i is suppor orte ted Items Target CH-A CH-B CH-A CH-B B0 B1 B0 B1 # of Stack 4/8(Core) + 1(Base) B2 B3 B2 B3 Ch./Slice 2 B4 B5 B4 B5 Total Ch. for KGSD 8/16 B6 B7 B6 B7 (8ch based operation) IO/Ch. 128 ADD ADD 64 I/O 64 I/O 64 I/O 64 I/O CMD CMD Total IO/KGSD 1024(=128 x 8) Address/CMD Dual CMD Data Rate DDR 10

Next-Gen. System Architecture Leveraging HBM HBM and 2.5D SiP integration unlock new system architecture B/W & B/W Capacity HPC & Server + + + (B/W & Capacity) Bandwidth Bandwidth Solution Capacity Solution Solution HBM Network & Graphics B/W (B/W) Bandwidth Solution B/W & Cost B/W Client-DT & NB + + (B/W & Cost) Post-DDR4 Bandwidth Post-DDR4 Cost Solution Solution

HBM Test Flow HBM Test Flow General DRAM Test Flow

Quality and Reliability Features HBM Features enable high quality and reliability at post 2.5D assembly TSV micro bump DRAM DRAM DRAM Logic die PHY PHY PHY 3 4 5 1 1 2 Cell Repair HBM SoC PKG Substrate 6 Interposer Substrate 6 Proxy Package 2 Error Correcting Code Storage 5 Microbump Repair 4 3 PMBIST BISS (Built In Self Stress) 13

Collaterals Available from HBM vendors Item Remarks Functionality Datasheet (Jedec/Vendor) Verilog (mission mode and DFT) IBIS Hspice Mechanical/Interposer design GDS Bump pad netlist Bump Ballout Thermal Simulation Flotherm Icepak 14

Future of HBM Solution HBM would penetrate various market segments in the short future. Moving to Volume Market HBM3 512GBps GFX 2G/4G/8G/16GB Expansion to various 105C HPC Applications NT ECC/ In DRAM HBM2 ECC W Optimized Client Base die 256GBps GFX 1G/2G/4G/8GB PC HBM1 SVR 105C HP Consum 128GBps ECC NT er C GFX W Auto 1GB Low Cost Ver. Client motiv e 95C SVR Con sumer 15

16

17