6/24/2013 WHAT IS THIS TALK ABOUT ? Study of power and energy - PDF document

6/24/2013 WHAT IS THIS TALK ABOUT ? • Study of power and energy profile of different optimization ANALYZING OPTIMIZATION techniques used in heterogeneous applications TECHNIQUES FOR POWER • Evaluation of power/performance of such optimization techniques EFFICIENCY ON HETEROGENEOUS on heterogeneous applications such as FFT PLATFORMS PLATFORMS Yash Ukidave and David Kaeli Department of Electrical and Computer Engineering Northeastern University, Boston, USA AsHES 2013 Boston, MA 20 th May, 2013 1 | AsHES 2013 | May 2013 2 | AsHES 2013 | May 2013 TOPICS MOTIVATION � Increasing use of the CPU-GPU � Applications using OpenCL and power consumption of heterogeneous environment to accelerate data-parallel devices heterogeneous applications � Fast Fourier Transforms (FFT) & evaluation methodology � Thermal Design Power (TDP) of latest generation of GPUs used for � Optimization techniques used for analysis heterogeneous compute � Results for power-performance of FFT implementations � Understanding the effects of software R lt f f f FFT i l t ti design methods contributing to power � Analysis of power-performance of different optimization techniques consumption � Energy profile of different optimization techniques � Power and Energy aspect of different optimization techniques for � Conclusion heterogeneous platforms � Future work 3 | AsHES 2013 | May 2013 4 | AsHES 2013 | May 2013 1

6/24/2013 FAST FOURIER TRANSFORM (FFT) FFT IMPLEMENTATIONS x(0) X(0) • MR-SC FFT : Multi-Radix single � FFT is an algorithm to compute Discrete Fourier FFT Kernel Memory Twiddle Implementation Calls Access factor kernel call, based on the FFT Transforms (DFT) Patterns Computation X(1) x(1) implementation in AMD SDK. -1 � Reduces time complexity to O(nlogn) from O(n 2 ) MR-SC Global & Kernel • MR-MC FFT : Based on Cooley- � FFT classified as Decimation in Time(DIT) or Single Local Compile-Time Tukey algorithm, uses Multiple Memory Decimation in Frequency (DIF) kernel calls and Multiple Radix � DIT : Operates on odd and even components of � DIT : Operates on odd and even components of combinations for compute combinations for compute Stockham FFT signal Kernel • Stockham FFT : Based on the � DIF : Operates on two halves of the signal Global Run-Time Apple FFT Stockham algorithm for FFT. Single Multiple Memory � Butterfly structure is a major component of FFT of a Radix and Single Kernel call MR-MC FFT computation given Radix � Each FFT work item performs computes butterfly on • Apple FFT : A Multiple Kernel call given data points based FFT provided by Apple Inc. using OpenCL 5 | AsHES 2013 | May 2013 6 | AsHES 2013 | May 2013 PLATFORMS FOR EVALUATION OPTIMIZATION TECHNIQUES CLASSIFIED IN SETS � Discrete GPUs � Shared Memory APUs � Set S0 Device Features Intel AMD Device Features Nvidia AMD No modifications Core i7 Fusion A8 GTX 680 Radeon HD ( “ out-of-box”) 3300 APU 7770 Device Generation Ivy Evergreen Device Generation Kepler Southern performance Bridge Islands Compute 16 5 Compute 8 10 Units(CU) ( ) Units(CU) ( ) Processing 4 16 Processing 192 64 � Set S1 � Set S2 � Set S3 Elements(PE)/CU Elements(PE)/CU � Coalesced Global � Data Transformation � Local Memory Memory Accesses Usage TDP(Watts) 70 100 TDP(Watts) 195 80 float � float2 � Loop Unrolling Memory 26 17 Memory 192 72 � Stage overlapping Bandwidth(GB/s) Bandwidth(GB/s) float � float4 for stage-based compute float � float8 Register File 64 64 Register File 256 256 Per CU(KB) Per CU(KB) 7 | AsHES 2013 | May 2013 8 | AsHES 2013 | May 2013 2

6/24/2013 EXECUTION PERFORMANCE OF FFT POWER PERFORMANCE OF NON-OPTIMIZED FFT • Performance is compared to the baseline on each platform � High power consumption of Nvidia GPUs affect their power efficiency over AMD • Evaluation is done for three data set sizes 64K, 1M and 2M data points GPUs � Low compute capability of APUs exhibit decrease in performance over GPUs Execution Performance on GPUs Execution Performance on APUs Power Performance on GPUs Power Performance on APUs 9 | AsHES 2013 | May 2013 10 | AsHES 2013 | May 2013 ANALYSIS OF MEMORY BASED OPTIMIZATIONS ANALYSIS OF MEMORY BASED OPTIMIZATIONS � S1 & S2 optimizations use coalesced memory accesses in FFT kernels � Improved throughput causes increase in � FFTs are modified to perform coalesced memory accesses and loop structures execution performance are unrolled � Coalesced accesses are � Improved throughput on devices leads to increase in performance handled using specialized hardware paths � Average improvement in throughput is 2X on discrete GPUs � Coalesced accesses increase power consumption � Average increase in power is 32% 11 | AsHES 2013 | May 2013 12 | AsHES 2013 | May 2013 3

6/24/2013 ANALYSIS OF MEMORY BASED OPTIMIZATIONS EFFECTS OF LOCAL MEMORY USAGE AND STAGE-OVERLAPPING S2 data transform evaluation on Nvidia GPU � S2 optimizations use data • S3 optimizations use local transformation memory and stage-overlapping � Data transforms in OpenCL • Stages of multi-kernel allow contiguous memory implementation are overlapped access in single kernel call � This increases coalesced � This increases coalesced • MRMC and Apple FFT are • MRMC and Apple FFT are accesses for the kernel modified for stage overlap � Power consumption increases • Kernel call overhead is due to coalesced access avoided and coalesced access are also used � Per-workitem compute increases due to increase in • Large performance gains are input data access observed on GPUs and APUs 13 | AsHES 2013 | May 2013 14 | AsHES 2013 | May 2013 ARCHITECTURAL FACTORS CONTRIBUTING TO POWER ENERGY PROFILE OF OPTIMIZATION TECHNIQUES � Energy relates power � S3 optimizations are evaluated consumption and execution for this analysis performance directly � Memory stalls and ALU � Power consumption can utilization is evaluated for power increase due to increase in consumption performance � Memory unit stalls directly y y � Energy consumption exposes affects power consumption of this trade-off GPUs and APUs � Variation in energy across � Number of in-flight memory optimizations is 11% on average accesses can cause memory- unit stalls � S2 optimizations show 13% � ALU utilization does not directly increase in energy over other optimization sets affect power consumption 15 | AsHES 2013 | May 2013 16 | AsHES 2013 | May 2013 4

6/24/2013 RESULTS SUMMARY RESULTS SUMMARY � Highly Efficient � � Moderately Efficient Less Efficient GPUs APUs More than 40% Less than 10% 10-40% improvement improvement improvement • S1 & S2 improve performance • S1 & S2 are not power efficient Power Performance Power Performance efficiency with cost for power y p Efficiency Efficiency Efficiency Efficiency (Gflops/Watts) (Gflops/Watts) Optimization Techniques Optimization Techniques consumption • Local memory improves compute efficiency and power efficiency GPUs APUs GPUs APUs GPUs APUs • Local memory in S3 improve � � � � � � power-performance Coalesced Memory Access • Stage overlapping increases load � � � � � � on resources and increases power Loop Unrolling • S3 are compute and power efficient consumption � � � � � � Data Transformation � � � � � � Local Memory Usage � � � � � � Stage Overlapping 17 | AsHES 2013 | May 2013 18 | AsHES 2013 | May 2013 CONCLUSION FUTURE WORK � Analyzed different optimization techniques for their power-performance on once � Analyze power consumption on SoC (System-on-Chip) devices with GPUs, such class of applications as TI OMAP4, Samsung Exynos, Qualcomm Snapdragon � Study helps developer identify potential of power-aware kernel development � Power-performance analysis to multi-GPU environments such as clusters � Optimizations related to coalesced memory accesses exhibit increase in power � Study of microarchitectural features responsible for power consumption on GPUs consumption on GPUs and APUs � Extend study to different heterogeneous multi-core devices such as Adapteva Extend study to different heterogeneous multi core devices such as Adapteva � Local Memory utilization is observed as the most power efficient optimization Ephiphany Processor , Tilera TilePro processors technique � Power increment due to performance improvement is captured effectively in the energy profile 19 | AsHES 2013 | May 2013 20 | AsHES 2013 | May 2013 5

6/24/2013 THANK YOU ! QUESTIONS ? COMMENTS ? EXTRA SLIDES EXTRA SLIDES Yash Ukidave yukidave@ece.neu.edu 21 | AsHES 2013 | May 2013 22 | AsHES 2013 | May 2013 TITLE PAGE PICTURE FFT IMPLEMENTATIONS WITH SINGLE KERNEL CALL � MR-SC FFT : FFT implementation based on AMD SDK. Multiple Radix sizes used � Stockham FFT : Stockham algorithm for FFT computation. Single Radix computation FFT Source: http://commons.wikimedia.org/wiki/File:Mona_Lisa_bw_square.jpeg 23 | AsHES 2013 | May 2013 24 | AsHES 2013 | May 2013 6