comment on bitonic merging more cuda performance tuning
play

Comment on bitonic merging; more CUDA performance tuning CSE 6230: - PowerPoint PPT Presentation

Jan 30, 2024 •990 likes •1.88k views

Comment on bitonic merging; more CUDA performance tuning CSE 6230: HPC Tools & Apps Tu Sep 18, 2012 Tuesday, September 18, 12 Comment on bitonic merging , including ideas & hints for Lab 3 Note: Some figures taken from Grama et al.

  1. Comment on bitonic merging; more CUDA performance tuning CSE 6230: HPC Tools & Apps Tu Sep 18, 2012 Tuesday, September 18, 12

  2. ๏ Comment on bitonic merging , including ideas & hints for Lab 3 Note: Some figures taken from Grama et al. book (2003) http://www-users.cs.umn.edu/~karypis/parbook/ This book is also available online through the GT library – see our course website. Tuesday, September 18, 12

  3. Source: Grama et al. (2003) Tuesday, September 18, 12

  4. Summary so far: bitonicMerge (bitonic sequence) == sorted Q: How do we get a bitonic sequence? Tuesday, September 18, 12

  5. Source: Grama et al. (2003) Tuesday, September 18, 12

  6. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  7. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  8. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  9. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  10. Source: Grama et al. (2003) Tuesday, September 18, 12

  11. Source: Grama et al. (2003) Tuesday, September 18, 12

  12. Bitonic sort parallel complexity (work-depth)? Tuesday, September 18, 12

  13. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 Tuesday, September 18, 12

  14. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  15. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 rounds of communication = O (log n ) 5: 0101 6: 0110 number of pairwise exchanges per round = O ( P ) 7: 0111 words sent per exchange = O ( n / P ) 8: 1000 9: 1001 total words sent = O ( n log n ) 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  16. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  17. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (n/p): No comm log (p): Comm req’d Cyclic Layout (p=4) Tuesday, September 18, 12

  18. These (block or cyclic) examples are binary exchange algorithms. Question: Can we get the “best” of these two schemes? Tuesday, September 18, 12

  19. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 … All-to-all 7: 0111 exchange 8: 1000 … 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  20. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 rounds of communication = 1 5: 0101 6: 0110 number of pairwise exchanges per round = O ( P 2 ) … All-to-all 7: 0111 exchange words sent per exchange = O ( n / P 2 ) 8: 1000 … 9: 1001 total words sent = O ( n ) 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  21. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 … All-to-all 7: 0111 exchange 8: 1000 … 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  22. Cyclic Block 0 1 2 3 0 4 8 12 All-to-all 4 5 6 7 1 5 9 13 exchange ≡ 8 9 10 11 2 6 10 14 Matrix transpose 12 13 14 15 3 7 11 15 Tuesday, September 18, 12

  23. “Binary exchange” algorithm (block or cyclic): rounds of communication = O (log n ) number of pairwise exchanges per round = O ( P ) total number of pairwise exchanges = O ( P log n ) words sent per exchange = O ( n / P ) total words sent = O ( n log n ) “Transpose” algorithm (cyclic → all-to-all → block): rounds of communication = 1 number of pairwise exchanges per round = O ( P 2 ) total number of pairwise exchanges = O ( P 2 ) words sent per exchange = O ( n / P 2 ) total words sent = O ( n ) Tuesday, September 18, 12

  24. ๏ More CUDA tuning: Occupancy and ILP References: http://developer.nvidia.com/cuda/get-started-cuda-cc http://developer.download.nvidia.com/CUDA/training/cuda_webinars_WarpsAndOccupancy.pdf http://www.cs.berkeley.edu/~volkov/volkov10-GTC.pdf http://www.cs.berkeley.edu/~volkov/volkov11-unrolling.pdf Tuesday, September 18, 12

  25. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  26. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  27. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  28. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  29. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  30. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  31. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  32. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  33. /opt/cuda-4.0/cuda/bin/nvcc -arch=sm_20 --ptxas-options=-v -O3 \ -o bitmerge-cuda.o -c bitmerge-cuda.cu ptxas info : Compiling entry function '_Z12bitonicSplitjPfj' for 'sm_20' ptxas info : Function properties for _Z12bitonicSplitjPfj 0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads ptxas info : Used 8 registers , 52 bytes cmem[0] icpc -O3 -g -o bitmerge timer.o bitmerge.o bitmerge-seq.o \ bitmerge-cilk.o bitmerge-cuda.o \ -L/opt/cuda-4.0/cuda/bin/../lib64 \ -Wl,-rpath /opt/cuda-4.0/cuda/bin/../lib64 -lcudart Tuesday, September 18, 12

  34. Occupancy Limiters: Registers Register usage: compile with --ptxas-options=-v Fermi has 32K registers per SM Example 1 Kernel uses 20 registers per thread (+1 implicit) Active threads = 32K/21 = 1560 threads > 1536 thus an occupancy of 1 Example 2 Kernel uses 63 registers per thread (+1 implicit) Active threads = 32K/64 = 512 threads 512/1536 = .3333 occupancy Can control register usage with the nvcc flag: --maxrregcount Occupancy = (Active warps) / (Max active warps) Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  35. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  36. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  37. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  38. Recall: Reduction example Tuesday, September 18, 12

  39. Recall: Reduction example Tuesday, September 18, 12

  40. Recall: Reduction example Tuesday, September 18, 12

  41. Recall: Reduction example b = 256 threads/block ⇒ shmem = 256 * (4 Bytes/ int ) = 1024 Bytes Tuesday, September 18, 12

  42. Occupancy Limiters: Shared Memory Shared memory usage: compile with --ptxas-options=-v Reports shared memory per block Fermi has either 16K or 48K shared memory Example 1, 48K shared memory Kernel uses 32 bytes of shared memory per thread 48K/32 = 1536 threads occupancy=1 Example 2, 16K shared memory Kernel uses 32 bytes of shared memory per thread 16K/32 = 512 threads occupancy=.3333 Don’t use too much shared memory Choose L1/Shared config appropriately. Occupancy = (Active warps) / (Max active warps) Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  43. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

Recommend

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU computing as first-class capability Retain traditional DirectX/OpenGL graphics performance CUDA C Based on industry-standard C A handful of

1.31k views • 62 slides
Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael

Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael

Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael Goesele TU Darmstadt Motivation Memory access is one of the most important performance factors in CUDA applications CUDA Programming Guide

644 views • 36 slides
PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel

PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel

PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel efficiency Com pany Profile Viezu specialize in electronic engine tuning, rolling roads, diagnostic and test equipment Dynocom EU launched 2010;

281 views • 15 slides
Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable

Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable

CUDA Deep Dive Performance CSE 6230 Jee Choi September 10, 2013 Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable of issuing 1 FMA (2 flops) instruction per cycle throughput = 512 1.3

756 views • 58 slides
Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior

Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior

Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior Linux Consultant May 28, 2015 NLUUG Conf, Utrecht, Netherlands Overview Introduction Performance tuning is Science! A little Law and

595 views • 47 slides
Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA

Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA

Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA Performance Optimization 2 Marc Moreno Maza Parallel Reduction 3 University of Western Ontario, London, Ontario (Canada) Parallel Scan 4

572 views • 29 slides
Performance Factors and Tuning Tips for JavaScript RIA

Performance Factors and Tuning Tips for JavaScript RIA

PUG Challenge Americas 2011 Performance Factors and Tuning Tips for JavaScript RIA Frameworks Presented by Dustin Grau 6/5/11-6/8/11 Performance Tuning

701 views • 44 slides
CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement Learning Yan li, Kenneth Chang, Oceane Bel, Ethan L. Miller, Darrel D. E. Long Performance Tuning Tuning systems parameters for high

343 views • 15 slides
Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA

Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA

Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA performance tuning (wrap up) n Equational Programming (intro) 2 n Parallel Programming is hard n End of the free lunch [Sut05] n

284 views • 16 slides
Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is

Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is

Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is Auto-tuning? Making programs better based on empirical feedback from observed runs without programmers having to in the loop. What to Tune

363 views • 13 slides
Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn

Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn

GPU Teaching Kit Accelerated Computing Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn the main venues and developer resources for GPU computing Where CUDA C fits in the big picture 2 3

1.28k views • 12 slides
Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany

Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany

Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany September 20, 2016 Introduction Graph Drawing 2014 Bitonic st-orderings of biconnected planar graphs. A special st -ordering that is similar

729 views • 44 slides
Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that

Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that

GPU Teaching Kit Accelerated Computing Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that memory coalescing is important for effectively utilizing memory bandwidth in CUDA Its origin in DRAM

623 views • 13 slides
GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA

GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA

GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA Department of Computer Science. University of Cape Town April 20 th -24 th , 2015 1. Introduction. [17 slides] 2. Architecture. [21] 1. CUDA hardware

503 views • 38 slides
Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing /

Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing /

Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing / Performance Tuning - FAQ How many servers will I need? Which cloning protocols to offer? How to set those gazillion gerrit.config options?

731 views • 51 slides
CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires

CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires

CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires a performance computer program. good? 1.5 1 0.5 0 -0.5 0 5 101520253035404550 COMBINED DEFINITION OF TUNING 1 0.8 v 0.6 First

753 views • 22 slides
CUDA/Ada An Ada binding to CUDA Reto B urki, Adrian-Ken R uegsegger University of Applied

CUDA/Ada An Ada binding to CUDA Reto B urki, Adrian-Ken R uegsegger University of Applied

Introduction Bindings in Ada CUDA/Ada Design CUDA Binding Conclusion CUDA/Ada An Ada binding to CUDA Reto B urki, Adrian-Ken R uegsegger University of Applied Sciences Rapperswil (HSR), Switzerland 1/16/2012 Master seminar: Program

469 views • 36 slides
E3T E3T E nergy E fficiency E merging T echnologies The Technology Performance Exchange

E3T E3T E nergy E fficiency E merging T echnologies The Technology Performance Exchange

E3T E3T E nergy E fficiency E merging T echnologies The Technology Performance Exchange Emerging Technologies Showcase August 27, 2014 Welcome. Todays webinar is being recorded and will be posted at: www.E3Tnw.org

429 views • 20 slides
Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2012/13 Lecture VI:

Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2012/13 Lecture VI:

Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2012/13 Lecture VI: Performance Tuning and Benchmarking in Databases Performance Tuning Performance tuning involves adjusting various parameters and design choices

635 views • 40 slides
Squeezing GPU performance GPGPU 2015: High Performance Computing with CUDA University of Cape Town

Squeezing GPU performance GPGPU 2015: High Performance Computing with CUDA University of Cape Town

Squeezing GPU performance GPGPU 2015: High Performance Computing with CUDA University of Cape Town (South Africa), April, 20 th -24 th , 2015 Manuel Ujaldn Associate Professor @ Univ. of Malaga (Spain) Conjoint Senior Lecturer @ Univ. of

352 views • 14 slides
Artificial Intelligence Database Performance Tuning Roel Van de Paar Percona Agenda GA:

Artificial Intelligence Database Performance Tuning Roel Van de Paar Percona Agenda GA:

Artificial Intelligence Database Performance Tuning Roel Van de Paar Percona Agenda GA: How it works, terminology, variables, example Database Tuning & Surrounding thoughts gaai POC Results 2 Define: GA A Genetic

791 views • 37 slides
IMPROVING $PORT PERFORMANCE ON $ARCH PLATFORM-BASED PERFORMANCE TUNING OF WEBKIT (PORT=QT

IMPROVING $PORT PERFORMANCE ON $ARCH PLATFORM-BASED PERFORMANCE TUNING OF WEBKIT (PORT=QT

IMPROVING $PORT PERFORMANCE ON $ARCH PLATFORM-BASED PERFORMANCE TUNING OF WEBKIT (PORT=QT ARCH=MIPS74KF) Adrin Prez de Castro Embedded Linux Conference April 29 May 1, 2014 WHOAMI aperez@igalia.com +AdrianPerezDeCastro @aperezdc

399 views • 22 slides
Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative

Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative

Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative Corrections (Hadron Decays) Hadronisation Parton Shower Matching & Merging Hard process Peter Skands (Monash University) Feynrules/Madgraph

698 views • 27 slides
Lecture 2.4 Introduction to CUDA C Introduction to the CUDA Toolkit Objective To become

Lecture 2.4 Introduction to CUDA C Introduction to the CUDA Toolkit Objective To become

GPU Teaching Kit Accelerated Computing Lecture 2.4 Introduction to CUDA C Introduction to the CUDA Toolkit Objective To become familiar with some valuable tools and resources from the CUDA Toolkit Compiler flags Debuggers

887 views • 34 slides

More recommend