programming model centric debugging for openmp
play

Programming-Model Centric Debugging for OpenMP Kevin Pouget - PowerPoint PPT Presentation

Oct 31, 2022 •111 likes •1.28k views

Programming-Model Centric Debugging for OpenMP Kevin Pouget Jean-Fran cois M ehaut, Miguel Santana Universit e Grenoble Alpes / LIG, STMicroelectronics, France Nano2017-DEMA project Dema Workshop, Grenoble December 12 th , 2016 Kevin

  1. Information Capture Strategies Information Capture Strategies Breakpoints Preloaded and Debug Library Information Capturable Info. High Limited to API Execution Overhead Significant Limited Cooperation between None Low Debugger and Env. Portability Low Very Good Kevin Pouget Programming-Model Centric Debugging Dema workshop 8 / 39

  2. Information Capture Strategies Information Capture Strategies Breakpoints Specialized Preloaded and Debug Debug Library Information Module Capturable Info. High Limited to API Full Execution Overhead Significant Limited Limited Cooperation between None Low Strong Debugger and Env. Vendor Portability Low Very Good Specific Kevin Pouget Programming-Model Centric Debugging Dema workshop 8 / 39

  3. Before DEMA Before DEMA Model-Centric Debugging Before DEMA components (STHORM NPM) dataflow (STHORM PEDF) kernel-based programming (GPU/STHORM OpenCL) Kevin Pouget Programming-Model Centric Debugging Dema workshop 9 / 39

  4. Before DEMA Before DEMA Model-Centric Debugging Before DEMA components (STHORM NPM) dataflow (STHORM PEDF) kernel-based programming (GPU/STHORM OpenCL) Kevin Pouget Programming-Model Centric Debugging Dema workshop 9 / 39

  5. Before DEMA Before DEMA Dataflow Debugging for ST/CEA MPSoC STHROM logo by bullboykennels Illustration 1: understanding a deadlock situation Kevin Pouget Programming-Model Centric Debugging Dema workshop 10 / 39

  6. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (gdb) info threads Id Target Id Frame 1 Thread 0xf7e77b 0xf7ffd430 in __kernel_vsyscall () * 2 Thread 0xf7e797 operator= (val=..., this=0xa0a1330) Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  7. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (gdb) info threads Id Target Id Frame 1 Thread 0xf7e77b 0xf7ffd430 in __kernel_vsyscall () * 2 Thread 0xf7e797 operator= (val=..., this=0xa0a1330) (mcgdb) info graph pred_controller ipred ipf hwcfg pipe Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  8. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (gdb) info threads Id Target Id Frame 1 Thread 0xf7e77b 0xf7ffd430 in __kernel_vsyscall () * 2 Thread 0xf7e797 operator= (val=..., this=0xa0a1330) (mcgdb) info graph +state pred_controller ipred ipf hwcfg pipe z Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  9. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (mcgdb) info graph +state pred_controller ipred ipf hwcfg pipe z (mcgdb) info actors +state #0 Controller ‘pred_controller’: Blocked, waiting for step completion #1/2/3 Actors ‘pipe/ipref/ipf’: Blocked, reading from #4 ‘hwcfg’ #4 Actor ‘hwcfg’: Asleep, Step completed Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  10. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (mcgdb) info graph +state pred_controller ipred ipf hwcfg pipe z (gdb) thread apply all where Thread 1 (Thread 0xf7e77b): #0 0xf7ffd430 in __kernel_vsyscall () #1 0xf7fcd18c in pthread_cond_wait@ () #2 0x0809748f in wait_for_step_completion(struct... *) #3 0x0809596e in pred_controller_work_function() #4 0x08095cbc in entry(int, char**) () Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  11. Dataflow Debugging: Deadlock Detection Dataflow Debugging: Deadlock Detection (mcgdb) info graph +state pred_controller ipred X ipf hwcfg pipe z (gdb) thread apply all where Thread 2 (Thread 0xf7e797): #0 operator= (val=..., this=0xa0a1330) #1 pipeRead (data=0) at pipeFilter.c:154 ւ 154 Smb = pedf.io.hwcfgSmb[count]; #2 0x0804da63 in PipeFilter_work_function () at pipe.c:361 #3 0x080a4132 in PedfBaseFilter::controller (this=0xa0d18) Kevin Pouget Programming-Model Centric Debugging Dema workshop 11 / 39

  12. Before DEMA Before DEMA OpenCL debugging BigDFT Density functional theory solver OpenCL (and Cuda) High performance computing Running on Sthorm , but primarily used with GPU Hybrid CPU/GPU Host-side debugging only MPI + OpenCL (C/Fortran) Illustration 2: Why execution visualization is needed Kevin Pouget Programming-Model Centric Debugging Dema workshop 12 / 39

  13. Before DEMA: How execution visualization can help Before DEMA: How execution visualization can help Let’s consider an example ... C code reductionKernel ( int n, double * in, double * out) { ... } checkStatus( int * ptr, char * msg) { if(ptr == 0) exit(-1); } { void main() double * in = malloc(...) ; checkStatus(in, "in failed" ); double * out = malloc(...); checkStatus(out, "out failed" ); initialize(in); reductionKernel(N, in, out); // free ... } Kevin Pouget Programming-Model Centric Debugging Dema workshop 13 / 39

  14. Before DEMA: How execution visualization can help Before DEMA: How execution visualization can help OpenCL equivalent: /* Instantiate the runtime. */ command_queue = clCreateCommandQueue((*context)->context, aDevices[0], 0, &ciErrNum); kerns->reduction_kernel_d=clCreateKernel(reductionProgram, "reductionKernel_d",&ciErrNum); oclErrorCheck(ciErrNum,"Failed to create kernel!"); /* Allocate the buffers on the GPU. */ *buff_ptr = clCreateBuffer((*context)->context, CL_MEM_READ_ONLY, *size, NULL, &ciErrNum); oclErrorCheck(ciErrNum,"Failed to create read buffer!"); /* Push the initial values to the GPU memory. */ cl_int ciErrNum = clEnqueueWriteBuffer((*command_queue)->command_queue, *buffer, CL_TRUE, 0, *size, p... oclErrorCheck(ciErrNum,"Failed to enqueue write buffer!"); /* Set the kernel parameters. */ clSetKernelArg(kernel, i++,sizeof(*ndat), (void*)ndat); clSetKernelArg(kernel, i++,sizeof(*in), (void*... clSetKernelArg(kernel, i++,sizeof(*out), (void*)out); clSetKernelArg(kernel, i++,sizeof(cl_dbl)*blk_... /* Trigger the kernel execution. */ ciErrNum = clEnqueueNDRangeKernel(command_queue->command_queue, kernel, 1, NULL, globalWorkSz, localWo... oclErrorCheck(errNum,"Failed to enqueue reduction kernel!"); /* Get the result back. */ cl_int ciErrNum = clEnqueueReadBuffer((*command_queue)->command_queue, *input, CL_TRUE, 0, sizeof(cl_d... oclErrorCheck(ciErrNum,"Failed to enqueue read buffer!"); /* Then release the memory ... */ Kevin Pouget Programming-Model Centric Debugging Dema workshop 14 / 39

  15. Programming Model Centric Debugging: (before Dema) Dataflo Programming Model Centric Debugging: (before Dema) Dataflo (mcgdb) print flow (an Eclipse visualization engine) Update on user request / automatically on exec. stops, step-by-step, ... Kevin Pouget Programming-Model Centric Debugging Dema workshop 15 / 39

  16. Programming Model Centric Debugging: (before Dema) Dataflo Programming Model Centric Debugging: (before Dema) Dataflo (mcgdb) print flow (an Eclipse visualization engine) Set the kernel arguments. ◮ 2 scalars ◮ 2 GPU buffers clSetKernelArg(kernel, i++, sizeof(*ndat),(void*)ndat); clSetKernelArg(kernel, i++, sizeof(*in), (void*)in); clSetKernelArg(kernel, i++, sizeof(*out), (void*)out); clSetKernelArg(kernel, i++, sizeof(*sz), (void*)sz); Kevin Pouget Programming-Model Centric Debugging Dema workshop 15 / 39

  17. Programming Model Centric Debugging: (before Dema) Dataflo Programming Model Centric Debugging: (before Dema) Dataflo (mcgdb) print flow (an Eclipse visualization engine) Set the kernel arguments. ◮ 2 scalars ◮ 2 GPU buffers Trigger the kernel execution ◮ 2 buffers involved ciErrNum = clEnqueueNDRangeKernel(command_queue->command_q, kernel, 1, NULL, globalWorkSz, localWorkSize, 0, NULL, NULL); Kevin Pouget Programming-Model Centric Debugging Dema workshop 15 / 39

  18. Programming Model Centric Debugging: (before Dema) Dataflo Programming Model Centric Debugging: (before Dema) Dataflo (mcgdb) print flow (an Eclipse visualization engine) Set the kernel arguments. ◮ 2 scalars ◮ 2 GPU buffers Trigger the kernel execution ◮ 2 buffers involved Retrieve the result ◮ buffer content is saved cl_int ciErrNum = clEnqueueReadBuffer( (*command_queue)->command_queue, *input, CL_TRUE, 0, sizeof(cl_double), out, 0, NULL, NULL); Kevin Pouget Programming-Model Centric Debugging Dema workshop 15 / 39

  19. Agenda Agenda 1 Research Context 2 Programming Model Centric Debugging 3 Dema Year 1: Model-Centric Debugging for OpenMP 4 Dema Year 2: Interactive Performance Profiling and Debugging Kevin Pouget Programming-Model Centric Debugging Dema workshop 15 / 39

  20. Nano2017/Dema project Nano2017/Dema project Debugging Embedded and Multicore Applications ARM Juno OpenMP Parallel Programming Fork/join multithreading Tasks with dependencies GNU Gomp, Intel OpenMP, ... mcGDB debugger Python extension of GDB asymmetric arch. Support for dataflow, components, ... ARM big. litle + Mali GPU Developed in partnership with ST Kevin Pouget Programming-Model Centric Debugging Dema workshop 16 / 39

  21. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { ① // beginning of main function 3 omp step #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical { // execute critical } Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  22. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier ①②③④ // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical { // execute critical } Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  23. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier ②③④ // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next ① // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical { // execute critical } Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  24. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next } ①②③④ //implicit barrier 8 omp critical next #pragma omp critical { // execute critical } Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  25. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical ①③④ { ② // execute critical } Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  26. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical ③④ { ① // execute critical } ② Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  27. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical ④ { ③ // execute critical } ①② Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  28. OpenMP: OpenMP Execution Control OpenMP: OpenMP Execution Control control the execution of the entities 1 start 2 omp start int main() { 3 omp step // beginning of main function #pragma omp parallel { 4 omp next barrier // beginning of parallel region 5 omp critical next #pragma omp single { 6 omp critical next // execute single 7 omp critical next }//implicit barrier 8 omp critical next #pragma omp critical { ④ // execute critical } ①②③ Kevin Pouget Programming-Model Centric Debugging Dema workshop 17 / 39

  29. OpenMP: structural representation OpenMP: structural representation ... provide a structural representation ... provide details about entity state 1 fork-join = ⇒ OpenMP sequence diagrams 2 task-based = ⇒ mcGDB+Temanejo cooperation Kevin Pouget Programming-Model Centric Debugging Dema workshop 18 / 39

  30. OpenMP: structural representation OpenMP: structural representation ... provide a structural representation ... provide details about entity state 1 fork-join = ⇒ OpenMP sequence diagrams 2 task-based = ⇒ mcGDB+Temanejo cooperation Kevin Pouget Programming-Model Centric Debugging Dema workshop 18 / 39

  31. OpenMP: structural representation OpenMP: structural representation ... provide a structural representation ... provide details about entity state 1 fork-join = ⇒ OpenMP sequence diagrams 2 task-based = ⇒ mcGDB+Temanejo cooperation Kevin Pouget Programming-Model Centric Debugging Dema workshop 18 / 39

  32. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  33. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  34. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  35. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  36. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  37. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  38. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  39. OpenMP: OpenMP Sequence Diagram OpenMP: OpenMP Sequence Diagram 1 start 2 omp start 3 omp step 4 omp next barrier 5 thread 2 6 omp critical next 7 omp critical next 8 omp critical next Kevin Pouget Programming-Model Centric Debugging Dema workshop 19 / 39

  40. OpenMP: structural representation OpenMP: structural representation ... provide a structural representation ... provide details about entity state 1 fork-join = ⇒ OpenMP sequence diagrams 2 task-based = ⇒ mcGDB+Temanejo cooperation Kevin Pouget Programming-Model Centric Debugging Dema workshop 20 / 39

  41. Task-Graph Visualization Task-Graph Visualization (HLRS Stuttgart) Temanejo ... ✓ is a great visualization tool for task debugging, ✗ and does not support source-level debugging. Kevin Pouget Programming-Model Centric Debugging Dema workshop 21 / 39

  42. Task-Graph Visualization Task-Graph Visualization (HLRS Stuttgart) Temanejo ... ✓ is a great visualization tool for task debugging, ✗ and does not support source-level debugging. GDB/mcGDB ... ✗ has no visualization engine, ✓ but provides source debugging at language (gdb) and model level. Kevin Pouget Programming-Model Centric Debugging Dema workshop 21 / 39

  43. Task-Graph Visualization Task-Graph Visualization (HLRS Stuttgart) Temanejo ... ✓ is a great visualization tool for task debugging, ✗ and does not support source-level debugging. GDB/mcGDB ... ✗ has no visualization engine, ✓ but provides source debugging at language (gdb) and model level. So let’s combine them! Kevin Pouget Programming-Model Centric Debugging Dema workshop 21 / 39

  44. Task-Graph Visualization Task-Graph Visualization mcGDB – Temanejo cooperation: Temanejo task graph visualization simple model-level execution control. sequence diagram visualization. mcGDB task graph and exec. events capture, advanced model-level exec. control. GDB language and assembly level execution control, memory inspection. Kevin Pouget Programming-Model Centric Debugging Dema workshop 22 / 39

  45. Task-Graph Visualization Task-Graph Visualization mcGDB – Temanejo cooperation: Temanejo task graph visualization simple model-level execution control. sequence diagram visualization. mcGDB task graph and exec. events capture, advanced model-level exec. control. GDB language and assembly level execution control, memory inspection. Kevin Pouget Programming-Model Centric Debugging Dema workshop 22 / 39

  46. Task-Graph Visualization Task-Graph Visualization mcGDB – Temanejo cooperation: Temanejo task graph visualization simple model-level execution control. sequence diagram visualization. mcGDB task graph and exec. events capture, advanced model-level exec. control. GDB language and assembly level execution control, memory inspection. Kevin Pouget Programming-Model Centric Debugging Dema workshop 22 / 39

  47. Task-Graph Visualization Task-Graph Visualization mcGDB – Temanejo cooperation: Temanejo task graph visualization simple model-level execution control. sequence diagram visualization. mcGDB task graph and exec. events capture, advanced model-level exec. control. GDB language and assembly level execution control, memory inspection. Kevin Pouget Programming-Model Centric Debugging Dema workshop 22 / 39

  48. mcGDB + Temanejo mcGDB + Temanejo Node color ◮ sources files Kevin Pouget Programming-Model Centric Debugging Dema workshop 23 / 39

  49. mcGDB + Temanejo mcGDB + Temanejo Node color ◮ sources files Links color ◮ dependencies Kevin Pouget Programming-Model Centric Debugging Dema workshop 23 / 39

  50. mcGDB + Temanejo mcGDB + Temanejo Node color ◮ sources files ◮ debug state Links color ◮ dependencies Task 3 blocked blue finished purple blocked Kevin Pouget Programming-Model Centric Debugging Dema workshop 23 / 39

  51. mcGDB + Temanejo mcGDB + Temanejo Node color ◮ sources files ◮ debug state ◮ executed by Links color ◮ dependencies Task 3 blocked blue finished purple blocked Exec. finished Kevin Pouget Programming-Model Centric Debugging Dema workshop 23 / 39

  52. Agenda Agenda 1 Research Context 2 Programming Model Centric Debugging 3 Dema Year 1: Model-Centric Debugging for OpenMP 4 Dema Year 2: Interactive Performance Profiling and Debugging Kevin Pouget Programming-Model Centric Debugging Dema workshop 23 / 39

  53. Interactive Performance Debugging Interactive Performance Debugging Performance Debugging Methodology 1 Benchmark the code 2 Locate the time-expensive areas 3 Estimate their (in)efficiency: how is the time spent? can it be reduced? 4 Optimize the code accordingly 5 Go back to step 1. Kevin Pouget Programming-Model Centric Debugging Dema workshop 24 / 39

  54. Interactive Performance Debugging Interactive Performance Debugging Performance Debugging Methodology 1 Benchmark the code 2 Locate the time-expensive areas 3 Estimate their (in)efficiency: how is the time spent? can it be reduced? 4 Optimize the code accordingly 5 Go back to step 1. Profiling tools gprof perf stat, Papi trace-based analyzers (aftermath) Kevin Pouget Programming-Model Centric Debugging Dema workshop 24 / 39

  55. Interactive Performance Debugging Interactive Performance Debugging Performance Debugging Methodology 1 Benchmark the code 2 Locate the time-expensive areas 3 Estimate their (in)efficiency: how is the time spent? can it be reduced? 4 Optimize the code accordingly 5 Go back to step 1. Profiling tools : not really interactive gprof, perf stat, aftermath, . . . ◮ profile all or nothing ( perf can attach/detach) Papi ◮ customizable, but from within the code Kevin Pouget Programming-Model Centric Debugging Dema workshop 24 / 39

  56. Interactive Performance Debugging Interactive Performance Debugging Performance Debugging Methodology 1 Benchmark the code 2 Locate the time-expensive areas 3 Estimate their (in)efficiency: how is the time spent? can it be reduced? 4 Optimize the code accordingly 5 Go back to step 1. Source-level debuggers (gdb/mcgdb) have interactivity! execute the code step-by-step, study the state, alter it to test hypotheses on-the-fly . . . but nothing for performance debugging! Kevin Pouget Programming-Model Centric Debugging Dema workshop 24 / 39

  57. Interactive Performance Debugging Interactive Performance Debugging Performance Debugging Methodology 1 Benchmark the code 2 Locate the time-expensive areas 3 Estimate their (in)efficiency: how is the time spent? can it be reduced? 4 Optimize the code accordingly 5 Go back to step 1. Source-level debuggers (gdb/mcgdb) have interactivity! execute the code step-by-step, study the state, alter it to test hypotheses on-the-fly . . . but nothing for performance debugging! Kevin Pouget Programming-Model Centric Debugging Dema workshop 24 / 39

  58. Interactive Performance Debugging Interactive Performance Debugging This is an on-going work 1 Interactive profiling ◮ What to measure? ◮ Where to profile? 2 OpenMP profiling 3 MG benchmark performance bug and mcGDB ◮ loop profiling ◮ intermediate profiling charts ◮ execution control and profiling ◮ performance optimization and results Kevin Pouget Programming-Model Centric Debugging Dema workshop 25 / 39

  59. Interactive Performance Debugging Interactive Performance Debugging What to measure? /proc/$PID/... values (mem usage, context switches, ...) gprof counters function/address execution counter (breakpoints involved) perf stat counters Kevin Pouget Programming-Model Centric Debugging Dema workshop 26 / 39

  60. Interactive Performance Debugging Interactive Performance Debugging What to measure? /proc/$PID/... values (mem usage, context switches, ...) gprof counters function/address execution counter (breakpoints involved) perf stat counters ◮ cache-misses, cache-references ◮ instructions ◮ cpu-clock, task-clock ◮ node-load-misses, node-store-misses Kevin Pouget Programming-Model Centric Debugging Dema workshop 26 / 39

  61. Interactive Performance Debugging Interactive Performance Debugging Where to profile? During the execution: ◮ a function execution ◮ a region: from line ... to line ... (breakpoints involved) ◮ start and stop on user request Outside of the normal execution (base on gdb+gcc dynamic compilation) ◮ code compiled on-demand and inserted in the process address-space ◮ custom function calls, ◮ repeat n times ◮ test different compilation flags, ... Kevin Pouget Programming-Model Centric Debugging Dema workshop 27 / 39

  62. Interactive Performance Debugging Interactive Performance Debugging Where to profile? During the execution: ◮ a function execution ◮ a region: from line ... to line ... (breakpoints involved) ◮ start and stop on user request ◮ what about OpenMP? Outside of the normal execution (base on gdb+gcc dynamic compilation) ◮ code compiled on-demand and inserted in the process address-space ◮ custom function calls, ◮ repeat n times ◮ test different compilation flags, ... Kevin Pouget Programming-Model Centric Debugging Dema workshop 27 / 39

  63. OpenMP Profiling OpenMP Profiling Profiling the whole execution: Aftermath 1 Dema SP2 Fine-grain Interactive Profiling: mcGDB profiler use mcGDB for a fine-grained profiling of loops and tasks use mcGDB to trigger the generation of on-going Aftermath traces 1 http://www.openstream.info/aftermath Kevin Pouget Programming-Model Centric Debugging Dema workshop 28 / 39

  64. Before going further: mg.C performance bug Before going further: mg.C performance bug performance bug on idchire (numa arch, 24 nodes, 192 cores) #pragma omp for /* mc.c function resid */ for (i3 = 1; i3 < n3-1; i3++) { for (i2 = 1; i2 < n2-1; i2++) { for (i1 = 0; i1 < n1; i1++) { u1[i1] = u[i3][i2-1][i1] + u[i3][i2+1][i1] + u[i3-1][i2][i1] + u[i3+1][i2][i1]; u2[i1] = u[i3-1][i2-1][i1] + u[i3-1][i2+1][i1] + u[i3+1][i2-1][i1] + u[i3+1][i2+1][i1]; } for (i1 = 1; i1 < n1-1; i1++) { r[i3][i2][i1] = v[i3][i2][i1] - a[0] * u[i3][i2][i1] - a[2] * (u2[i1] + u1[i1-1] + u1[i1+1]) - a[3] * (u2[i1-1] + u2[i1+1]); } } } Kevin Pouget Programming-Model Centric Debugging Dema workshop 29 / 39

  65. Before going further: mg.C performance bug Before going further: mg.C performance bug performance bug on idchire (numa arch, 24 nodes, 192 cores) CPU 8 CPU 12 CPU 16 CPU 20 CPU 24 CPU 28 CPU 32 CPU 36 CPU 40 CPU 44 CPU 48 CPU 52 CPU 56 CPU 60 CPU 64 CPU 68 CPU 72 CPU 76 CPU 80 CPU 84 CPU 88 CPU 92 CPU 96 CPU 100 CPU 104 CPU 108 CPU 112 CPU 116 CPU 120 CPU 124 CPU 128 CPU 132 CPU 136 CPU 140 CPU 144 CPU 148 3.000e+12 3.100e+12 3.200e+12 3.300e+12 3.400e+12 3.500e+12 3.600e+12 3.700e+12 3.800e+12 3.900e+12 4.000e+12 4.100e+12 4.200e+12 4.300e+12 4.400e+12 4.500e+12 4.600e+12 Y axis is time Kevin Pouget Programming-Model Centric Debugging Dema workshop 29 / 39

  66. Before going further: mg.C performance bug Before going further: mg.C performance bug performance bug on idchire (numa arch, 24 nodes, 192 cores) Use mcGDB knowledge for a fine-grained profiling of loops and tasks attach/detach perf stat when a loop iteration starts/stops ◮ force sequentiality for accuracy / feasibility | #23 loop profile | cache-references: 20,322 | cycles: 41,501,975 | node-stores: 2,828 | node-misses: 2,445 | instructions: 78,896,610 | omp_loop_len: 1 | omp_loop_start: 441 | numa node/code: 19/156 | Kevin Pouget Programming-Model Centric Debugging Dema workshop 29 / 39

  67. mg.C performance bug: intermediate chart view mg.C performance bug: intermediate chart view Instructions count sorted by numa core id ; columns are loop iterations Two phases (2 then 1 chunk), but the instruction count is constant. Kevin Pouget Programming-Model Centric Debugging Dema workshop 30 / 39

Recommend

Programming-Model Centric Debugging for OpenMP Nano2017/Dema Project Meeting June 24 th , 2015

Programming-Model Centric Debugging for OpenMP Nano2017/Dema Project Meeting June 24 th , 2015

Programming-Model Centric Debugging for OpenMP Nano2017/Dema Project Meeting June 24 th , 2015 Universit Paris Jussieu June 24 th , 2015 Kevin Pouget OMP Programming-Model Centric Debugging 1 / 20 Introduction OpenMP and GDB Hard to

886 views • 51 slides
Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget, UJF-LIG,

Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget, UJF-LIG,

s t m i c r o e l e c t r o n i c s , u n i v e r s i t y o f g r e n o b l e / l i g l a b o r a t o r y STMicroelectronics LIG University of Grenoble Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget, UJF-LIG,

843 views • 69 slides
Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget Under the

Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget Under the

s t m i c r o e l e c t r o n i c s , u n i v e r s i t y o f g r e n o b l e / l i g l a b o r a t o r y STMicroelectronics LIG University of Grenoble Programming-Model Centric Debugging for Multicore Embedded Systems Kevin Pouget Under the

1.69k views • 129 slides
Threaded Programming Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP

Threaded Programming Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP

Threaded Programming Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP History of OpenMP Compiling and running OpenMP programs 2 What is OpenMP? OpenMP is an API designed for programming shared memory parallel

396 views • 14 slides
Introduction to OpenMP Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP

Introduction to OpenMP Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP

Introduction to OpenMP Lecture 2: OpenMP fundamentals Overview Basic Concepts in OpenMP History of OpenMP Compiling and running OpenMP programs 2 What is OpenMP? OpenMP is an API designed for programming shared memory parallel

547 views • 17 slides
Shared Memory ... Programming Model Hardware Languages ( OpenMP , Cilk, pthreads, ...)

Shared Memory ... Programming Model Hardware Languages ( OpenMP , Cilk, pthreads, ...)

Shared Memory ... Programming Model Hardware Languages ( OpenMP , Cilk, pthreads, ...) Memory Model ... Homework ... Parallel Programming Models Programming model gives an abstract view of the machine describing Control

868 views • 75 slides
COMP 633 - Parallel Computing Lecture 7 September 3, 2020 SMM (2) OpenMP Programming Model

COMP 633 - Parallel Computing Lecture 7 September 3, 2020 SMM (2) OpenMP Programming Model

COMP 633 - Parallel Computing Lecture 7 September 3, 2020 SMM (2) OpenMP Programming Model Reading for next time OpenMP tutorial: look through secns 3-5 plus secn 6 up to exercise 1 COMP 633 - Prins Shared Memory Multiprocessing

428 views • 27 slides
CS 4230: Parallel Programming Lecture 4: OpenMP Open Multi-Processing January 23, 2017

CS 4230: Parallel Programming Lecture 4: OpenMP Open Multi-Processing January 23, 2017

CS 4230: Parallel Programming Lecture 4: OpenMP Open Multi-Processing January 23, 2017 01/23/2017 CS4230 1 Outline OpenMP another approach for thread parallel programming Fork-Join execution model OpenMP constructs syntax

175 views • 17 slides
Parallel Programming with OpenMP CS240A, T. Yang 1 A Programmer s View of OpenMP What

Parallel Programming with OpenMP CS240A, T. Yang 1 A Programmer s View of OpenMP What

Parallel Programming with OpenMP CS240A, T. Yang 1 A Programmer s View of OpenMP What is OpenMP? Open specification for Multi-Processing Standard API for defining multi-threaded shared-memory programs openmp.org

612 views • 35 slides
SHARED MEMORY PROGRAMMING WITH OPENMP Lecture 9: OpenMP Performance 2 A common scenario.....

SHARED MEMORY PROGRAMMING WITH OPENMP Lecture 9: OpenMP Performance 2 A common scenario.....

SHARED MEMORY PROGRAMMING WITH OPENMP Lecture 9: OpenMP Performance 2 A common scenario..... So I wrote my OpenMP program, and I checked it gave the right answers, so I ran some timing tests, and the speedup was, well, a bit disappointing

596 views • 33 slides
OpenMP Instructor PanteA Zardoshti Department of Computer Engineering Sharif University of

OpenMP Instructor PanteA Zardoshti Department of Computer Engineering Sharif University of

OpenMP Instructor PanteA Zardoshti Department of Computer Engineering Sharif University of Technology e-mail: azad@sharif.edu What Is OpenMP? OpenMP in an application programming interface that provides a parallel programming model for

1.23k views • 49 slides
Programming Distributed Memory Sytems Using OpenMP Rudolf Eigenmann, Ayon Basumallik, Seung-Jai

Programming Distributed Memory Sytems Using OpenMP Rudolf Eigenmann, Ayon Basumallik, Seung-Jai

Programming Distributed Memory Sytems Using OpenMP Rudolf Eigenmann, Ayon Basumallik, Seung-Jai Min, School of Electrical and Computer Engineering, Purdue University, http://www.ece.purdue.edu/ParaMount Is OpenMP a useful programming model

798 views • 30 slides
THREADED PROGRAMMING OpenMP Performance 2 A common scenario..... So I wrote my OpenMP

THREADED PROGRAMMING OpenMP Performance 2 A common scenario..... So I wrote my OpenMP

THREADED PROGRAMMING OpenMP Performance 2 A common scenario..... So I wrote my OpenMP program, and I checked it gave the right answers, so I ran some timing tests, and the speedup was, well, a bit disappointing really. Now what?. Most

705 views • 32 slides
OpenMP: a shared-memory parallel programming model Eduard Ayguad Computer Sciences Department

OpenMP: a shared-memory parallel programming model Eduard Ayguad Computer Sciences Department

OpenMP: a shared-memory parallel programming model Eduard Ayguad Computer Sciences Department Associate Director (BSC) Professor of the Computer Architecture Department (UPC) OpenMP for shared memory OpenMP for shared memory First

352 views • 17 slides
Shared Memory Programming Introduction to OpenMP Overview Shared memory systems Basic

Shared Memory Programming Introduction to OpenMP Overview Shared memory systems Basic

Shared Memory Programming Introduction to OpenMP Overview Shared memory systems Basic Concepts in OpenMP Brief history of OpenMP Compiling and running OpenMP programs Shared memory systems OpenMP is designed for programming

473 views • 26 slides
Parallel Programming using OpenMP Qin Liu The Chinese University of Hong Kong 1 Overview Why

Parallel Programming using OpenMP Qin Liu The Chinese University of Hong Kong 1 Overview Why

Parallel Programming using OpenMP Qin Liu The Chinese University of Hong Kong 1 Overview Why Parallel Programming? Overview of OpenMP Core Features of OpenMP More Features and Details... One Advanced Feature 2 Introduction OpenMP is

1.49k views • 89 slides
Introduction to MPI and OpenMP myson @ postech.ac.kr CSE700-PL @ POSTECH Programming Language

Introduction to MPI and OpenMP myson @ postech.ac.kr CSE700-PL @ POSTECH Programming Language

Introduction to MPI and OpenMP myson @ postech.ac.kr CSE700-PL @ POSTECH Programming Language Laboratory p.1/18 Outline MPI and OpenMP Definition Characteristics Flow models Examples Compiling and Execution Resources Programming

564 views • 18 slides
Optimization on one core OpenMP, MPI and hybrid programming An introduction to the de-facto

Optimization on one core OpenMP, MPI and hybrid programming An introduction to the de-facto

Optimization on one core OpenMP, MPI and hybrid programming An introduction to the de-facto industrial standards Vincent Keller Nicolas Richart July 10, 2019 Table of Contents Parallelization with OpenMP Lecture based on specifications ver

566 views • 42 slides
Programming with OpenMP CS240A, T. Yang, 2013 Modified from Demmel/Yelicks and Mary Halls

Programming with OpenMP CS240A, T. Yang, 2013 Modified from Demmel/Yelicks and Mary Halls

Parallel Programming with OpenMP CS240A, T. Yang, 2013 Modified from Demmel/Yelicks and Mary Halls Slides 1 Introduction to OpenMP What is OpenMP? Open specification for Multi-Processing Standard API for defining

280 views • 25 slides
Recommended Reading A Brief Introduction to OpenMP OpenMP FAQ http://openmp.org/openmp-faq.html

Recommended Reading A Brief Introduction to OpenMP OpenMP FAQ http://openmp.org/openmp-faq.html

Recommended Reading A Brief Introduction to OpenMP OpenMP FAQ http://openmp.org/openmp-faq.html Will Knottenbelt OpenMP on Wikipedia http://en.wikipedia.org/wiki/OpenMP Imperial College London wjk@doc.ic.ac.uk OpenMP Tutorial

253 views • 4 slides
Programming Shared-memory Platforms with OpenMP Xu Liu Topics for Today Introduction to

Programming Shared-memory Platforms with OpenMP Xu Liu Topics for Today Introduction to

Programming Shared-memory Platforms with OpenMP Xu Liu Topics for Today Introduction to OpenMP OpenMP directives concurrency directives parallel regions loops, sections, tasks synchronization directives

452 views • 43 slides
Parallel programming using OpenMP Computer Architecture J. Daniel Garca Snchez (coordinator)

Parallel programming using OpenMP Computer Architecture J. Daniel Garca Snchez (coordinator)

Parallel programming using OpenMP Parallel programming using OpenMP Computer Architecture J. Daniel Garca Snchez (coordinator) David Expsito Singh Francisco Javier Garca Blas ARCOS Group Computer Science and Engineering Department

971 views • 49 slides
IoT Device Programming and Debugging Tom Spink Programming Programming is the act of uploading

IoT Device Programming and Debugging Tom Spink Programming Programming is the act of uploading

IoT Device Programming and Debugging Tom Spink Programming Programming is the act of uploading code and data to an embedded device. In general, it encompasses writing data into EEPROM and usually the configuration of the target device, via

521 views • 49 slides
Parallel programming 03 Walter Boscheri walter.boscheri@unife.it University of Ferrara -

Parallel programming 03 Walter Boscheri walter.boscheri@unife.it University of Ferrara -

Parallel programming 03 Walter Boscheri walter.boscheri@unife.it University of Ferrara - Department of Mathematics and Computer Science A.Y. 2018/2019 - Semester I Outline Introduction to OpenMP 1 OpenMP directives 2 OpenMP synchronization

279 views • 26 slides

More recommend