detecting and fixing memory related performance problems
play

Detecting and Fixing Memory-Related Performance Problems in Managed - PowerPoint PPT Presentation

Oct 16, 2022 •177 likes •1.28k views

Detecting and Fixing Memory-Related Performance Problems in Managed Languages Lu Fang Committee: Prof. Guoqing Xu (Chair), Prof. Alex Nicolau, Prof. Brian Demsky University of California, Irvine May 26, 2017, Irvine, CA, USA Lu Fang (UC

  1. Heap Reference Path Leak is reference by whom? 1 Object leak is referenced by array Object[] array = new Object[10]; 2 Knowing allocation site 1 is not // Allocation site 1, creating the leak. Object leak = new Object(); enough // Object leak is referenced by array array[0] = leak; 3 Key point: array keeps a reference to // Keep using Object leak ... leak , which can be shown by leak ’s // ... Never use leak again. heap reference path // However, leak is referenced by array, // GC cannot reclaim object leak. Lu Fang (UC Irvine) Final Defense May 26, 2017 14 / 51

  2. Mirroring the reference path Original Objects Mirroring Ref. Path Stack stack = new stack; // Allocation site 1, creating the leak. Object obj = new Object(); // stack.elements[0] = leak stack.push(); // Keep using Object leak ... // ... Never use obj again Mirror Objects // However, leak is referenced by stack, // GC cannot reclaim object leak. Lu Fang (UC Irvine) Final Defense May 26, 2017 15 / 51

  3. Experiments Three detectors ◮ Memory leak amplifier ◮ Under-utilized container amplifier ◮ Over-populated container amplifier DaCapo benchmarks with 500MB heap Lu Fang (UC Irvine) Final Defense May 26, 2017 16 / 51

  4. Memory Leak Amplifier 60 VSOs caused by confirmed memory leaks 50 Basic VSOs 40 30 20 VSO is large  The program is likely to have leaks 10 0 antlr bloat eclipse fop luindexlusearch pmd hsqldb jython xalan Programs with confirmed unknown leaks Lu Fang (UC Irvine) Final Defense May 26, 2017 17 / 51

  5. Under-Utilized Container Amplifier 60 VSOs caused by confirmed under-utilized containers 50 Basic VSOs 40 30 20 VSO is large  The program is very likely to have UUCs 10 0 antlr bloat eclipse fop luindexlusearch pmd hsqldb jython xalan Programs with confirmed unknown UUCs Lu Fang (UC Irvine) Final Defense May 26, 2017 18 / 51

  6. Over-Populated Container Amplifier 30 VSOs caused by confirmed over-populated containers 25 Basic VSOs 20 15 10 VSO is large  The program is very likely to have OPCs 5 0 antlr bloat eclipse fop luindexlusearch pmd xalan hsqldb jython Programs with confirmed unknown OPCs Lu Fang (UC Irvine) Final Defense May 26, 2017 19 / 51

  7. Performance Improvements Benchmark Space Reduction Time Reduction xalan-leak 25.4% 14.6% jython-leak 24.3% 7.4% hsqldb-leak 15.6% 3.1% xalan-UUC 5.4% 34.1% jython-UUC 19.1% 1.1% hsqldb-UUC 17.4% 0.7% hsqldb-OPC 14.9% 2.9% Lu Fang (UC Irvine) Final Defense May 26, 2017 20 / 51

  8. The Effectiveness of PerfBlower VSOs indicate the existence of problems ◮ 8 unknown problems are detected ◮ All reports contain useful diagnostic information Low overhead ◮ Space overheads are 1.23–1.25 × ◮ Time overheads are 2.39–2.74 × Lu Fang (UC Irvine) Final Defense May 26, 2017 21 / 51

  9. Fixing Performance Problems Fixing performance problems is hard ◮ Enough information is necessary ◮ Have to understand the logic of the system ◮ The problem exists deeply in the system Memory pressure ◮ A common performance problem in data-paralle systems Lu Fang (UC Irvine) Final Defense May 26, 2017 22 / 51

  10. Lu Fang , Khanh Nguyen, Guoqing Xu, Brian Demsky, Shan Lu Interruptible Tasks: Treating Memory Pressure As Interrupts for Highly Scalable Data-Parallel Programs SOSP’15 Lu Fang (UC Irvine) Final Defense May 26, 2017 23 / 51

  11. Memory Pressure in Data-Parallel Systems Data-parallel system ◮ Input data are divided into independent partitions ◮ Many popular big data systems Lu Fang (UC Irvine) Final Defense May 26, 2017 24 / 51

  12. Memory Pressure in Data-Parallel Systems Data-parallel system ◮ Input data are divided into independent partitions ◮ Many popular big data systems � Memory pressure on single nodes Our study ◮ Search “out of memory” and “data parallel” in StackOverflow ◮ We have collected 126 related problems Lu Fang (UC Irvine) Final Defense May 26, 2017 24 / 51

  13. Memory Pressure in the Real World Memory pressure on individual nodes ◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory Memory consumption Heap size OutOfMemoryError point Long and useless GC Execution time Lu Fang (UC Irvine) Final Defense May 26, 2017 25 / 51

  14. Memory Pressure in the Real World Memory pressure on individual nodes ◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory Memory consumption Heap size OutOfMemoryError point Long and useless GC Execution time CRASH OutOfMemory Error Lu Fang (UC Irvine) Final Defense May 26, 2017 25 / 51

  15. Memory Pressure in the Real World Memory pressure on individual nodes ◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory Memory consumption Heap size OutOfMemoryError point Long and useless GC Execution time CRASH OutOfMemory Error SLOW Huge GC effort Lu Fang (UC Irvine) Final Defense May 26, 2017 25 / 51

  16. Root Cause 1: Hot Keys Key-value pairs Lu Fang (UC Irvine) Final Defense May 26, 2017 26 / 51

  17. Root Cause 1: Hot Keys Key-value pairs Popular keys have many associated values Lu Fang (UC Irvine) Final Defense May 26, 2017 26 / 51

  18. Root Cause 1: Hot Keys Key-value pairs Popular keys have many associated values Case study (from StackOverflow) ◮ Process StackOverflow posts ◮ Long and popular posts ◮ Many tasks process long and popular posts Lu Fang (UC Irvine) Final Defense May 26, 2017 26 / 51

  19. Root Cause 2: Large Intermediate Results Temporary data structures Lu Fang (UC Irvine) Final Defense May 26, 2017 27 / 51

  20. Root Cause 2: Large Intermediate Results Temporary data structures Case study (from StackOverflow) ◮ Use NLP library to process customers’ reviews ◮ Some reviews are quite long ◮ NLP library creates giant temporary data structures for long reviews Lu Fang (UC Irvine) Final Defense May 26, 2017 27 / 51

  21. Existing Solutions More memory? Not really! ◮ Data double in size every two years, [ http://goo.gl/tM92i0 ] ◮ Memory double in size every three years, [ http://goo.gl/50Rrgk ] Lu Fang (UC Irvine) Final Defense May 26, 2017 28 / 51

  22. Existing Solutions More memory? Not really! ◮ Data double in size every two years, [ http://goo.gl/tM92i0 ] ◮ Memory double in size every three years, [ http://goo.gl/50Rrgk ] Application-level solutions ◮ Configuration tuning ◮ Skew fixing Lu Fang (UC Irvine) Final Defense May 26, 2017 28 / 51

  23. Existing Solutions More memory? Not really! ◮ Data double in size every two years, [ http://goo.gl/tM92i0 ] ◮ Memory double in size every three years, [ http://goo.gl/50Rrgk ] Application-level solutions ◮ Configuration tuning ◮ Skew fixing System-level solutions ◮ Cluster-wide resource manager, such as YARN Lu Fang (UC Irvine) Final Defense May 26, 2017 28 / 51

  24. Existing Solutions More memory? Not really! ◮ Data double in size every two years, [ http://goo.gl/tM92i0 ] ◮ Memory double in size every three years, [ http://goo.gl/50Rrgk ] Application-level solutions ◮ Configuration tuning ◮ Skew fixing System-level solutions ◮ Cluster-wide resource manager, such as YARN We need a systematic and effective solution! Lu Fang (UC Irvine) Final Defense May 26, 2017 28 / 51

  25. Our Solution I nterruptible Task : treat memory pressure as interrupt Dynamically change parallelism degree Lu Fang (UC Irvine) Final Defense May 26, 2017 29 / 51

  26. Why Does Our Technique Help Heap size Memory consumption Program starts with multiple tasks Execution time Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  27. Why Does Our Technique Help Heap size Memory consumption Program pushes heap limit Execution time Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  28. Why Does Our Technique Help Heap size Memory consumption Long and useless GC Execution time Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  29. Why Does Our Technique Help Heap size Memory consumption OutOfMemory Error Execution time Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  30. Why Does Our Technique Help Heap size Memory consumption Long and useless GCs are detected Execution time Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  31. Why Does Our Technique Help Heap size Memory consumption Long and useless GCs are detected, start interrupting tasks Execution time Killed Killed Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  32. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Consumed Consumed Consumed Consumed Processed Input Memory Memory Memory Memory Unprocessed Input Output Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  33. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Released Consumed Consumed Consumed Consumed Processed Input Memory Memory Memory Memory Unprocessed Input Output Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  34. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Released Consumed Consumed Consumed Consumed Processed Input Released Memory Memory Memory Memory Unprocessed Input Output Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  35. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Released Consumed Consumed Consumed Consumed Processed Input Released Memory Memory Memory Memory Kept in memory, Unprocessed Input can be serialized Output Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  36. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Released Consumed Consumed Consumed Consumed Processed Input Released Memory Memory Memory Memory Kept in memory, Unprocessed Input can be serialized Final result: push Output out and released Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  37. Why Does Our Technique Help Heap size Memory consumption Release the memory, memory pressure is gone Execution time Consumed Memory Killed Killed Task Task Task Task Local Data Structures Released Consumed Consumed Consumed Consumed Processed Input Released Memory Memory Memory Memory Kept in memory, Unprocessed Input can be serialized Final result: push Output out and released Intermediate result: kept in memory, can be serialized Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  38. Why Does Our Technique Help Heap size Memory consumption Program executes without memory pressure Execution time Killed Killed Task Task Task Task Consumed Consumed Consumed Consumed Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  39. Why Does Our Technique Help Heap size Memory consumption If there is enough memory, increase parallelism degree Execution time Killed Killed Newly created Task Task Task Task Task Consumed Consumed Consumed Consumed Consumed Memory Memory Memory Memory Memory Lu Fang (UC Irvine) Final Defense May 26, 2017 30 / 51

  40. Making Task Interruptible Is Non-trivial Interrupted Consumed Memory Task Released or Kept in ? Memory? Consumed Released or Kept in ? Memory Memory? Released or Kept in ? Memory? Released or Kept in ? Memory? Lu Fang (UC Irvine) Final Defense May 26, 2017 31 / 51

  41. Making Task Interruptible Is Non-trivial Interrupted Consumed Memory Task Released or Kept in ? Memory? Consumed Released or Kept in ? Memory Require Semantics Memory? Released or Kept in ? Memory? Released or Kept in ? Memory? Lu Fang (UC Irvine) Final Defense May 26, 2017 31 / 51

  42. Challenges How to expose semantics How to interrupt/reactivate tasks Lu Fang (UC Irvine) Final Defense May 26, 2017 32 / 51

  43. Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks Lu Fang (UC Irvine) Final Defense May 26, 2017 32 / 51

  44. Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks → a runtime system Lu Fang (UC Irvine) Final Defense May 26, 2017 32 / 51

  45. Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks → a runtime system Lu Fang (UC Irvine) Final Defense May 26, 2017 32 / 51

  46. The Programming Model An ITask requires more semantics ◮ Separate processed and unprocessed input ◮ Specify how to serialize and deserialize ◮ Safely interrupt tasks ◮ Specify the actions when interrupt happens ◮ Merge the intermediate results Lu Fang (UC Irvine) Final Defense May 26, 2017 33 / 51

  47. The Programming Model An ITask requires more semantics ◮ Separate processed and unprocessed input ◮ Specify how to serialize and deserialize ◮ Safely interrupt tasks ◮ Specify the actions when interrupt happens ◮ Merge the intermediate results A unified representation of input/output A definition of an interruptible task Lu Fang (UC Irvine) Final Defense May 26, 2017 33 / 51

  48. Representing Input/Output as DataPartitions ◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data DataPartition Abstract Class // The DataPartition abstract class abstract class DataPartition { // Some fields and methods ... // A cursor points to the first // unprocessed tuple int cursor; // Serialize the DataPartition abstract void serialize(); // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang (UC Irvine) Final Defense May 26, 2017 34 / 51

  49. Representing Input/Output as DataPartitions ◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data DataPartition Abstract Class 1 A cursor points to the first // The DataPartition abstract class abstract class DataPartition { // Some fields and methods unprocessed tuple ... // A cursor points to the first // unprocessed tuple int cursor; // Serialize the DataPartition abstract void serialize(); // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang (UC Irvine) Final Defense May 26, 2017 34 / 51

  50. Representing Input/Output as DataPartitions ◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data DataPartition Abstract Class 1 A cursor points to the first // The DataPartition abstract class abstract class DataPartition { // Some fields and methods unprocessed tuple ... // A cursor points to the first // unprocessed tuple int cursor; 2 Users implement serialize and // Serialize the DataPartition abstract void serialize(); deserialize methods // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang (UC Irvine) Final Defense May 26, 2017 34 / 51

  51. Defining an ITask ◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task ITask Abstract Class // The ITask interface in the library abstract class ITask { // Some methods ... abstract void interrupt(); boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt if (HasMemoryPressure()) { interrupt(); return false; } process(); } } } Lu Fang (UC Irvine) Final Defense May 26, 2017 35 / 51

  52. Defining an ITask ◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task ITask Abstract Class // The ITask interface in the library abstract class ITask { // Some methods 1 In interrupt, we define how to deal ... abstract void interrupt(); with partial results boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt if (HasMemoryPressure()) { interrupt(); return false; } process(); } } } Lu Fang (UC Irvine) Final Defense May 26, 2017 35 / 51

  53. Defining an ITask ◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task ITask Abstract Class // The ITask interface in the library abstract class ITask { // Some methods 1 In interrupt, we define how to deal ... abstract void interrupt(); with partial results boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt 2 Tasks are always interrupted at the if (HasMemoryPressure()) { interrupt(); beginning in the scaleLoop return false; } process(); } } } Lu Fang (UC Irvine) Final Defense May 26, 2017 35 / 51

  54. Multiple Input for an ITask ◮ How to merge intermediate results MITask Abstract Class // The MITask interface in the library abstract class MITask extends ITask{ // Most parts are the same as ITask ... // The only difference boolean scaleLoop( PartitionIterator<DataPartition> i) { // Iterate partitions through iterator while (i.hasNext()) { DataPartition dp = (DataPartition) i.next(); // Iterate all the data tuples in this partition ... } return true; } } Lu Fang (UC Irvine) Final Defense May 26, 2017 36 / 51

  55. Multiple Input for an ITask ◮ How to merge intermediate results MITask Abstract Class // The MITask interface in the library abstract class MITask extends ITask{ // Most parts are the same as ITask ... // The only difference 1 scaleLoop takes a boolean scaleLoop( PartitionIterator<DataPartition> i) { // Iterate partitions through iterator PartitionIterator as input while (i.hasNext()) { DataPartition dp = (DataPartition) i.next(); // Iterate all the data tuples in this partition ... } return true; } } Lu Fang (UC Irvine) Final Defense May 26, 2017 36 / 51

  56. ITask WordCount on Hyracks HDFS ... Map Operator Map Operator Map Operator MapOperator Final Final Final class MapOperator extends ITask implements HyracksOperator { void interrupt() { // Push out final Shuffling // results to shuffling ... } ... // Some other fields and methods Reduce Operator Reduce Operator Reduce Operator ... } 1 1 1 n n Merge Operator Merge Operator HDFS Lu Fang (UC Irvine) Final Defense May 26, 2017 37 / 51

  57. ITask WordCount on Hyracks HDFS ... Map Operator Map Operator Map Operator ReduceOperator Final Final Final class ReduceOperator extends ITask implements HyracksOperator { void interrupt() { // Tag the results; Shuffling // Output as intermediate // results ... } ... Reduce Operator Reduce Operator Reduce Operator // Some other fields and methods ... } 1 1 1 n n Merge Operator Merge Operator HDFS Lu Fang (UC Irvine) Final Defense May 26, 2017 37 / 51

  58. ITask WordCount on Hyracks HDFS ... Map Operator Map Operator Map Operator MergeOperator Final Final Final class MergeTask extends MITask { void interrupt() { // Tag the results; Shuffling // Output as intermediate // results } // Some other fields and methods ... Reduce Operator Reduce Operator Reduce Operator ... } 1 1 1 n n Merge Operator Merge Operator HDFS Lu Fang (UC Irvine) Final Defense May 26, 2017 37 / 51

  59. Challenges How to expose semantics → a programming model How to interrupt/activate tasks → a runtime system Lu Fang (UC Irvine) Final Defense May 26, 2017 38 / 51

  60. ITask Runtime System Partition Manager Scheduler Data Partition Data Partition Data Partition Monitor Data Partition ITask Runtime System Lu Fang (UC Irvine) Final Defense May 26, 2017 39 / 51

  61. ITask Runtime System Partition Manager Scheduler Data Partition Grow/Reduce Data Partition Data Partition Check Reduce Monitor Data Partition Memory ITask Runtime System Lu Fang (UC Irvine) Final Defense May 26, 2017 39 / 51

  62. ITask Runtime System Disk ITasks Input/Output Serialize/Deserialize Partition Manager Scheduler Data Partition Grow/Reduce Data Partition Data Partition Check Reduce Monitor Data Partition Memory ITask Runtime System Lu Fang (UC Irvine) Final Defense May 26, 2017 39 / 51

  63. ITask Runtime System Disk ITasks Interrupt/Create Input/Output Serialize/Deserialize Partition Manager Scheduler Data Partition Grow/Reduce Data Partition Data Partition Check Reduce Monitor Data Partition Memory ITask Runtime System Lu Fang (UC Irvine) Final Defense May 26, 2017 39 / 51

  64. Evaluation Environments We have implemented ITask on ◮ Hadoop 2.6.0 ◮ Hyracks 0.2.14 Lu Fang (UC Irvine) Final Defense May 26, 2017 40 / 51

  65. Evaluation Environments We have implemented ITask on ◮ Hadoop 2.6.0 ◮ Hyracks 0.2.14 An 11-node Amazon EC2 cluster ◮ Each machine: 8 cores, 15GB, 80GB*2 SSD Lu Fang (UC Irvine) Final Defense May 26, 2017 40 / 51

  66. Experiments on Hadoop Goal ◮ Show the effectiveness on real-world problems Lu Fang (UC Irvine) Final Defense May 26, 2017 41 / 51

  67. Experiments on Hadoop Goal ◮ Show the effectiveness on real-world problems Benchmarks ◮ Original: five real-world programs collected from Stack Overflow ◮ RFix: apply the fixes recommended on websites ◮ ITask: apply ITask on original programs Name Dataset Map-Side Aggregation (MSA) Stack Overflow Full Dump In-Map Combiner (IMC) Wikipedia Full Dump Inverted-Index Building (IIB) Wikipedia Full Dump Word Cooccurrence Matrix (WCM) Wikipedia Full Dump Customer Review Processing (CRP) Wikipedia Sample Dump Lu Fang (UC Irvine) Final Defense May 26, 2017 41 / 51

  68. Improvements Benchmark Original Time RFix Time ITask Time Speed Up MSA 1047 (crashed) 48 72 -33.3% IMC 5200 (crashed) 337 238 41.6% IIB 1322 (crashed) 2568 1210 112.2% WCM 2643 (crashed) 2151 1287 67.1% CRP 567 (crashed) 6761 2001 237.9% ◮ With ITask, all programs survive memory pressure ◮ On average, ITask versions are 62.5% faster than RFix Lu Fang (UC Irvine) Final Defense May 26, 2017 42 / 51

  69. Experiments on Hyracks Goal ◮ Show the improvements on performance ◮ Show the improvements on scalability Lu Fang (UC Irvine) Final Defense May 26, 2017 43 / 51

Recommend

Diagnosing and Fixing MySQL Performance Problems Percona, Inc. http://www.percona.com/ 1

Diagnosing and Fixing MySQL Performance Problems Percona, Inc. http://www.percona.com/ 1

Diagnosing and Fixing MySQL Performance Problems Percona, Inc. http://www.percona.com/ 1 Monday, April 12, 2010 Table Of Contents 0. Welcome 5. Your Toolchest 1. Defining Performance 6. CPUs and Tasks 2. The Stack at 10000 Feet 7.

2.2k views • 198 slides
in Production Markus Weninger Johannes Kepler University Linz, Austria Institute for System

in Production Markus Weninger Johannes Kepler University Linz, Austria Institute for System

Analyzing Memory Errors in Production Markus Weninger Johannes Kepler University Linz, Austria Institute for System Software WHO ARE WE? Fixing Memory Problems In Production 2 WHO ARE WE? Fixing Memory Problems In Production 2 WHO ARE WE?

1.44k views • 119 slides
Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet

Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet

Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet Photos Tobias Weyand, Chih-Yun Tsai, Bastian Leibe Fixing WTFs Poster No. 26 Overview Many Computer Vision Applications use Internet photos,

818 views • 38 slides
Detecting Problems in the Database Access Code of Large Scale Systems An industrial Experience

Detecting Problems in the Database Access Code of Large Scale Systems An industrial Experience

Detecting Problems in the Database Access Code of Large Scale Systems An industrial Experience Report 1 Existing static analysis tools focus on language-related problems PMD Google error-prone Coverity FindBugs Facebook Infer However,

632 views • 31 slides
Solving Difficult Memory Performance Problems Jiri Olsa Joe Mario January 27, 2017 Red Hat

Solving Difficult Memory Performance Problems Jiri Olsa Joe Mario January 27, 2017 Red Hat

Solving Difficult Memory Performance Problems Jiri Olsa Joe Mario January 27, 2017 Red Hat Engineering Red Hat Performance Engineering Agenda Overview: Where does my program get its memory from? Types of expensive memory accesses

672 views • 63 slides
Memory II. Memory improvement III. Problems with memory 3 systems/stages of Memory: memory

Memory II. Memory improvement III. Problems with memory 3 systems/stages of Memory: memory

Main Focus I. Memory as a process Memory II. Memory improvement III. Problems with memory 3 systems/stages of Memory: memory the process by which I. Sensory Memory information is - acquired, II. Short -Term Memory - stored,

169 views • 5 slides
Dissecting Memory Problems A Semantic Approach Alfredo Gimenez Motivation Historical trends

Dissecting Memory Problems A Semantic Approach Alfredo Gimenez Motivation Historical trends

Dissecting Memory Problems A Semantic Approach Alfredo Gimenez Motivation Historical trends in memory performance and energy efficiency show that memory access is becoming one of the most significant bottlenecks to increasing performance

568 views • 32 slides
Detecting and Eliminating Memory Leaks Using Cyclic Memory Allocation Huu Hai Nguyen and Martin

Detecting and Eliminating Memory Leaks Using Cyclic Memory Allocation Huu Hai Nguyen and Martin

Introduction Implementation Experiments Conclusion Detecting and Eliminating Memory Leaks Using Cyclic Memory Allocation Huu Hai Nguyen and Martin Rinard Massachusetts Institute of Technology presented by: Hannes Payer Computational Systems

224 views • 20 slides
Fixing problems with grammars Informatics 2A: Lecture 12 John Longley &amp; Alex Simpson School

Fixing problems with grammars Informatics 2A: Lecture 12 John Longley &amp; Alex Simpson School

LL(1) grammars: summary Fixing problems with non-LL(1) grammars Self-assessment on context-free grammars Chomsky Normal Form Fixing problems with grammars Informatics 2A: Lecture 12 John Longley &amp; Alex Simpson School of Informatics

384 views • 21 slides
Memphis on an XT5 Pinpointing Memory Performance Problems on Cray Platforms Collin McCurdy,

Memphis on an XT5 Pinpointing Memory Performance Problems on Cray Platforms Collin McCurdy,

Memphis on an XT5 Pinpointing Memory Performance Problems on Cray Platforms Collin McCurdy, Jeffrey Vetter , Patrick Worley and Don Maxwell Overview Current projections: each chip in an Exascale system will contain 100s to 1000s of

430 views • 27 slides
Welcome to The Memory Class An Introduction to Memory Problems and the Memory Center Agenda For

Welcome to The Memory Class An Introduction to Memory Problems and the Memory Center Agenda For

Welcome to The Memory Class An Introduction to Memory Problems and the Memory Center Agenda For Todays Class Importance of the Questionnaire Description of the Memory Center Program Advanced Care Planning Lecture about Memory

695 views • 38 slides
Memory Leak A bug in a program that prevents it from C++: Memory Problems freeing up memory

Memory Leak A bug in a program that prevents it from C++: Memory Problems freeing up memory

Memory Leak A bug in a program that prevents it from C++: Memory Problems freeing up memory that it no longer needs. As a result, the program grabs more and more memory until it finally crashes or because there is no more memory left.

230 views • 6 slides
Combating the Advanced Memory Exploitation Techniques: Detecting ROP with Memory Information Leak

Combating the Advanced Memory Exploitation Techniques: Detecting ROP with Memory Information Leak

Combating the Advanced Memory Exploitation Techniques: Detecting ROP with Memory Information Leak nEINEI, Research Scientist @ McAfee Labs Chong Xu, Director of IPS Research @ McAfee Labs CanSecWest2014 March 20, 2014 Bio nEINEI

546 views • 31 slides
Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking

Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking

Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking Mateusz j00ru Jurczyk REcon 2017, Montreal Alternative title (cheers Alex Ionescu!) Memory Disclosure Alternative title KERNELBLEED Agenda

1.7k views • 140 slides
The I/O-Model Aggarwal and Vitter, The Input/Output Complexity of Sorting and Related Problems

The I/O-Model Aggarwal and Vitter, The Input/Output Complexity of Sorting and Related Problems

The I/O-Model Aggarwal and Vitter, The Input/Output Complexity of Sorting and Related Problems . Communications of the ACM, 31(9), p. 1116-1127, 1988. Page 1 Analysis of algorithms The standard model: Memory CPU Add : 1 unit of time

83 views • 7 slides
Speeding Up Crossbar Resistive Memory by Exploiting In-memory Data Patterns Wen Wen Lei Zhao,

Speeding Up Crossbar Resistive Memory by Exploiting In-memory Data Patterns Wen Wen Lei Zhao,

March 12, 2018 Speeding Up Crossbar Resistive Memory by Exploiting In-memory Data Patterns Wen Wen Lei Zhao, Youtao Zhang, Jun Yang Ex Execut utive Sum ummary Problems: performance and reliability of write operations The large sneaky

560 views • 37 slides
MC-Checker: Detecting Memory Consistency Errors in MPI One-Sided Applications Zhezhe Chen 1 ,

MC-Checker: Detecting Memory Consistency Errors in MPI One-Sided Applications Zhezhe Chen 1 ,

MC-Checker: Detecting Memory Consistency Errors in MPI One-Sided Applications Zhezhe Chen 1 , James Dinan 2 , Zhen Tang 3 , Pavan Balaji 4 , Hua Zhong 3 , Jun Wei 3 , Tao Huang 3 , and Feng Qin 5 1. Twitter Inc. 2. Intel Corporation 3. Chinese

398 views • 27 slides
Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking

Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking

Bochspwn Reloaded Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking Mateusz j00ru Jurczyk Black Hat USA 2017, Las Vegas Agenda User kernel communication pitfalls in modern operating systems Introduction

1.21k views • 120 slides
Today Memory Management Segmentation, Paging Improving memory performance MMU

Today Memory Management Segmentation, Paging Improving memory performance MMU

Today Memory Management Segmentation, Paging Improving memory performance MMU Translation Lookaside Buffer Dec 3, 2018 Sprenkle - CSCI330 1 Review What abstraction does virtual memory provide? What requirements do we

661 views • 25 slides
12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic graphics 2.Detecting manipulated images Photo manipulation is the application of image editing techniques to photographs in order to create an

306 views • 13 slides
Vlserver Memory Cache Mark Vitale OpenAFS Workshop 2019 19 Jun 2019 vlserver performance

Vlserver Memory Cache Mark Vitale OpenAFS Workshop 2019 19 Jun 2019 vlserver performance

Vlserver Memory Cache Mark Vitale OpenAFS Workshop 2019 19 Jun 2019 vlserver performance problems? For typical OpenAFS sites, fileservers and cache managers have the highest impact on overall cell performance; vlserver performance is close to

524 views • 23 slides
The Memory Hierarchy Many problems that modern computers are given to solve (analyzing

The Memory Hierarchy Many problems that modern computers are given to solve (analyzing

E XTERNAL M EMORY C OMPUTING hierarchical memory management B-trees external sorting External Memory Computing 1 The Memory Hierarchy Many problems that modern computers are given to solve (analyzing scientific data, running

655 views • 11 slides
High Performance Hardware, High Performance Hardware, Memory &amp; CPU Memory &amp; CPU Step

High Performance Hardware, High Performance Hardware, Memory &amp; CPU Memory &amp; CPU Step

High Performance Hardware, High Performance Hardware, Memory &amp; CPU Memory &amp; CPU Step Back, Look Inside, Many Dont Insight, not numbers! Face real world 6 GB/s CPU B K 2 3 cache 2 GB Rubin H. Landau Rubin H. Landau RAM 2

671 views • 13 slides
DETECTING ILLICIT DISCHARGES AND ILLEGAL CONNECTIONS (IC/ID) Workshop on Identifying Problems

DETECTING ILLICIT DISCHARGES AND ILLEGAL CONNECTIONS (IC/ID) Workshop on Identifying Problems

DETECTING ILLICIT DISCHARGES AND ILLEGAL CONNECTIONS (IC/ID) Workshop on Identifying Problems Overview Our MS4 Permits with the Regional Water Boards require us to inspect our facilities for spills, dumpings, illegal pipes, connections

483 views • 27 slides

More recommend