MMap (Memory Mapping) Simple, minimalist approach to scale up - PowerPoint PPT Presentation

Mar 15, 2024 •355 likes •531 views

Class Website CX4242: MMap (Memory Mapping) Simple, minimalist approach to scale up computation Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech When should you use Spark/Hadoop, AWS, Azure? And when should you

Class Website CX4242: MMap (Memory Mapping) Simple, minimalist approach to scale up computation Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech
When should you use Spark/Hadoop, AWS, Azure? And when should you not?
MMap Fast Billion-Scale Graph Computation on a PC via Memory Mapping Lead by Zhiyuan (Jerry) Lin Georgia Tech CS Undergrad Now: Stanford PhD student MMap: Fast Billion-Scale Graph Computation on a PC via Memory Mapping . Zhiyuan Lin, Minsuk Kahng, Kaeser Md. Sabrin, Duen Horng Chau, Ho Lee, and U Kang. Proceedings of IEEE BigData 2014 conference. Oct 27-30, Washington DC, USA. Towards Scalable Graph Computation on Mobile Devices. Yiqi Chen, Zhiyuan Lin, Robert Pienta, Minsuk Kahng, Duen Ho 3
Graph Computation on Computer Cluster? Steep learning curve Cost Overkill for smaller graphs Image source: http://www.drupaltky.org/en/article/20
Best-of-breed Single-PC Approaches GraphChi – OSDI 2012 • TurboGraph – KDD 2013 • What do they have in common? Sophisticated Data Structures • Explicit Memory Management •
Can We Do Less? To get same or better performance? e.g., auto memory management, faster, etc.
Main Idea: Memory-mapped the Graph 7
How to compute PageRank for huge matrix? 2 3 1 Use the power iteration method http://en.wikipedia.org/wiki/Power_iteration 4 p = c B p + (1-c) 1 5 n p’ B p (1-c) = c + n Can initialize this vector to any non- zero vector, e.g., all “1”s 8
Example: PageRank (implemented using MMap) http://www.cc.gatech.edu/~dchau/papers/14-bigdata-mmap.pdf 9
8000 lines of code 10
11
Why Memory Mapping Works? High- degree nodes’ info automatically cached/kept in memory for future frequent access Read-ahead paging preemptively loads edges from disk. Highly-optimized by the OS No need to explicitly manage memory (less book-keeping)
Also works on tablets! (If you want.) Big Data on Small Devices (270M+ Edges) 14
“Mobile” devices are now very powerful https://www.macrumors.com/2018/11/01/2018-ipad-pro-benchmarks-geekbench/ 15
Lead by Dezhi (Andy) Fang, Georgia Tech CS Undergrad. Now: Airbnb software engineer
MMap project website http://poloclub.gatech.edu/mmap/ 17

Recommend

Graph Computation on Computer Cluster? Steep learning curve Cost Overkill for smaller

MMap Fast Billion-Scale Graph Computation on a PC via Memory Mapping Lead by Zhiyuan (Jerry) Lin Georgia Tech CS Undergrad Now: Stanford 1st year PhD student MMap: Fast Billion-Scale Graph Computation on a PC via Memory Mapping .

130 views • 11 slides

Dirty COW Race Condition Attack Outline Dirty COW vulnerability Memory Mapping using

Dirty COW Race Condition Attack Outline Dirty COW vulnerability Memory Mapping using mmap() Map_shared, Map_Private Mapping Read-Only Files How to exploit? Dirty COW vulnerability Interesting case of the race

667 views • 24 slides

MMAP Michigan Medicare/Medicaid Assistance Program Bob Callery - MMAP Regional Coordinator Last

MMAP Michigan Medicare/Medicaid Assistance Program Bob Callery - MMAP Regional Coordinator Last Update 7-27-15 This publication has been created and produced by Area Agency on Aging of Western Michigan with the financial assistance, in whole

652 views • 22 slides

Dirty COW Attack Instructor: Fengwei Zhang 1 SUSTech CS 315 Computer Security Outline

Dirty COW Attack Instructor: Fengwei Zhang 1 SUSTech CS 315 Computer Security Outline Dirty COW vulnerability Memory Mapping using mmap() Map_shared, Map_Private Mapping Read-Only Files How to exploit? 2 Dirty COW

466 views • 24 slides

Memory System Design Chapter 16 S. Dandamudi Outline Introduction Building larger

Memory System Design Chapter 16 S. Dandamudi Outline Introduction Building larger memories A simple memory block Mapping memory Memory design with D flip Full mapping flops Partial mapping Problems with the

500 views • 15 slides

Memory System Design Chapter 16 S. Dandamudi Outline Introduction Building larger

466 views • 30 slides

NUMA obliviousness through memory mapping Mrunal Gawade Martin Kersten CWI, Amsterdam

NUMA obliviousness through memory mapping Mrunal Gawade Martin Kersten CWI, Amsterdam DaMoN 2015 (1 st June 2015) Melbourne, Australia NUMA architecture Intel Xeon E5-4657L v2 @2.40GHz Memory mapping What is it? Operating system maps

832 views • 23 slides

1 3. Simple Memory System Cache Address Translation Example #1 16 lines, 4-byte block size

Today Simple memory system example Case study: Core i7/Linux memory system Virtual Memory: Systems Memory mapping CSci 2021: Machine Architecture and Organization April 20th-22nd, 2020 Your instructor: Stephen McCamant Based on

587 views • 5 slides

virtual memory 2 1 xv6 memory layout 0x80000000 (KERNBASE) page tables store this mapping (one

virtual memory 2 1 xv6 memory layout 0x80000000 (KERNBASE) page tables store this mapping (one kernel, one user) for user memory two virtual address same in every process = PA VA 0x8000000 + kernel-only memory 2 Virtual 4 Gig Device

1.7k views • 122 slides

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of memory systems Memory hierarchy Cache memory principles Elements of cache design Cache size Mapping function Replacement

793 views • 64 slides

TEXTURE MAPPING 1 OUTLINE Introduce Mapping Methods Texture Mapping Environment

TEXTURE MAPPING 1 OUTLINE Introduce Mapping Methods Texture Mapping Environment Mapping Bump Mapping Basic mapping strategies Forward vs backward mapping Point sampling vs area averaging Introduce

775 views • 25 slides

Image Warping Image Mapping Image Mapping - Examples Forward Mapping Forward Mapping -

Image Warping Image Mapping Image Mapping - Examples Forward Mapping Forward Mapping - Disadvantages Example Forward Mapping Original Rotated Zoom In Backward Mapping The Problem: (u,v) are not integers! "# Nearest Neighbor

715 views • 49 slides

Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy Principle of Locality Some

Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy Principle of Locality Some Definitions Cache Architectures Fully Associative Direct Mapping Set Associative Replacement Policy Main Memory Update Polic

675 views • 40 slides

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and Feng Chen Louisiana State University SoCC '18 Carlsbad, CA Key-value Systems in Internet Services Key-value systems are widely used today

1.21k views • 84 slides

Memory II. Memory improvement III. Problems with memory 3 systems/stages of Memory: memory

Main Focus I. Memory as a process Memory II. Memory improvement III. Problems with memory 3 systems/stages of Memory: memory the process by which I. Sensory Memory information is - acquired, II. Short -Term Memory - stored,

169 views • 5 slides

CAM: CONSTRAINT AWARE APPLICATION MAPPING FOR APPLICATION MAPPING FOR EMBEDDED SYSTEMS Luis

CAM: CONSTRAINT AWARE APPLICATION MAPPING FOR APPLICATION MAPPING FOR EMBEDDED SYSTEMS Luis A. Bathen, Nikil D. Dutt Outline 2 Introduction & Motivation Introduction & Motivation CAM Overview Memory aware Macro

508 views • 40 slides

Texture Mapping Surface mapping OpenGl and Implementation Details Texture mapping Bump

Texture Mapping Surface mapping OpenGl and Implementation Details Texture mapping Bump Mapping CS351-50 Displacement mapping Actually moving geometry ie Create screw from cylinder, Terrain, etc 11.05.03 An Overview An

496 views • 11 slides

1 Memory SoC Persistent Memory-Driven Memory Memory Processor-Centric Memory SoC SoC

1 Memory SoC Persistent Memory-Driven Memory Memory Processor-Centric Memory SoC SoC Computing Computing + Fabric SoC Memory HYPERCONVERGED Exascale EDGE DEVICE SYSTEM Eliminate data movement via shared

400 views • 11 slides

Texture and other Mappings Texture Mapping Texture Mapping Bump Mapping Bump Mapping

Texture and other Mappings Texture Mapping Texture Mapping Bump Mapping Bump Mapping Displacement Mapping Displacement Mapping Environment Mapping Environment Mapping Angel Chapter 7 Last Time - Shading Flat shading: Inexpensive to

629 views • 26 slides

Speeding things up: Getting sloooower The TLB Memory Exception Every new level of paging no p

Speeding things up: Getting sloooower The TLB Memory Exception Every new level of paging no p o CPU reduces the memory overhead for computing yes page # frame # Access the mapping function f but increases the time necessary

201 views • 4 slides

Texture and other Mappings Texture Mapping Bump Mapping Displacement Mapping Environment

Texture and other Mappings Texture Mapping Bump Mapping Displacement Mapping Environment Mapping Example: Checkerboard Particularly severe problems in regular textures 1 The Beginnings of a Solution: Mipmapping Pre-calculate how

552 views • 14 slides

RGB-D Mapping Overview CSE 571 Robotics Map RGB-D Mapping `` University of Washington Dieter

11/6/16 RGB-D Mapping Overview CSE 571 Robotics Map RGB-D Mapping `` University of Washington Dieter Fox RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments. Henry et al. ISER 2010 RGB-D Mapping: Using

240 views • 13 slides

C P S C 314 WHY IS TEXTURE IMPORTANT? TEXTURE MAPPING TEXTURE MAPPING TEXTURE MAPPING real

C P S C 314 WHY IS TEXTURE IMPORTANT? TEXTURE MAPPING TEXTURE MAPPING TEXTURE MAPPING real life objects have hide geometric simplicity nonuniform colors, normals images convey illusion of geometry to generate realistic objects,

448 views • 3 slides

UMBC M A L F T U M B C I O M Y O T R 1 (April 10, 2000 12:36 pm) I E S R C

Systems Programming 8086/88 Memory Interface II CMPE 310 Memory Address Decoding The processor can usually address a memory space that is much larger than the memory space covered by an individual memory chip. In order to splice a memory device

713 views • 21 slides