Algorithms and Methods for Distributed Storage Networks 1. - PowerPoint PPT Presentation

Algorithms and Methods for Distributed Storage Networks 1. Motivation, Organization, Overview Christian Schindelhauer Albert-Ludwigs-Universität Freiburg Institut für Informatik Rechnernetze und Telematik Wintersemester 2007/08

Organization ‣ Lecture • Thursday 11 am - 1pm, room 101/SR 01-009/13 • Friday 11 am - 12 pm, room 101/SR 01-009/13 ‣ Exercise (Stefan Rührup) • starts Oct 29, 2008 • Friday 12 - 1 pm, room 101/SR 01-009/13 • appear every Friday on the web-pages • solved voluntarily by students • are the bases for the oral exam • solutions of the exercises are discussed in the following week Rechnernetze und Telematik Distributed Storage Networks 2 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Web ‣ Web page • http://cone.informatik.uni-freiburg.de/teaching/ vorlesung/distributed-storage-w08 • Slides, exercises, link to forum ‣ Forum • for discussion, links, funnies etc. • http://cone.informatik.uni-freiburg.de/forum/ viewforum.php?f=28 Computer Networks and Telematics Peer-to-Peer-Networks Albert-Ludwigs-Universität Freiburg 3 Summer 2008 Christian Schindelhauer

Exam ‣ Dates by appointment • possible dates - Monday, 23.02.2009 - Thursday, 05.03.2009 - Tuesday, 07.04.2009 • Contact me during the lecture or send an E-Mail to ‣ Oral exam • based on the lecture and the exercises ‣ Mandatory registration • Students of computer science register at the secretary of exams (Prüfungssekretariat) Computer Networks and Telematics Peer-to-Peer-Networks Albert-Ludwigs-Universität Freiburg 4 Summer 2008 Christian Schindelhauer

Algorithms and Methods for Distributed Storage Networks Motivation Evolution of Disks Rechnernetze und Telematik Distributed Storage Networks 5 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 6 Cost of storage at the disk drive and system level Fall in Storage 1000 SYSTEMS W/LARGE SYSTEMS W/SMALL FF DRIVES FF DRIVES Prices 100 10 PRICE/MBYTE, DOLLARS 1 INDUSTRY PROJECTION FROM 0.1 MULTIPLE SOURCES HIGH 0.01 PERFOR- MANCE, SERVER HDD DESKTOP Technological impact of magnetic 0.001 HDD hard disk drives on storage systems, SMALL FF HDD STORAGE SYSTEM Grochowski, R. D. Halem 0.0001 IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 1980 1985 1990 1995 2000 2005 2010 PRODUCTION YEAR Figure 7 Cost of storage for disk drive, paper, film, and semiconductor memory Rechnernetze und Telematik Distributed Storage Networks 6 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 7 Cost of storage for disk drive, paper, film, and semiconductor memory Price Fall of 1000 RAM and Disk 100 SEMICONDUCTOR Storage PRICE (DOLLARS PER MEGABYTE) MEMORY (DRAM AND FLASH) 10 DISK DRIVES 1 PAPER AND FILM 0.1 2.5 INCH 0.01 3.5 INCH Technological impact of magnetic 0.001 hard disk drives on storage systems, Grochowski, R. D. Halem 0.0001 IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 1980 1985 1990 1995 2000 2005 PRODUCTION YEAR Rechnernetze und Telematik Distributed Storage Networks 7 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 1 Hard disk drive areal density trend 10 6 IBM DISK DRIVE PRODUCTS RANGE OF Increase of INDUSTRY LAB DEMOS POSSIBLE VALUES 10 5 Density 1st AFC MEDIA AREAL DENSITY MEGABITS/SQUARE INCH 10 4 100% CGR 1st GMR HEAD 10 3 60% CGR 1st MR HEAD 10 2 1st THIN FILM HEAD 10 ~35 MILLION X 25% CGR INCREASE 1 10 – 1 Technological impact of magnetic 10 – 2 hard disk drives on storage systems, Grochowski, R. D. Halem IBM RAMAC ™ (FIRST HARD DISK DRIVE) IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 10 – 3 1960 1970 1980 1990 2000 2010 PRODUCTION YEAR AFC=ANTIFERROMAGNETICALLY COUPLED GMR=GIANT MAGNETORESISTIVE Rechnernetze und Telematik Distributed Storage Networks 8 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 2 Storage floor space utilization trend — IBM storage systems Increase of 10 4 Density RAW CAPACITY PER FLOOR SPACE AREA, GBYTES/SQUARE FOOT RANGE OF POSSIBLE VALUES 10 3 (Floor Space) IBM ESS MOD 800 ESS MOD F 10 2 60% CGR RAMAC 2 10 1 10 0 25% CGR 3380 10 – 1 10 – 2 Technological impact of magnetic hard disk drives on storage systems, 10 – 3 Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 IBM RAMAC (FIRST HARD DISK DRIVE) 10 – 4 1960 1970 1980 1990 2000 2010 PRODUCTION YEAR ESS=IBM TOTALSTORAGE ™ ENTERPRISE STORAGE SERVER ™ Rechnernetze und Telematik Distributed Storage Networks 9 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 3 Floor space required to store 1 terabyte Increase of 10 7 IBM RAMAC (FIRST HARD DISK DRIVE) Density 10 6 (Floor Space) FLOOR SPACE AREA, SQUARE FOOT 10 5 10 4 3380 10 3 10 2 RAMAC 2 10 ESS MOD F ESS MOD 800 1 Technological impact of magnetic RANGE OF hard disk drives on storage systems, POSSIBLE VALUES Grochowski, R. D. Halem 0.1 1960 1970 1980 1990 2000 2010 IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 PRODUCTION YEAR Rechnernetze und Telematik Distributed Storage Networks 10 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 4 Hard disk drive volumetric density trend Increase of 10 4 HARD DISK DRIVE RANGE OF RANGE OF FORM FACTOR Density POSSIBLE VALUES POSSIBLE VALUES 14/10.8 INCH 5.25 10 3 3.5 2.5 (Cubic Space) VOLUMETRIC DENSITY, GBITS/CUBIC INCH 1.0 10 2 CGR = 100% 10 1 10 0 10 – 1 CGR = 25% 10 – 2 Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem 10 – 3 IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 1980 1985 1990 1995 2000 2005 2010 PRODUCTION YEAR Rechnernetze und Telematik Distributed Storage Networks 11 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Figure 8 The evolution of disk drive form factors 1956 – 2002 24-INCH FORM FACTOR 5 MB Evolution of (NOT SHOWN) Disk Form 14-INCH FORM FACTOR 3.8 GB Factors 8-INCH FORM FACTOR 65 MB 10.8-INCH FORM FACTOR 5.7 GB Technological impact of magnetic 3.25-INCH FORM FACTOR 120 GB hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 3.25-INCH FORM FACTOR 84-MM DISKS/10000 RPM 146 GB 70-MM DISKS/15000 RPM 36 GB 2.5-INCH FORM FACTOR 60 GB Rechnernetze und Telematik Distributed Storage Networks 1.0-INCH 12 Albert-Ludwigs-Universität Freiburg FORM FACTOR Winter 2008/09 1 GB Christian Schindelhauer

Figure 10 Hard disk drive maximum internal data rate for enterprise/server drives Increase of 1000 Speed MAXIMUM INTERNAL DATA RATE, 100 MBYTES/SECONDS 40% CGR 10 <10% CGR 1 DATA RATE = LINEAR x RPM x DISK DENSITY DIAMETER 0.1 1970 1980 1990 2000 Technological impact of magnetic AVAILABILITY YEAR hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 Rechnernetze und Telematik Distributed Storage Networks 13 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Increase of Speed Figure 11 Disk drive access/seek times 100 ULTRASTAR XP ULTRASTAR 2XP ULTRASTAR 18XP ULTRASTAR 9ZX TIME, MILLISECONDS ULTRASTAR 36XP ULTRASTAR 18ZX ULTRASTAR 36ZX 10 ULTRASTAR 73LZX ULTRASTAR 36LZX ULTRASTAR 36Z15 SEEK TIME ~ (ACTUATOR INERTIALPOWER) 1/3 x (DATA BAND) 2/3 ~ ~ ROTATIONAL TIME (LATENCY) ~ (RPM) -1 ACCESS TIME = SEEK TIME + LATENCY SEEKING 1 ACCESSING 1970 1975 1980 1985 1990 1995 2000 2005 AVAILABILITY YEAR Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 Rechnernetze und Telematik Distributed Storage Networks 14 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Algorithms and Methods for Distributed Storage Networks Motivation Consumer Behavior Rechnernetze und Telematik Distributed Storage Networks 15 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Consumer Usage ‣ Consumer Survey on Digital Storage in Consumer Electronics 2008, Coughlin Associates (Dec. 2007) • 51% said that 1 TB disk would be useful • Most storage of content was on hard disk • 46% backup data less than once per year - except pictures most of them do not backup - but most think it is important to have backups out of their homes • Most people want to store entire TV series, copies of their entire music collection ‣ Projection • by 2013 average home has 9 Terabyte • by 2015 user content sums up to 650 Exabyte Rechnernetze und Telematik Distributed Storage Networks 16 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer

Storage Hierarchy ‣ Primary storage • Processors registers • Processor cache • RAM ‣ Secondary storage • Hard disks • Solid state disks • CD, DVD ‣ Tertiary storage • tape libraries • optical jukeboxes Rechnernetze und Telematik Distributed Storage Networks 17 Albert-Ludwigs-Universität Freiburg Winter 2008/09 Christian Schindelhauer