HYDRAstor: a Scalable Secondary Storage 7th TF-Storage Meeting - PowerPoint PPT Presentation

HYDRAstor: a Scalable Secondary Storage 7th TF-Storage Meeting September 9 th 2010 Łukasz Heldt

Largest Japanese IT company Polish R&D company $43 Billion in annual revenue 50 engineers and scientists 143,000 staff www.9livesdata.com www.nec.com Owns R&D of critical & sells backend component Scalable disk based storage for backup with global deduplication Started in 2003 in NEC Labs by Cezary Dubnicki 2007 Product of the year award by SearchStorage.com 2008 Product innovation award by Network Products Guide 2009/2010 FAST conference publication in San Jose Sold in US and Japan since 2007 Will be sold in Poland in 2011 by 9LivesData in coop. with NEC

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 3 Backup storage ● Tapes are most common, despite: ● Sensitive environment requirements ● Unreliable restore ● Low performance ● Manual labor or expensive robots ● Problematic replication

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 4 Backup storage size ● Usual backup policy ● Secondary storage size: ● 4-12+ full backups ● 5x-20x more than ● 7-30+ incremental primary storage ● Majority of data does ● Includes many copies not change of the same data ● Data compression 2:1 ● Each data chunk stored 5-10+ times

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 5 Backup storage size ● Usual backup policy ● Secondary storage size: ● 4-12+ full backups ● 5x-20x more than ● 7-30+ incremental primary storage ● Majority of data does ● Includes many copies not change of the same data ● Data compression 2:1 ● Each data chunk stored 5-10+ times High potential for the deduplication technology.

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 6 Deduplication ● Save disk space by eliminating duplicates ● Sample reduction ratio 10:1 (depends on backup policy) ● Lowers price of gigabyte Sub-file level deduplication File A A B C Stored blocks Only unique blocks Only unique blocks File B A D E A B C D E are stored are stored File A A B C

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 7 Global deduplication ● Prevent silos of deduped data ● One system to manage Global vs. siloed dedup

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 8 HYDRAstor product ● Provides ● global deduplication using DataRedux™ ● performance, storage scalability and data resiliency using Distributed Resilient Data™

HYDRAstor deployment 9 ● Interface: CIFS, NFS, Symantec OST ● Marker filtering for: Tivoli, Netbackup, Networker, CommVault

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 10 HYDRAstor architecture ● Accelerator Nodes realize performance ● Storage Nodes realize capacity NFS / CIFS / OST over Ethernet Accelerator Nodes Internal Network Storage Nodes

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 11 HYDRAstor architecture ● Accelerator Nodes realize performance ● Storage Nodes realize capacity NFS / CIFS / OST over Ethernet Accelerator Nodes Non-disruptive Internal grid expansion Network Storage Nodes

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 12 HYDRAstor scalability ● MiniHYDRA – single server ● Storage: 12 TB – 240 TB* ● Performance: 1.3 TB / hour ● 2AN 4SN ● Storage: 48 TB – 960 TB* ● Performance: 3.6 TB / hour ● 20AN 40SN (4 racks) ● Storage: 480 TB – 9600 TB* ● Performance: 36 TB / hour * - assuming 20x data reduction through DataRedux™

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 13 HYDRAstor scalability ● Slide from Curtis Preston presentation Curtis Preston is a famous storage analyst owning independent consulting company

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 14 HYDRAstor other features ● Fully automatic/non-disruptive mgmt ● Recovery of lost data resiliency ● Periodic data scrubbing ● Machine and disk failure recovery ● Configurable redundancy level ● erasure coding – better than RAID6 ● Optimized replication ● Smart resource management

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 15 HYDRAstor backend design Details of the design: http://www.usenix.org/events/fast09/tech/full_papers/dubnicki/dubnicki.pdf

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 16 Programming Model ● Repository of blocks ● Content-addressed ● Immutable ● Variable-sized hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 17 Programming Model ● Repository of blocks ● Content-addressed ● Immutable ● Variable-sized ● Exposed pointers to other blocks E 011..0 hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 18 Programming Model ● Repository of blocks hash=010..1 Root1 E ● Content-addressed ● Immutable ● Variable-sized ● Exposed pointers to other E E blocks ● Trees of blocks E 011..0 hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 19 Programming Model ● Repository of blocks hash=010..1 Root2 E Root1 E ● Content-addressed ● Immutable hash=110..0 ● Variable-sized ● Exposed pointers to other E E blocks 0 ● Trees of blocks . . 1 1 E 0 ● DAGs due to deduplication 011..0 ● No cycles possible hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 20 Programming Model ● Repository of blocks hash=010..1 Root2 E Root1 E ● Content-addressed ● Immutable hash=110..0 ● Variable-sized ● Exposed pointers to other E E blocks 0 ● Trees of blocks . . 1 1 E 0 ● DAGs due to deduplication 011..0 ● No cycles possible ● Deletion of whole trees hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 21 Programming Model ● Repository of blocks hash=010..1 Root2 E Root1 E ● Content-addressed ● Immutable hash=110..0 ● Variable-sized ● Exposed pointers to other E E blocks 0 ● Trees of blocks . . 1 1 E 0 ● DAGs due to deduplication 011..0 ● No cycles possible ● Deletion of whole trees hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 22 Programming Model ● Repository of blocks hash=010..1 Root2 E Root1 E ● Content-addressed ● Immutable hash=110..0 ● Variable-sized ● Exposed pointers to other E E blocks 0 ● Trees of blocks . . 1 1 E 0 ● DAGs due to deduplication 011..0 ● No cycles possible ● Deletion of whole trees hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 23 Programming Model ● Repository of blocks Root2 E ● Content-addressed ● Immutable hash=110..0 ● Variable-sized ● Exposed pointers to other blocks 0 ● Trees of blocks . . 1 1 E 0 ● DAGs due to deduplication 011..0 ● No cycles possible ● Deletion of whole trees hash=011..0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 24 Failure tolerance: erasure coding Example: N=8, m=5 t s n t n a d e m n u g d a e r R F Decode Encode s l t a n n e i m g i g r a O Original r F block Any 3 fragments can be lost

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 25 Failure tolerance: erasure coding Example: N=8, m=5 t s n t n a d e m n u g d a e r R F Decode Encode s l t a n n e i m g i g r a O Original r F block Any 3 fragments can be lost Assuming 12 disks array Mirror 3-copy RAID6 Erasure coding Resiliency 1 2 2 2 3 Overhead 100% 200% 20% 20% 33%

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 26 Scalability with DHT: data placement ● Block location: DHT with prefix routing empty prefix 0 0 1 00 01 01 10 11

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 27 Scalability with DHT: data placement ● Block location: DHT with prefix routing ● Block mapped to hash prefix hash= 01 1..0 empty prefix Block 0 0 1 00 01 01 10 11

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 28 Scalability with DHT: data placement ● Block location: DHT with prefix routing ● Block mapped to hash prefix hash= 01 1..0 empty prefix Block ● Prefix components 0 1 ● Hosted on SNs N=4 ● N components 00 01 10 11 per prefix 1 0 2 Node 1 Node 1 Node 2 Node 1 3 1 1 2 Node 3 Node 1 3 0 2 Node 1 Node 4 0 Node 1 Node 5 2 1 Node 1 Node 6 3 3 0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 29 Scalability with DHT: data placement ● Block location: DHT with prefix routing ● Block mapped to hash prefix hash= 01 1..0 empty prefix Block ● Prefix components 0 1 ● Hosted on SNs N=4 ● N components 00 01 10 11 per prefix 1 0 2 Node 1 Node 1 ● Store fragments Node 1 Node 2 3 1 1 2 Node 1 Node 3 3 0 2 Node 1 Node 4 0 Node 5 Node 1 2 1 Node 6 Node 1 3 3 0

HYDRAstor: a Scalable Secondary Storage. 9LivesData, LLC 30 Scalability with DHT: data placement ● Block location: DHT with prefix routing ● Block mapped to hash prefix hash= 01 1..0 empty prefix Block ● Prefix components 0 1 ● Hosted on SNs N=4 ● N components 00 01 10 11 per prefix 1 0 2 Node 1 Node 1 ● Store fragments Node 1 Node 2 3 1 1 ● Distributed 2 Node 1 Node 3 3 0 2 Node 4 Node 1 0 consensus Node 5 Node 1 2 1 Node 6 Node 1 3 3 0