Partial MDS Codes with Local Regeneration Lukas Holzbaur , Sven - PowerPoint PPT Presentation

Partial MDS Codes with Local Regeneration Lukas Holzbaur , Sven Puchinger, Eitan Yaakobi, Antonia Wachter-Zeh Technical University of Munich Institute for Communications Engineering

Introduction • Large scale distributed storage systems need to be efficiently repairable • Two important measures of efficiency: 1 Kamath, Govinda M., et al. "Codes with local regeneration and erasure correction." IEEE Transactions on Information Theory 60.8 (2014): 4637-4660. 2 Rawat, Ankit Singh, et al. "Optimal locally repairable and secure codes for distributed storage systems." IEEE Transactions on Information Theory 60.1 (2013): 212-236. 3 Krishnan, M. Nikhil, and P . Vijay Kumar. "Codes with Combined Locality and Regeneration Having Optimal Rate, d min and Linear Field Size." 2018 IEEE International Symposium on Information Theory (ISIT). IEEE, 2018. 4 Calis, Gokhan, and O. Ozan Koyluoglu. "A general construction for PMDS codes." IEEE Communications Letters 21.3 (2016): 452-455. Lukas Holzbaur (TUM) 2

Introduction • Large scale distributed storage systems need to be efficiently repairable • Two important measures of efficiency: ◮ Locality: Number of nodes involved in repair → LRCs / PMDS codes 1 Kamath, Govinda M., et al. "Codes with local regeneration and erasure correction." IEEE Transactions on Information Theory 60.8 (2014): 4637-4660. 2 Rawat, Ankit Singh, et al. "Optimal locally repairable and secure codes for distributed storage systems." IEEE Transactions on Information Theory 60.1 (2013): 212-236. 3 Krishnan, M. Nikhil, and P . Vijay Kumar. "Codes with Combined Locality and Regeneration Having Optimal Rate, d min and Linear Field Size." 2018 IEEE International Symposium on Information Theory (ISIT). IEEE, 2018. 4 Calis, Gokhan, and O. Ozan Koyluoglu. "A general construction for PMDS codes." IEEE Communications Letters 21.3 (2016): 452-455. Lukas Holzbaur (TUM) 2

Introduction • Large scale distributed storage systems need to be efficiently repairable • Two important measures of efficiency: ◮ Locality: Number of nodes involved in repair → LRCs / PMDS codes ◮ Repair Bandwidth: Amount of data transmitted for repair → Regenerating codes 1 Kamath, Govinda M., et al. "Codes with local regeneration and erasure correction." IEEE Transactions on Information Theory 60.8 (2014): 4637-4660. 2 Rawat, Ankit Singh, et al. "Optimal locally repairable and secure codes for distributed storage systems." IEEE Transactions on Information Theory 60.1 (2013): 212-236. 3 Krishnan, M. Nikhil, and P . Vijay Kumar. "Codes with Combined Locality and Regeneration Having Optimal Rate, d min and Linear Field Size." 2018 IEEE International Symposium on Information Theory (ISIT). IEEE, 2018. 4 Calis, Gokhan, and O. Ozan Koyluoglu. "A general construction for PMDS codes." IEEE Communications Letters 21.3 (2016): 452-455. Lukas Holzbaur (TUM) 2

Introduction • Large scale distributed storage systems need to be efficiently repairable • Two important measures of efficiency: ◮ Locality: Number of nodes involved in repair → LRCs / PMDS codes ◮ Repair Bandwidth: Amount of data transmitted for repair → Regenerating codes • Several constructions of locally regenerating codes are known 123 But: Only one with PMDS 24 property, for which the field size is exponential in the length 1 Kamath, Govinda M., et al. "Codes with local regeneration and erasure correction." IEEE Transactions on Information Theory 60.8 (2014): 4637-4660. 2 Rawat, Ankit Singh, et al. "Optimal locally repairable and secure codes for distributed storage systems." IEEE Transactions on Information Theory 60.1 (2013): 212-236. 3 Krishnan, M. Nikhil, and P . Vijay Kumar. "Codes with Combined Locality and Regeneration Having Optimal Rate, d min and Linear Field Size." 2018 IEEE International Symposium on Information Theory (ISIT). IEEE, 2018. 4 Calis, Gokhan, and O. Ozan Koyluoglu. "A general construction for PMDS codes." IEEE Communications Letters 21.3 (2016): 452-455. Lukas Holzbaur (TUM) 2

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols Example: n = 8, r = 3 1 2 3 4 5 6 7 8 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols • Minimum Storage Regenerating (MSR) Codes ◮ Maximum Distance Separable (MDS) Code ◮ The repair bandwidth is 5 ( n − 1 ) · ℓ r Example: n = 8, r = 3 1 2 3 4 5 6 7 8 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols • Minimum Storage Regenerating (MSR) Codes ◮ Maximum Distance Separable (MDS) Code ◮ The repair bandwidth is 5 ( n − 1 ) · ℓ r Example: n = 8, r = 3 1 2 3 4 5 6 7 8 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols • Minimum Storage Regenerating (MSR) Codes ◮ Maximum Distance Separable (MDS) Code ◮ The repair bandwidth is 5 ( n − 1 ) · ℓ r Example: n = 8, r = 3 1 2 3 4 5 6 7 8 4 ′ 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols • Minimum Storage Regenerating (MSR) Codes ◮ Maximum Distance Separable (MDS) Code ◮ The repair bandwidth is 5 ( n − 1 ) · ℓ r Example: n = 8, r = 3 1 2 3 4 5 6 7 8 ℓ Repair Bandwidth: ( n − r ) · ℓ 4 ′ 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Minimum Storage Regenerating (MSR) Codes • Storage system storing ℓ ( n − r ) data symbols on n nodes using [ n , n − r ; ℓ ] code ⇒ r “redundancy nodes”; Each node stores ℓ symbols • Minimum Storage Regenerating (MSR) Codes ◮ Maximum Distance Separable (MDS) Code ◮ The repair bandwidth is 5 ( n − 1 ) · ℓ r Example: n = 8, r = 3 1 2 3 4 5 6 7 8 ℓ r Repair Bandwidth: ( n − 1 ) · ℓ r 4 ′ 5 Dimakis, Alexandros G., et al. "Network coding for distributed storage systems." IEEE transactions on information theory 56.9 (2010): 4539-4551. Lukas Holzbaur (TUM) 3

Ye-Barg MSR Codes 6 Construction: Ye-Barg MSR Codes, Construction 1; Ye et al. 6 Let C ⊂ F q be an [ n , n − r ; ℓ ] array code over F q , where q ≥ rn . Let { β i , j } i ∈ [ n ] , j ∈ [ r ] be a set of rn distinct elements of F q . Then each codeword is an array with ℓ = r n rows and n columns, where the a -th row fulfills the parity check equations   1 1 . . . 1 β 1 , a 1 β 2 , a 2 . . . β n , a n H ( a ) =    ,   . . . . . .   . . .  β r − 1 1 , a 1 β r − 1 2 , a 2 . . . β r − 1 n , a n for a ∈ [ 0 , ℓ − 1 ] and a = � n i = 1 a i r i − 1 . 6 Ye, Min, and Alexander Barg. "Explicit constructions of high-rate MDS array codes with optimal repair bandwidth." IEEE Transactions on Information Theory 63.4 (2017): 2001-2014. Lukas Holzbaur (TUM) 4

Ye-Barg MSR Codes 6 Construction: Ye-Barg MSR Codes, Construction 1; Ye et al. 6 Let C ⊂ F q be an [ n , n − r ; ℓ ] array code over F q , where q ≥ rn . Let { β i , j } i ∈ [ n ] , j ∈ [ r ] be a set of rn distinct elements of F q . Then each codeword is an array with ℓ = r n rows and n columns, where the a -th row fulfills the parity check equations   1 1 . . . 1 β 1 , a 1 β 2 , a 2 . . . β n , a n H ( a ) =    ,   . . . . . .   . . .  β r − 1 1 , a 1 β r − 1 2 , a 2 . . . β r − 1 n , a n for a ∈ [ 0 , ℓ − 1 ] and a = � n i = 1 a i r i − 1 . 6 Ye, Min, and Alexander Barg. "Explicit constructions of high-rate MDS array codes with optimal repair bandwidth." IEEE Transactions on Information Theory 63.4 (2017): 2001-2014. Lukas Holzbaur (TUM) 4

Ye-Barg MSR Codes 6 Parity check matrix of row a   1 1 . . . 1 β 1 , a 1 β 2 , a 2 . . . β n , a n H ( a ) =     . . . . . .   . . .   β r − 1 1 , a 1 β r − 1 2 , a 2 . . . β r − 1 n , a n 6 Ye, Min, and Alexander Barg. "Explicit constructions of high-rate MDS array codes with optimal repair bandwidth." IEEE Transactions on Information Theory 63.4 (2017): 2001-2014. Lukas Holzbaur (TUM) 5