V I N E U R S E I H T Y T O H F G R E U D B I N On The Scalability of Storage Sub-System Back-end Network Yan Li, Roland Ibbett, Nigel Topham and Tim Courtney ‡ School of Informatics University of Edinburgh ‡ Xyratex, UK Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Project Motivation 1 O H F G R E U D B I Project Motivation N • Theoretically the more disks used in a disk array, the higher the degree of parallesim, so leading to larger the performance potential performance benefits. • However, in a real system there is a limitation on the scale of RAID systems due to the limitation of interconnection network. • The more disks are added to the system, the higher the contention for the shared media. • When the number of disks and cache size in a RAID system reaches a certain threshold, there will be no further gain in performance by adding more disk or cache due to the saturation of the back-end network. Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Project Goal 2 O H F G R E U D B I Project Goal N • Investigate the capacity of interconnection networks in terms of the numbers of disks that can be included in one chain. In particular, Fibre Channel (FC) SBOD is chosen as the interconnection network. – Give a certain number of disks and cache size, how much bandwidth is necessary to support them to get the maximum performance? – Likewise, how many disks (and cache) can be connected to a 2GFC (4GFC) port? Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Analytical Models 3 O H F G R E U D B I N • Assumptions: – The request size of random access workload is equal to the size of stripe unit. – The access address of each request is aligned to the stripe unit boundary. – The capacity of the RAID system keeps fixed for all the study. – The queue length of disk is 1, ie., no disk command waits in the disk for service. – The FC SBOD is chosen as the research subject. • Disk command transmission time Tx = S B + overhead . . S S • Disk command execution time Td = + + T seek . B port B media • S size of stripe unit; B network bandwidth; Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Analytical Models 4 O H F G R E U D B I Large Sequential Access N T v T x T d D T x Time = D T x + T v T d • For sequential access workload, a large volume command is divided into D disk commands, so that the response time for that command Tv = DTx + Td . • Throughput is the major performance metric: Throughput = K K Tv = DTx + Td. Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Analytical Models 5 O H F G R E U D B I Small Random Access N T v T x T d T v T x T d � T x + T d (D−1)T (D−1)T x x Time Time > = 5) = 3) T d (D (D−1)T < T d (D (D−1)T x x = + T v = DT T v T x T d x 8 > < ( D − 1) ∗ T x < T d Tv = ( D − 1) ∗ T x > = T d D ∗ T x > : • For random access workload, IOPS is the major performance metric, D ( D − 1) ∗ T x < T d T x + T d IOPS = 1 T d ( D − 1) ∗ T x > = T d T x ∗ D ∗ T x Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Analytical Models 6 O H F G R E U D B I Analytical Results (no cache) N 7 16 x 10 12000 14 10000 12 Seq Read, K=128KBytes 8000 throughput (Bytes/s) 10 IOPS 6000 8 RANOM, S=16KBytes 4000 6 2000 4 2 0 0 20 40 60 80 0 20 40 60 80 100 number of disks number of disks Sequential Access Random Access Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Analytical Models 7 O H F G R E U D B I General Model N • B = F ( D, C, S, L, P ) = Num d ∗ S + overhead B , the bandwidth required to achieve the maximum performance with D disks and C cache in system. D : number of disks C : cache size S : size of stripe unit L : workload characteristic P : cache destage threshold, P=0 for our study Num d : number of disk commands send to disk per second Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Simulation Results 8 O H F G R E U D B I Systems without Cache ( S = 16 KB ) N 80 1 0.8 60 Max SPC−1 BSU utilization 0.6 40 Disk Uti RAID5 RAID5 0.4 Disk Uti RAID6 RAID6 Net Uti RAID5 20 Net Uti RAID6 0.2 0 0 0 20 40 60 80 100 0 20 40 60 80 100 number of disks number of disks Port Bandwidth = 2.125 Gbps 150 1 0.8 Max SPC−1 BSU 100 utilization 0.6 Disk Uti RAID5 0.4 RAID5 Disk Uti RAID6 50 Net Uti RAID5 RAID6 Net Uti RAID6 0.2 0 0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 number of disks number of disks Port Bandwidth = 4.25 Gbps Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Simulation Results 9 O H F G R E U D B I Systems with Cache ( S = 32 KB ) N (a) (b) 200 7 RAID5 RAID5 6 RAID6 RAID6 bandwidth (Gbps) 150 Max SPC−1 BSU 5 4 100 3 2 50 1 0 0 0 20 40 60 0 20 40 60 number of disks number of disks (d) (c) 1 0.75 RAID5 RAID5 RAID6 RAID6 0.95 read miss rate disk utilization 0.9 0.7 0.85 0.8 0.65 0 20 40 60 0 20 40 60 number of disks number of disks Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Summary 10 O H F G R E U D B I Summary N • When this is no cache, a 2G FC port is able to support up to 46 disks for RAID5 and 53 disks for RAID6 (size of stripe unit = 16kBytes). • With enough cache, a 2G FC port is able to support up to 18 disks under OLTP like workload. (size of stripe unit = 32KBytes). • When there is enough cache, the bandwidth required to support a certain number of disks is fixed. It is irrelevant with protection level and cache size. Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
V I N E U R S E I H T Y T Future Work 11 O H F G R E U D B I Future Work N • Study the scalability of back-end network when the size of stripe unit is 16k and the system performance. • Study the network bandwidth requirement when there is cache coherency between two controllers. Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk
Recommend
More recommend
