CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD - PowerPoint PPT Presentation

CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD APPLICATIONS Zach Hill & Marty Humphrey Dept. of Computer Science, University of Virginia zjh5f@cs.virginia.edu

A Cloud Application Worker Worker Worker Front-End User Worker Requests Object/ Blob Queue Table Service Service Service 2 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Many clouds, many code versions Worker Storage Services Worker Front-End Blobs Worker Tables Queues Front-End Storage Services 3 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Single code version, many clouds Worker Storage CSAL Services Worker CSAL Front-End CSAL Blobs Worker Tables CSAL Queues Front-End Storage CSAL Services 4 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

CSAL Overview Application CSAL TableStore BlobStore QueueStore createTable createContainer createQueue deleteTable listContainers deleteQueue Insert deleteContainer getMessage Metadata Update getBlob putMessage Delete Manager putBlob peekMessage Query deleteBlob deleteMessage getItem Service Blob Namespace Table Namespace Q Namespace Manager SimpleDB Azure Queue S3 Plugin … Plugin Plugin Metadata Store table service(s) 5 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

CSAL Namespaces  One namespace for each abstraction type  Metadata only for containers  Service endpoint, identifier, user credentials  Each abstraction has an independent metadata store  Metadata caching  Container ops are not very common  If data is stale, simply re-fetch and retry 6 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

CSAL Namespaces 1. Call getBlob(“X”, “foo”); X Service1 5. Access “foo” in foo CSAL “X” in Service1 2. Lookup “X” BlobStore metadata – cache first Plugin Service2 Metadata 4. Use metadata to Y Manager Cache determine plugin to use 3. Retrieve from table if not in cache Service3 X (“X”, http://service1.com, testusr, testKey,…) 7 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

CSAL Implementation  Client-side java library  i.e. BlobStore.getBlob(“Container”,”foo”);  Metadata backing store in the cloud  Currently supports Azure & AWS storage  Both SOAP and REST 8 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Performance of CSAL  Adding software layers isn’t free  Compare CSAL to Azure’s and Amazon’s SDK APIs  Set of micro-benchmarks to test operation latency  Container Ops and Data Ops  Expect a slowdown for container ops due to metadata 9 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Performance – Container Ops CSAL in AWS CSAL in Windows Azure 1.6 1.6 1.4 1.4 Median Operation Time in Seconds Median Operation Time in Seconds 1.2 1.2 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 Create Container Create Table Queue Create Create Container Create Table Queue Create CSAL Total Time Native API CSAL Total Time Native API Core Op Time Metadata Op Time Core Op Time Metadata Op Time Note: Error Bars indicate 1 Standard Deviation 10 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Performance – Data Ops CSAL in AWS CSAL in Windows Azure 0.4 0.4 Median Operation Time in Seconds Median Operation Time in Seconds 0.35 0.35 0.3 0.3 0.25 0.25 0.2 0.2 0.15 0.15 0.1 0.1 0.05 0.05 0 0 CSAL Total Time Native API CSAL Total Time Native API Note: Error Bars indicate 1 Standard Deviation 11 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

What about Standards?  Standards Efforts  OCCI  OVF  Standards take time to develop and are resisted by vendors  Multi-cloud APIs  SimpleCloud, jClouds, DeltaCloud, LibCloud  SAGA 12 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Future work  What if Cloud X doesn’t have tables/blobs/queues?  Map one abstraction to other (i.e. filesystem)  3 rd party services: Hbase, HyperTable, Cassandra…  Placing, replicating, and migrating data in real-time for performance and/or cost  Real-world applications such as multi-cloud MR 13 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Summary  Application lock-in and portability are problems in clouds  Standards are great, but don’t hold your breath just yet  CSAL provides storage abstractions to make the application code itself portable with little performance impact for common data operations 14 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010

Questions? zjh5f@cs.virginia.edu 15 Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010