apache libcloud
play

Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, - PowerPoint PPT Presentation

Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, 2010 Tuesday, June 1, 2010 About Me Chief Architect at Cloudkick Developer on Apache HTTP Server Former VP Infrastructure @ ASF Libcloud developer! Tuesday, June


  1. Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, 2010 Tuesday, June 1, 2010

  2. About Me • Chief Architect at Cloudkick • Developer on Apache HTTP Server • Former VP Infrastructure @ ASF • Libcloud developer! Tuesday, June 1, 2010

  3. About the Cloud Tuesday, June 1, 2010

  4. About the Cloud • Awesome. Tuesday, June 1, 2010

  5. About the Cloud • Awesome. • Really. Tuesday, June 1, 2010

  6. About the Cloud • Awesome. • Really. • Maybe not always. Tuesday, June 1, 2010

  7. About the Cloud • Awesome. • Really. • Maybe not always. • But mostly. Tuesday, June 1, 2010

  8. Services (SaaS) • GMail, Google Docs, etc • Most any website cloud be called SaaS. • Cloudkick Tuesday, June 1, 2010

  9. Platforms (PaaS) • Google AppEngine • Heroku (Rails as PaaS) • SalesForce.com / VMForce Tuesday, June 1, 2010

  10. Storage • Amazon S3 • Rackspace CloudFiles • Google Storage for Developers Tuesday, June 1, 2010

  11. Compute • Amazon EC2 • Rackspace Cloud • Linode • GoGrid • Voxel • And many more! Tuesday, June 1, 2010

  12. I want a server. Tuesday, June 1, 2010

  13. I want a server: right now. Tuesday, June 1, 2010

  14. Enter Libcloud from libcloud.types import Provider from libcloud.providers import get_driver rs = get_driver(Provider.RACKSPACE)('rackspace-apikey') rs.create_node('serverA') Tuesday, June 1, 2010

  15. About Libcloud • Started in the summer of 2009 • Easy to use. • Portable. • Pure Python (proposed ports to others) • Socket & HTTP interfaces exist today! Tuesday, June 1, 2010

  16. Why? • API Styles: • Amazon: XML + Custom HMAC Auth • Rackspace: JSON + Auth Tickets • SoftLayer: XML RPC + User / Password Tuesday, June 1, 2010

  17. Libcloud Today • In the Apache Incubator • 15 Providers: • Dreamhost, Amazon EC2, Enomaly ECP , Eucalyptus, GoGrid, IBM Developer Cloud, Linode, OpenNebula, Slicehost, SoftLayer, Rackspace, RimuHosting, Terramark, VMWare vCloud, Voxel, VPS.net Tuesday, June 1, 2010

  18. Libcloud APIs • Originally 6 Core APIs • List Nodes • List Images • List Sizes • Create / Destroy / Reboot Node Tuesday, June 1, 2010

  19. list_nodes foo = d.list_nodes() for node in foo: print node.id print node.public_ip Tuesday, June 1, 2010

  20. list_images images = d.list_images() ubuntu = [i for i in images if i.name.find('Ubuntu') != -1] print ubuntu[0].id print ubuntu[0].name Tuesday, June 1, 2010

  21. list_sizes sizes = d.list_sizes() print sizes[0].id print sizes[0].ram print sizes[0].disk print sizes[0].price Tuesday, June 1, 2010

  22. Create Node images = d.list_images() sizes = d.list_sizes() print d.create_node(name="test22", image=images[0], size=sizes[0]) Tuesday, June 1, 2010

  23. Reboot/Destroy d.reboot(nodeA) d.destroy(nodeB) Tuesday, June 1, 2010

  24. Locations! loc = d.list_locations() print loc[0].name print loc[0].country Tuesday, June 1, 2010

  25. Extended APIs • Providers inconsistent about services. • Have a “ex_” prefix, documented per- driver. • Amazon Security Groups: • amz.create_node(‘foo’, ex_securitygroup=‘groupA’) Tuesday, June 1, 2010

  26. Getting Started • easy_install apache-libcloud Tuesday, June 1, 2010

  27. Get your Provider Info • Amazon: • http://aws-portal.amazon.com/gp/aws/ developer/account/index.html? action=access-key • Rackspace: • https://manage.rackspacecloud.com/ APIAccess.do Tuesday, June 1, 2010

  28. List your Machines from libcloud.types import Provider from libcloud.providers import get_driver d = get_driver(Provider.RACKSPACE)("xxxxxxx") nodes = d.list_nodes() for node in nodes: print "id: %s name: %s public_ips: %s" % (node.id, node.name, node.public_ip) Tuesday, June 1, 2010

  29. Cheapest 4 gig node outside the US possible = [] for d in drivers: loc = filter(lambda x: x.country != 'US', d.list_locations()) for l in loc: sizes = filter(lambda x: x.ram >= 4096, d.list_sizes(l)) for s in sizes: possible.append({'size': s, 'location': l, 'driver': d}) best = sorted(possible, lambda x,y: x['size'].price < y['size'].price)[0] print best Tuesday, June 1, 2010

  30. Integrating with Fabric env.hosts = [x.public_ip[0] for x in d.list_nodes()] def hostname(): run('hostname') Tuesday, June 1, 2010

  31. $ fab hostname [173.45.245.33] run: hostname [173.45.245.33] out: lctest3.k1k.me [173.45.245.32] run: hostname [173.45.245.32] out: lctest2.k1k.me Done. Disconnecting from 173.45.245.33... done. Disconnecting from 173.45.245.32... done. Tuesday, June 1, 2010

  32. One more thing! • deploy_node • Calls create_node • Consistent initial bootstrapping of machines. • SSH Keys • Configuration Management Tuesday, June 1, 2010

  33. Installing Puppet skey = SSHKeyDeployment(key) sd = ScriptDeployment("apt-get install -y puppet") msd = MultiStepDeployment([skey, sd]) node = d.deploy_node(name="lc-test", deploy=msd) Tuesday, June 1, 2010

  34. Up next for Libcloud Tuesday, June 1, 2010

  35. Image Formats • Hazy world between Operating System, Configuration Management and the Sysadmin. • People stick with Config Management, because dealing with base Images is painful today Tuesday, June 1, 2010

  36. Existing standards • Amazon AMI • VMWare Open Virtualization Format (OVF) Tuesday, June 1, 2010

  37. Hosting Provider Side • Technical challenges • Most commercial hosting is Xen based • Most hosting companies aren’t giant tech companies Tuesday, June 1, 2010

  38. User Side • Building Images is complicated. • Versioning Sucks • Time sink uploading Tuesday, June 1, 2010

  39. Proposed Image Format • Based on Cloudlets Project • JSON Metadata in single file • Filesystem in a tarball • Versioned in DVCS • Includes building server-side support infrastructure for Hosting providers! • More details on mailing list Tuesday, June 1, 2010

  40. Multiple Languages • Hundreds of emails exchanged on the mailing list. • Interest, something will happen. • If interested, join the lists, start hacking on code! Tuesday, June 1, 2010

  41. Contributing! • Open Community just as important as open code -- everything on list or IRC. • Hosting providers: Get your driver in! • Hackers: Make cool tools! • Sysadmins: Manage your infrastructure. Tuesday, June 1, 2010

  42. Related Projects • JClouds • Java • Apache Deltacloud • Ruby, started by Redhat, just joined Apache Incubator in May • Fog • Ruby Tuesday, June 1, 2010

  43. Questions? • Apache Libcloud: http://libcloud.org/ #libcloud on Freenode IRC • Slides online: http://paul.querna.org/slides Tuesday, June 1, 2010

Recommend


More recommend