Grid computing: yesterday, today and tomorrow? Dr. Fabrizio - PowerPoint PPT Presentation

Grid computing: yesterday, today and tomorrow? Dr. Fabrizio Gagliardi EMEA Director External Research Microsoft Research Cracow Grid Workshop 2008 Cracow, October 14 th

Outline Yesterday and today: • Achievements in the area of e-Infrastructures and Grid computing Examples beyond e-Science • Issues : Complexity, Cost, Security, Standards • The future: • • Cloud Computing, Virtualisation, Data Centers, Software as a Service, Multi-core architectures, Green IT Conclusions

The European Commission strategy for e-Science Mario Campolargo, EC, DG INFSOM, Director of Directorate F: Emerging Technologies and Infrastructures http://cordis.europa.eu/fp7/ict/programme/events-20070524_en.html 10/14/2008 CGW'08, Cracow, Poland 4

e-Infrastructure achievements: Research Networks 10/14/2008 CGW'08, Cracow, Poland 5

e-Infrastructure HPC achievements: EGEE and DEISA No. Cores 80000 70000 No. Cores 60000 50000 100000 40000 30000 50000 20000 10000 0 … Au … De … … Au … De … … Au … De … … Au … De … … 0 Apr Apr Apr Apr Apr Apr Jul Okt Jan Apr Jul Okt Jan Apr Jul Okt Jan Apr Jul Okt Jan Apr 04 04 04 05 05 05 05 06 06 06 06 07 07 07 07 08 08 10/14/2008 CGW'08, Cracow, Poland 6

e-Infrastructure HPC next steps: EGI and PRACE tier 0 European Ecosystem 10/14/2008 CGW'08, Cracow, Poland 7

In summary: Grid achievements for e-Science • Grid for e-Science: mainly a success story! – Several maturing Grid Middleware stacks – Many HPC applications using the Grid • Some (HEP, Bio) in production use • Some still in testing phase: more effort required to make the Grid their day-to-day workhorse – e-Health applications also part of the Grid – Some industrial applications: • Early deployment mainly in different EC projects 10/14/2008 CGW'08, Cracow, Poland 8

Achieving global e-Science Session convener Participant Courtesy EGEE Project Office CGW'08, Cracow, Poland 10/14/2008 9

Grid achievements beyond e-Science • Grid beyond e-Science? – Slower adoption: prefer different environments, tools and have different TCOs • Intra grids, internal dedicated clusters , cloud computing – e-Business applications • Finance, ERP, SMEs and Banking! • New economic and business models – Industrial applications • Energy, Automotive, Aerospace, Pharmaceutical industry, Telecom – e-Government applications • Earth Observation, Civil protection: • e.g. The Cyclops project CGW'08, Cracow, Poland 10/14/2008 10

Examples beyond e-Science CitiGroup ( Citigroup Inc., operating as Citi, is a major American financial services company based in New York City) adopted Grid computing http://www.americanbanker.com/usb_article.html?id=20080825IXTFW8BS - Citi chose Platform Computing's Symphony grid product to consolidate its computing assets into a single resource pool with increased utilization - At Citi, since the grid was implemented, individual business units are charged for the processing power they use, creating a shared services environment -Citi is now using near 20,000 CPUs and there are periods of the day where the utilization rate is 100 percent -Citi is planning of using the cloud in cases their data centers do not suffice ( overflow model or cooperative data centers) 10/14/2008 CGW'08, Cracow, Poland 11

Grid achievements in industry • IT Industry demonstrated interest in becoming an Grid infrastructure provider and/or user (intra-grids): – On-demand infrastructures: • Cloud and Elastic computing, pay as you go… • Data centers: Data getting more and more attention – Service hosting: outsourced integrated services • Software as a Service (SaaS) (e.g. Salesfoce.com services) – Virtualisation being exploited in Cloud and Elastic computing (e.g. Amazon EC2 virtual instances) • “Pre - commercial procurement” – Research-industry collaboration in Europe to achieve new leading-edge products • Example: PRACE building a PetaFlop Supercomputing Centre in Europe 10/14/2008 CGW'08, Cracow, Poland 12

The HPC view from …the clouds! Courtesy Peter Coffee, Salesforce.com 10/14/2008 CGW'08, Cracow, Poland 13

Today and the future: Green IT, pay per CPU/GB virtualisation and/or HPC in every lab? • Computer and data centers in energy and environmental favorable locations are becoming important • Elastic computing, Computing on the Cloud, Data Centers and Service Hosting - Software as a Service, are becoming the new emerging solutions for HPC applications • Many-multi-core and CPU accelerators are promising potential breakthroughs • Green IT initiatives: • The Green Grid (www.thegreengrid.org) consortium ( AMD, APC, Dell, HP, IBM, Intel, Microsoft, Rackable Systems, Sun Microsystems and Vmware) • IBM Project Big Green ( a $1 billion investment to dramatically increase the efficiency of IBM products) and other IT industry initiatives try to address current HPC limits in energy and environmental impact requirements 10/14/2008 CGW'08, Cracow, Poland 14

Today and the future: Cloud computing and storage on demand • Cloud Computing: http://en.wikipedia.org/wiki/Cloud_computing • Amazon, IBM, Google, Microsoft, Sun, Yahoo , major „Cloud Platform‟ potential providers • Operating compute and storage facilities around the world • Have developed middleware technologies for resource sharing and software services • First services already operational - Examples: • Amazon Elastic Computing Cloud (EC2) -Simple Storage Service (S3) • Google Apps www.google.com/a • Sun Network.com www.network.com (1$/CPU hour, no contract cost) • IBM Grid solutions (www.ibm.com/grid) • GoGrid – a division of ServePath company (www.gogrid.com) Beta released (pay-as-you-go and pre-paid plans, server manageability) 10/14/2008 CGW'08, Cracow, Poland 15

EGEE cost estimation (1/2) Capital Expenditures (CAPEX): a. Hardware costs: 80.000 CPUs ~ in the order of 120M Euros (80-160M) Depreciating the infrastructure in 4 years:30Meuros per year (20M to 40M) b. Cooling and power installations (supposing existing housing facilities available) 25% of H/W costs: 30M, depreciated over 5 years: 6M Euros Total: ~ 36M Euros / year (26M-46M) 10/14/2008 GridKA 2008, Karlsruhe 16 Slide Courtesy of Fotis Karayannis

EGEE cost estimation (2/2) Operational Expenditures (OPEX): a. 20 MEuros per year for all EGEE costs (including site administration, operations, middleware etc.) b. Electricity ~10% of h/w costs: 12M Euros per year (other calculations lead to similar results) c. Internet connectivity: Supposing no connectivity costs (existing over-provisioned NREN connectivity) *If other model is used (to construct the service from scratch), then network costs should be taken into account Total 32M / year CAPEX+OPEX= 68M per year (58-78M) 10/14/2008 GridKA 2008, Karlsruhe 17 Slide Courtesy of Fotis Karayannis

EGEE if performed with Amazon EC2 and S3 In the order of ~50M Euros, probably more cost effective of EGEE actual cost, depending on the promotion of the EC2/S3 service 10/14/2008 GridKA 2008, Karlsruhe 18 Slide Courtesy of Bob Jones

Cloud mature enough for big sciences? Probably not yet, as not designed for them; Does not support complex Scenarios: “S3 lacks in terms of flexible access control and support for delegation and auditing, and it makes implicit trust assumptions” http://www.symmetrymagazine.org/breaking/2008/05/23/are-commercial-computing-clouds- ready-for-high-energy-physics/ http://www.csee.usf.edu/~anda/papers/dadc108-palankar.pdf CGW'08, Cracow, Poland 10/14/2008 19

The future: “ To Distribute or Not To Distribute” • In the late 90s, petaflops were considered very hard and at least 20 years off … • while grids were supposed to happen right way • After 10 years (around now) petaflos are “real close” but there‟s still no “global grid” • What happened:  It was easier to put together massive clusters than to • Prof. Satoshi get people to agree about how to share their resources Matsuoka, TITech  For tightly coupled HPC applications, tightly coupled • machines are still necessary Keynote at Mardi  Grids are inherently suited for loosely coupled apps Gras Conference or enabling access to machines and/or data Baton Rouge, 31 Jan 2008 • With Gilder's Law*, bandwidth to the compute resources will promote thin client approach * “ Bandwidth grows at least three times faster than computer power." This means that if computer power doubles every eighteen months (per Moore's Law), then communications power doubles every six months • Example: Tsubame machine in Tokyo 10/14/2008 CGW'08, Cracow, Poland 20

Multi-core architectures • Computer CPUs have adopted multi-core architectures with increasing number of cores – 2-4 cores in PCs and laptops – 8-32 cores in servers, 64-80 cores under development – Intel announced a 6 core Xeon • The trend is driven by many factors: – Power consumption, heat dissipation, energy cost, availability of high bandwidth computing at lower cost, ecological impact • The entire software ecosystem will need to adapt including related applications 10/14/2008 CGW'08, Cracow, Poland 21