The UCLA Grid Portal (UGP) and the University of California Grid - PowerPoint PPT Presentation

The UCLA Grid Portal (UGP) and the University of California Grid (UC Grid) Joan Slottow, Prakashan Korambath, Kejian Jin UCLA Academic Technology Services 1

The University of California • UC Office of the President (UCOP) • 10 autonomous campuses • Medical Centers • Lawrence Berkeley National Laboratory 2 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2

UC-Wide IT Policy and Planning • IT Leadership Council (ITLC) – UC-wide IT Governing Body – Membership = CIOs and IT leaders from UC entities. – Strategic planning and priorities • IT Guidance Committee (ITGC) charged with recommending strategic directions to guide IT investments – Issued Report December 2007 3 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3

ITGC Report • IT infrastructure is critical – UC Grid is a key enabling technology • Scalable data center services • Sustainable funding models • Collaboration • Networking (CENIC) 4 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 4

UC must operate as one and leverage the strengths of the 10 campuses • NEED Multi-campus initiatives and coordination • UC Grid has to be implemented in collaboration at the campus level 5 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 5

Grid • Multiple, independent, geographically dispersed compute resources -- networked together -- used cooperatively • Autonomous units under local control • Named after the electricity grid 6 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 6

UC Grid • Started at UCLA and currently implemented at UCLA, UCOP, UCSB, and UCI 7 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 7

8 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 8

UCLA Grid Portal (UGP) • Joins computational clusters into a Grid. • Under development at UCLA since 2002. • Built on top of open source components. • A Globus Incubator Project 9 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 9

Motivation ??? Hoffman Cluster Professor 1’s Cluster Professor 2’s Hosted Hosted Cluster Cluster1 Cluster2 Professor3’s Hosted Hosted Cluster Cluster3 Cluster4 Myrinet Professor4’s Hosted Cluster Cluster5 MAC OS Professor5’s Cluster InfiniBand 10 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 0

Motivation • Common interface • Single sign-on • Work with/see all resources at once • Transfer files • Portal makes it easy for users to use 11 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 1

Single Campus Architecture Head node Cluster I Credential mapped to local ID Grid Appliance Grid Portal https Uniform browser Head node based interface Cluster II Credential mapped to local ID MyProxy Server Storage Server ION Visualization Server Grid Appliance 12 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 2

UC Grid Architecture UC Grid Portal UCLA Grid Portal UCSB Grid Portal UCI Grid Portal others C1 C2 … C3 C4 … C5 C6 … …… • User always applies from Campus Portal • One Certificate Authority(CA) throughout UC Grid • User CA is automatically created and pushed to myproxy server • Simple account creation process (approval/denial workflow) • Once the admin approves, the user is able to login to two portals instantly • Enabled by Register Web Service and Sync Web Service in UC Grid Portal 13 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 3

The UC Grid Portal • Each user of a Campus Grid is automatically able to use the UC Grid Portal • From the UC Grid Portal, users can: – work with clusters from all the campuses – transfer files across the campuses. – Use pooled resources across the campuses. 14 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 4

UGP Services 15 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 5

Resource Discovery 16 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 6

Data Manager 17 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 7

File Transfer 18 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 8

Visualization 19 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 1 9

Batch Jobs Generic 20 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 0

Batch Jobs Applications and pool Jobs 21 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 1

Pool Job Submission • A way to share/use unused cycles. • UGP: – Chooses the target cluster for the job. – Submits the job to run under a guest login id there. – Stages the input files. • UGP sends jobs to local scheduler. Decision for sharing resources made by the cluster owner. 22 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 2

Application Input Forms 23 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 3

Data Manager and Batch Jobs to Other Grids 24 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 4

Interactive GUI Applications 25 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 5

Xterm 26 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 6

UC System Wide Mandate • The CIOs of the campuses have agreed 27 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 7

Has to be Carried Out on each of the 10 Campuses • Implementation/maintenance/development has to happen on each of 10 campuses • People doing the work need: – a connection with the researchers – time to do the work – resources • ATS at UCLA has the leadership role. 28 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 8

Campus have very Distributed Research IT Infrastructures • Individual researchers acquire and maintain their own HPC resources. • Most campuses do not have: – central research computing organizations – HPC facilities/resources for the campus • This is not sustainable • Required: – Data centers, HP Networks, support staffs 29 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2 9

• UCLA is unique in that it has ATS – Emphasis on research computing • HPC, Scientific Visualization, Statistics • Close relationship with researchers – Started as the campus computer center, but: • Remained throughout the era of decentralization • Was on the forefront of: cluster building, cluster hosting, grid • Has the ability to effect changes • UGP Development 3 30 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 0

Who to work with on the Campuses 31 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 1

UC Grid Implementers • Monthly Meetings (Skype, ichat, VNC) • Discussion/Mailing List • ATS Provides Support for implementers 32 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 2

The UC Grid Provides for the Sharing of System-wide Resources • The majority of campuses are planning data center expansions • Research computing needs are growing • The grid provides a way to use virtual (networked) resources no matter where they are. 33 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 3

January 2008 -- UCLA -> Shared Cluster Hosting Model New Hoffman2 Cluster 34 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 4

Motivation for the Shared Cluster Hosting Model • Manpower to administer multiple hosted clusters >> to administer 1 cluster • > 50% cycles are idle • Power use by idle clusters is not significantly lower than for full operation • The ability for others to use your idle cycles is the price you pay to join the shared cluster 35 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 5

The UCLA Shared Cluster Model scales to the UC • Cost of data center space/location • Cost of electricity/location • Redundant and wasted capacity • Great need Sharing must be attractive to the resource owners. Only they can make the decision to share. 36 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 6

February 2008 -- UC Research Cyberinfrastructure Pilot -- Draft Document Phase I: Provide computing resources via two moderate institutional clusters housed at LBNL and SDSC Network-based services via UC Grid Phase II: Expand and develop 37 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 7

Learn more about UGP • UGP documentation, download and wiki http://www.ucgrid.org 38 CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3 8