Open Science Grid Ruth Pordes Fermilab V2 3/27/07 ISGC2007: OSG - PowerPoint PPT Presentation

Open Science Grid Ruth Pordes Fermilab V2 3/27/07 ISGC2007: OSG 1

First of All • I am in Taipai in spirit today but not able to be there in person. • A big thank you to Simon and Vicky for enabling me to give my talk remotely • And thank you to Stephen, my husband, is with my 86 year old father who arrived from England today - one of the several family reasons that kept me in the US this week. ISGC is now an annual event where truly the ISGC is now an annual event where truly the whole earth aspects of science on the whole earth aspects of science on the distributed infrastructure is clearly shown. distributed infrastructure is clearly shown. 3/27/07 ISGC2007: OSG 2

Outline • Status of OSG • Architectural Model • OSG & LHC • Some Sample Activities Note: OSG is International in scope International in scope with its direct goal to deliver a Virtual Distributed Facility Virtual Distributed Facility in the US. 3/27/07 ISGC2007: OSG 3

What is the Open Science Grid? People and Organizations as members & partners of a Consortium Consortium People and Organizations • • with Common Goals Common Goals to support Computer Based Research over wide and local areas of distribution.  Includes Facilities, Experiments, Computer Scientists, Grid organizations etc. − Global in Reach: Currently include Taiwan, South Africa, South Global in Reach: Currently include Taiwan, South Africa, South − America, Australia, UK America, Australia, UK Contributor to the vision of Cyberinfrastructure Cyberinfrastructure “.. a cultural Contributor • • community that .. enables distributed knowledge communities .. collaborate and communicate across disciplines, distances and cultures.”  Dr Arden Bement, Director of National Science Foundation in “Cyberinfrastructure Vision for 21st Century Science. − Global in Scope: NSF actively discussing how to increasingly Global in Scope: NSF actively discussing how to increasingly − sponsor international partnerships. sponsor international partnerships. “Each participant retains its separate legal status and the consortium's control • over each participant is generally limited to activities involving the joint endeavor”  From Wikipedia definition of Consortium 3/27/07 ISGC2007: OSG 4

And the OSG Project The OSG Project has been awarded ~$6m/year for 5 years to • deliver to the goals of the Consortium. Support across 6 program offices of the DOE Office of Science • SciDAC-2 program and the National Science Foundation. − International components for International Science Grid This International components for International Science Grid This − Week online newsletter, and outreach and educational Week online newsletter, and outreach and educational activities (currently to Scandinavia - IceCube - and South activities (currently to Scandinavia - IceCube - and South Africa - ATLAS). Africa - ATLAS). Support for 30 OSG staff at 16 DOE Laboratories and Universities: • Boston University Brookhaven National Laboratory California Institute of Techology Columbia University Cornell University Fermi National Accelerator Laboratory Indiana University Lawrence Berkeley National Laboratory Rennaisance Computing Institute Stanford Linear Accelerator Center University of California, San Diego University of Wisconsin, Madison University of Florida University of Chicago/Argonne National Laboratory 3/27/07 ISGC2007: OSG 5 University of Iowa

OSG Organization 3/27/07 ISGC2007: OSG 6

OSG Project Scope - independent of national boundaries Maintain and Operate a Secure Production Quality Common Secure Production Quality Common • Distributed Infrastructure providing access to & sharing of access to & sharing of OSG Distributed Infrastructure Consortium members and partners computational and storage computational and storage resources over production and research networks. resources A reference distribution of common technologies for high A reference distribution of common technologies for high • • throughput distributed computing -- the Virtual Data Toolkit (VDT) throughput distributed computing Virtual Data Toolkit (VDT) -- whose components are requested by OSG members and partners and which is built, tested, integrated, packaged and supported. − External projects that develop the software and/or we rely on and/or we deliver to are part of the OSG management structure. Support for Distributed Communities -- of resource owners, science Support for Distributed Communities • • and research users, educators and students, technology developers - - of both existing and new OSG members to to use and evolve the use and evolve the system. system. − Including a multi-site integration testbed for bringing new software and services into production use. 3/27/07 ISGC2007: OSG 7

External Projects: Partners: • Autralian Partnerships for Advanced Center for Enabling Distributed • Computing (APAC) Petascale Science (CEDPS) • Data Intensive Science University Community Driven Improvement • Network (DISUN) of Globus Software • Enabling Grids for E-SciencE (EGEE) (CDIGS)/Globus • Grid Laboratory of Wisconsin (GLOW) Condor • • Grid Operations Center at Indiana dCache • University Data Intensive Science • • Grid Research and Education Group at University Network(DISUN) Iowa (GROW) Disk Resource Manager (DRM) • • Nordic Data Grid Facility (NDGF) ESNET • • Northwest Indiana Computational Grid Internet2 • • New York State Grid (NYSGrid) LIGO Physics and the • • TeraGrid Information Frontier • Texas Internet Grid for Research and OSG Accounting • Education (TIGRE) OSG Privilege/Authorization • • TWGrid (from Academica Sinica Grid UltraLight/advanced networks • Computing) U.S. LHC + US representation to • • Worldwide LHC Computing Grid the WLCG Collaboration (WLCG) 3/27/07 ISGC2007: OSG 8

Science Community stakeholders - global in reach 3/27/07 ISGC2007: OSG 9

OSG Today 100 Resources across production & integration infrastructures !ncrease in ~15 in last 6 months Using production & research networks Sustaining through OSG submissions: Measuring ~180K CPUhours/day. ~Factor of 50% more (being measured) in last 6 months. ~20,000 cores (from 30 to 4000 cores per cluster) ~6 PB accessible Tapes 27 Virtual Organizations (+ 3 operations ~4 PB Shared Disk VOs) 25% non-physics. 3/27/07 ISGC2007: OSG 10

OSG Architectural Model (1) Importance of Virtual Organization (VO) Courtesy: Frank Wuerthwein 3/27/07 ISGC2007: OSG 11

OSG Architectural Model (2) Vertical and Horizontal Integration User Science Codes and Interfaces Applications VO Middleware Biology HEP Astrophysics Portals, Data and workflow databases etc Data replication etc management etc OSG Release Cache: OSG specific configurations, utilities etc. Infrastructure Virtual Data Toolkit (VDT) core technologies + software needed by stakeholders:many components shared with EGEE Core grid technology distributions: Condor, Globus, Myproxy: shared with TeraGrid and others Existing Farms, Storage, Networks 3/27/07 ISGC2007: OSG 12

OSG Architectural Model (2) Vertical and Horizontal Integration User Science Codes and Interfaces Applications VO Middleware Biology HEP Astrophysics Portals, Data and workflow databases etc Data replication etc management etc OSG Release Cache: OSG specific configurations, utilities etc. Infrastructure Virtual Data Toolkit (VDT) core technologies + software needed by stakeholders:many components shared with EGEE Core grid technology distributions: Condor, Globus, Myproxy: shared with TeraGrid and others Existing Farms, Storage, Networks 3/27/07 ISGC2007: OSG 13

OSG Architectural Model (2) Vertical and Horizontal Integration User Science Codes and Interfaces Applications VO Middleware Biology HEP Astrophysics Portals, Data and workflow databases etc Data replication etc management etc OSG Release Cache: OSG specific configurations, utilities etc. Infrastructure Virtual Data Toolkit (VDT) core technologies + software needed by stakeholders:many components shared with EGEE Core grid technology distributions: Condor, Globus, Myproxy: shared with TeraGrid and others • ≈ Existing Farms, Storage, Networks 3/27/07 ISGC2007: OSG 14

OSG Architectural Model (3) Equivalence of VOs and Resources • Site (Resource) and VO Administrators have equivalent equivalent security, testing, monitoring, Roles. security, testing, monitoring, Roles. • Site (Resource) Owners and VO Managers have equivalent trust, policy, accounting, Responsibilities & equivalent trust, policy, accounting, Responsibilities & Authorities. Authorities. Users share in VO and Site (Resource) Services. Users share • • 3/27/07 ISGC2007: OSG 15

OSG Architectural Model (4) Overlapping & Interfaced Environments Equivalence of Campus, Heterogeneity of Interfaces Community and National and Services - minimally Infrastructures with evidenced by multiplicity overlapping Environments. of versions. 3/27/07 ISGC2007: OSG 16