4/30/2014 1
University of California, San Diego • Prism@UCSD – Science DMZ – PI: P. Papadopulos, co-PI: L. Smarr – 01/01/2013 to 12/31/2014 • CHERuB – 100G campus gateway – PI: M. Norman, co-PI: T. Hutton, V. Polichar – 01/01/2014 to 12/31/2015
UCSD and its environment General Atomics Stem Cell Institute In addition to the 3 main UCSD units: SDSC Salk Institute - General Campus CalIT2 - Medical School - Scripps I.O. there are many other NCMIR Physics Skaggs research organizations Medical School on and around campus. Scripps Institute Venter Institute of Oceanography 4/30/2014 3
Connecting YOU on UCSD Campus with the World By Creating a Big Data Freeway System CHERuB NSF CC-NIE Has Awarded Prism@UCSD Optical Switch Phil Papadopoulos, SDSC, Calit2, PI
Prism@UCSD: A Researcher Defined 10 and 40Gbit/s Campus Scale Data Carrier Project in Brief • high-bandwidth end-to-end optical connections • routed by next generation Arista switches (7504) • c onnects lab “data producers” with SDSC data -intensive computing & storage resources • 10 Terabit/s of aggregate bandwidth, has full bisection similar to in- machine room clusters, but is deployed at a campus scale • builds upon and upgrades the Quartzite "campus-scale network laboratory" NSF MRI (awarded 2006) • adds IPv6 and OpenFlow • existing optical fiber connection to the SDSC is being expanded to 120Gbps as a high-bandwidth bridge to cloud/parallel storage and NSF XSEDE resources
PRISM Puts SDSC’s Big Data Gordon Supercomputer and Data Oasis Storage Into Your Lab 12
PRISM is Connecting CERN’s CMS Experiment To Our Physics Department 80 Gbps PRISM Connection Has Been Made
UCSD is a Tier-2 LHC Data Center: CMS Flow into UCSD Physics Dept. Peaks at 2.4 Gbps Source: Frank Wuerthwein, Physics UCSD
Planning for climate change in California substantial shifts on top of already high climate variability SIO Campus Climate Researchers Need to Download Results from Remote Supercomputer Simulations to Make Regional Climate Change Forecasts Dan Cayan USGS Water Resources Discipline Scripps Institution of Oceanography, UC San Diego much support from Mary Tyree, Mike Dettinger, Guido Franco and other colleagues Sponsors : California Energy Commission NOAA RISA program California DWR, DOE, NSF
average summer average summer afternoon temperature afternoon temperature 10 GFDL A2 1km downscaled to 1km Hugo Hidalgo Tapash Das Mike Dettinger
Ultra High Resolution Microscopy Images Created at the National Center for Microscopy Imaging
NIH National Center for Microscopy & Imaging Research Integrated Infrastructure of Shared Resources Shared Infrastructure Scientific Local SOM Instruments Infrastructure End User FIONA Workstation Source: Steve Peltier, Mark Ellisman, NCMIR
PRISM Links Calit2’s VROOM to NCMIR to Explore Confocal Light Microscope Images of Rat Brains
Protein Data Bank (PDB) Needs Bandwidth to Connect Resources and Users • Archive of experimentally determined 3D structures of proteins, nucleic acids, complex assemblies • One of the largest scientific resources in life sciences Virus Source: Phil Bourne and Hemoglobin Andreas Prlić, PDB
PDB Usage Is Growing Over Time • More than 300,000 Unique Visitors per Month • Up to 300 Concurrent Users • ~10 Structures are Downloaded per Second 7/24/365 • Increasingly Popular Web Services Traffic Source: Phil Bourne and Andreas Prlić, PDB
2010 FTP Traffic PDBj PDBe RCSB PDB 16 million 34 million 159 million entry downloads entry downloads entry downloads Source: Phil Bourne and Andreas Prlić, PDB
PDB Plans to Establish Global Load Balancing • Why is it Important? – Enables PDB to Better Serve Its Users by Providing Increased Reliability and Quicker Results • How Will it be Done? – By More Evenly Allocating PDB Resources at Rutgers and UCSD – By Directing Users to the Closest Site • Need High Bandwidth Between Rutgers & UCSD Facilities Source: Phil Bourne and Andreas Prlić, PDB
PRISM Will Link Computational Mass Spectrometry and Genome Sequencing Cores to the Big Data Freeway Source: proteomics.ucsd.edu ProteoSAFe: Compute-intensive MassIVE: repository and discovery MS at the click of a identification platform for all button MS data in the world
http://cherub.ucsd.edu SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
CHERuB*: SDSC-ACT partner to bring 100Gbps connectivity to UCSD Production late 2014 UWisc Madison - OSG FNAL - Tier-1 LHC UNL - OSG LBL - CMMAP NERSC - POLARBEAR, CAIDA NICS - UCSB CMMAP UCR UCSD/SDSC New 100G path Austin/TACC Pink line – New CENIC 100G Blue lines – Existing/planned ANI 100G Green lines – Existing PacWave 100G Maroon lines – XSEDE 10G network Thin lines – Other existing 10G or lower SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO *Configurable, High-speed, Extensible Research Bandwidth
The Plumbing (ask Tom Hutton) 818 W. 7th, Los Angeles, CA 10100 Hopkins Drive, La Jolla, CA SDSC NAP Equinix/L3/CENIC POP existing Existing ESnet DWDM DWDM CENIC fiber SD router Nx10G 100G 100G UCSD/SDSC transponders transponders 10G Gateway Juniper MX960 "MX0" up to 3 add'l 100G up to 3 add'l 100G 100G transponders can be transponders can be UCSD/SDSC New 2x100G/8x10G attached attached line card + optics Cisco 6509 10G 100G to CENIC/ New 40G PacWave SDSC Juniper UCSD line card + switch L2 MX960 "Medusa" DYNES optics New 100G card/ PacWave, 2x40G 4x10G 100G optics CENIC, add'l 10G card/optics Internet2, NLR, UCSD Primary Node mult. 10G Other ESnet, Cisco 6509 "Node B" connections SDSC Dual Arista 7508 StarLight, resources "Oasis" XSEDE & other UCSD R&E networks PRISM@UCSD Production users mult. 40G 256x10G Arista 7504 connections mult. 40G+ Key: connections 128x10G DataOasis/ Pink/black - UCSD existing UCSD SDSC Cloud PRISM@UCSD infrastructure - many UCSD big GORDON data users SAN DIEGO SUPERCOMPUTER CENTER compute Green/dashed lines - NEW cluster SDSC new component/ at the UNIVERSITY OF CALIFORNIA; SAN DIEGO DYNES equipment in proposal
CENIC/ESnet 100G Connection enables Big Data science collaborations between NERSC and SDSC UWisc Madison - OSG FNAL - Tier-1 LHC UNL - OSG LBL - CMMAP NERSC - POLARBEAR, CAIDA NICS - UCSB CMMAP UCR UCSD/SDSC New 100G path Austin/TACC Pink line – New CENIC 100G Blue lines – Existing/planned ANI 100G Green lines – Existing PacWave 100G Maroon lines – XSEDE 10G network Thin lines – Other existing 10G or lower SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
A Unique, Powerful, Data-Intensive Testbed for Scientific Discovery EDISON HPC SYSTEM GORDON HPD SYSTEM 2 PF, 434 TB RAM 0.3 PF, 364 TB RAM+SSD 150 GB/s 100 GB/s ESnet/CENIC DTN DTN 6 PB 4.5 PB 100 Gb/s SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
POLARBEAR Cosmology Telescope UC Berkeley/NERSC-UCSD/SDSC • Goal: Measure B-mode polarization in the CMB from inflation era • Data path: Chile (obs)- UCB/NERSC (analysis)- UCSD/SDSC (analysis) • Data acquisition rates: • 22 GB/mo. (current) • 3 TB/mo. (2014-2016) • Map making data analysis NERSC & SDSC • 100 MC realizations of 100 TB data = 10 PB Atacama Desert, Chile SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
Next Generation Network Measurement CAIDA (SDSC)-NERSC • CAIDA operates the UCSD Network Telescope , which collects Internet Background Radiation • Data paths: global internet, ESnet unassigned • Data rates: 3-4 TB/mo IPv4 addresses • Using NERSC tape archive to replicate 100 TB historical data • Other projects: network measurement tools, Future 100’s TB archival data Internet Architecture SDSC/NERSC SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
High Energy Physics LHC/US-CMS UCSD Tier-2 — US-CMS collaboration • Goals: Higgs boson, supersymmetry, BSM • Data Paths: CERN- FNAL (Tier 1)-UCSD (Tier 2) via ESnet and CENIC/I2 • Peak Bandwidths: • Current: 10+5 Gbps • 2015: 40 Gbps when LHC operates @ 14 Tev SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
Education & Training: UCSD Telemedicine Center SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
CHERuB Implementation Status • January, 2014 • Project funded, equipment on order • February, 2014 • Equipment received • Production network switch upgraded • March, 2014 • Campus gateway upgraded, connected to regional 100G feed • Successful border-to-regional test @100Gbps • Next steps (April/May): • Connect Prism switch, test @2x40Gbps • Connect SDSC infrastructure, test @100Gbps • Connect production switch, test @4x10Gbps • Production Goal: September 2014 SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO
Comet is a ~2000TeraFLOP System Architected for the “Long Tail of Science” NSF Track 2 award to SDSC $12M NSF award to acquire $3M/yr x 4 yrs to operate Production early 2015
Recommend
More recommend
