wh t h what have we learned from what have we learned
play

Wh t h What have we learned from What have we learned from Wh t h - PowerPoint PPT Presentation

Sep 08, 2022 •22 likes •352 views

Wh t h What have we learned from What have we learned from Wh t h l l d f d f building the LHC (CMS) DAQ building the LHC (CMS) DAQ systems. systems. S. Cittolin PH-CMD. CERN Openlab meeting. 3-4 March 2009 DAQ at LHC overview CMS

  1. Wh t h What have we learned from What have we learned from Wh t h l l d f d f building the LHC (CMS) DAQ building the LHC (CMS) DAQ systems. systems. S. Cittolin PH-CMD. CERN Openlab meeting. 3-4 March 2009 DAQ at LHC overview CMS systems Project timeline CMS experience CMS experience

  2. Trigger and DAQ overview Trigger and DAQ overview Collisions at LHC Th The four experiments f i Readout and event selection Trigger levels architecture S. Cittolin. CERN/CMS 2

  3. Proton-proton collisions at LHC Proton-proton collisions at LHC Collision rate ~10 9 Hz Collision Rates: E Event Selection: t S l ti ~1/10 13 1/10 13 S. Cittolin. CERN/CMS 3

  4. Data detection and event selection Data detection and event selection Operating conditions: Operating conditions: one “good” event (e.g Higgs in 4 muons ) one “good” event (e.g Higgs in 4 muons ) Collision rate + ~20 minimum bias events) + ~20 minimum bias events) All charged tracks with pt > 2 GeV All charged tracks with pt > 2 GeV ~ 10 8 cells Detector granularity Event size: ~ 1 Mbyte Processing Power: Processing Power: ~ Multi-TFlop Multi TFlop Reconstructed tracks with pt > 25 GeV Reconstructed tracks with pt > 25 GeV S. Cittolin. CERN/CMS 4

  5. General purpose p-p detectors at LHC General purpose p-p detectors at LHC S. Cittolin. CERN/CMS 5

  6. The Experiments The Experiments ATLAS ATLAS ATLAS ATLAS Study of pp collisions µ Tracker: Si( Pixel and SCT), TRT Calorimeters:LAr, Scintillating Tiles Muon System: MDT, RPC, TGC, CSC, Mag Magnets: Solenoid and Toroid gnetic character Study of pp & heavy ion collisions CMS CMS Tracker: Si (Pixel, Strips, Discs) µ µ Calorimeters: BGO, Brass Scintillators, Preshower Muon System: RPC, MDT, CSC, Supraconducting solenoid ALICE ALICE Study of heavy ion collisions Tracker: Si (ITS), TPC, Chambers, TRD, TOF Particle Id: RICH, PHOS (scintillating crystals) RPC, FMD(froward mult.; Si) ZDC (0 degree cal) Magnets: Solenoid, Dipol Magnets: Solenoid Dipol LHCb LHCb Study of CP violation in B decays (pp) Tracker (Si, Velo), 2 RICH, 4 Tracking stations (Straw- Tubes, Si), SPD (scinitll. Pads), Preshower, ECAL (lead scintillator) HCAL(steel scintillator), Muon stations (MWPCs) S. Cittolin. CERN/CMS 6

  7. 40 MHz crossing. Front-end structure 40 MHz crossing. Front-end structure High precision (~ 100ps) timing, trigger and control distribution 40 MHz digitizers and 25ns pipeline readout buffers 40 MHz Level-1 trigger ( massive parallel pipelined processors ) M lti l Multi-level event selection architecture l t l ti hit t Front-end pipeline readout S. Cittolin. CERN/CMS 7

  8. CMS front-end readout systems CMS front-end readout systems S. Cittolin. CERN/CMS S.C. CERN/EP-CMD 8 8

  9. Level-1 trigger systems. Pipelines massive parallel Level-1 trigger systems. Pipelines massive parallel S. Cittolin. CERN/CMS 9

  10. Multi-level trigger DAQ architecture Multi-level trigger DAQ architecture On-line requirements Event rate Event rate 1 GHz Level-1 input Event size Event size 1 Mbyte 1 Mbyte O ON-line Level-1 Trigger input 40 MHz Level-2 Trigger input 100 kHz Level-2 input Mass storage rate ~100 Hz Level-3 …. Level 3 Online rejection 99.999% System dead time ~ % Selected events to archive DAQ design issues OFF-l Data network bandwidth (EVB) ~ Tb/s Computing power (HLT) ~ 10 Tflop C Computing cores ti ~ 10000 10000 ine Local storage ~ 300 TB Minimize custom design Exploit data communication and computing technologies DAQ staging by modular design (scaling) S. Cittolin. CERN/CMS 10

  11. LHC trigger and DAQ summary LHC trigger and DAQ summary No.Levels Level-0,1,2 Event Readout HLT Out Trigger Rate (Hz) Size (Byte) Bandw.(GB/s) MB/s (Event/s) 3 3 LV-1 10 5 10 5 1 5 10 6 1.5x10 6 4 5 4.5 300 300 (2x10 2 ) 2 LV-2 3x10 3 2 LV-1 10 5 10 6 100 O(1000) (10 2 ) 2 LV-0 10 6 3x10 4 30 40 (2x10 2 ) 4 Pb-Pb 500 5x10 7 25 1250 (10 2 ) 10 3 3 2x10 6 6 200 (10 2 ) p-p S. Cittolin. CERN/CMS 11

  12. LHC DAQ architecture LHC DAQ architecture DAQ technologies DAQ systems at LHC S. Cittolin. CERN/CMS 12

  13. Evolution of DAQ technologies and structures Evolution of DAQ technologies and structures PS:1970-80: Minicomputers Detector Readout Detector Readout Readout custom design Event building First standard: CAMAC On-line processing S ft Software: noOS, Assembler OS A bl Off-line data store • kByte/s p-p/LEP:1980-90: Microprocessors p-p/LEP:1980-90: Microprocessors HEP proprietary (Fastbus), Industry standards (VME) Embedded CPU, servers Software: RTOS, Assembler, Fortran • MByte/s MByte/s LHC 200X N t LHC: 200X: Networks/Clusters/Grids k /Cl t /G id PC, PCI, Clusters, point to point switches Software: Linux, C,C++,Java,Web services Protocols: TCP/IP, I2O, SOAP, • TByte/s • TByte/s S. Cittolin. CERN/CMS 13

  14. LHC trigger and data acquisition systems LHC trigger and data acquisition systems LHC DAQ : A computing&communication network p g Alice Alice A single network cannot satisfy at once all the LHC requirements, therefore present LHC DAQ designs are implemented as multiple (specialized) networks ATLAS LHCb LHCb S. Cittolin. CERN/CMS 14

  15. CMS DAQ baseline structure CMS DAQ baseline structure Collision rate Collision rate 40 MHz 40 MHz Readout concentrators/links Readout concentrators/links 512 x 4 Gb/s 512 x 4 Gb/s Level-1 Maximum trigger rate Level-1 Maximum trigger rate 100 kHz 100 kHz Event Builder bandwidth max. Event Builder bandwidth max. 2 Tb/s 2 Tb/s Average event size Average event size ≈ 1 Mbyte ≈ 1 Mbyte Event filter computing power Event filter computing power ≈ 10 TeraFlop ≈ 10 TeraFlop ≈ 10 6 Mssg/s ≈ 10 6 Mssg/s Flow control&monitor Flow control&monitor Data production Data production ≈ Tbyte/day ≈ Tbyte/day Processing nodes Processing nodes ≈ Thousands ≈ Thousands S. Cittolin. CERN/CMS 15

  16. Two trigger levels Two trigger levels Level-1: Massive parallel processors 40 MHz synchronous -Particle identification: Particle identification: -high pT electron, muon, jets, missing ET -Local pattern recognition and energy -evaluation on prompt macro-granular -information from calorimeter and muon detectors 99.99 % rejected: 0.01 Accepted Level-2: Full event readout into PC farms 100 kHz asynchronous farms - Clean particle signature Clean particle signat re - Finer granularity precise measurement - Kinematics. effective mass cuts and event topology - Track reconstruction and detector matching - Event reconstruction and analysis y 100-1000 Hz. Mass storage 99.9 % rejected: 0.1 Accepted Reconstruction and analysis. S. Cittolin. CERN/CMS 16

  17. 8-fold DAQ structure 8-fold DAQ structure S. Cittolin. CERN/CMS 17

  18. Sept. 2008 first events Sept. 2008 first events S. Cittolin. CERN/CMS

  19. March 09 Technical Global Run March 09 Technical Global Run S. Cittolin. CERN/CMS 19

  20. CMS experience CMS experience DAQ project timeline Industry trends/DAQ Hardware/Software components DAQ at Super LHC p S. Cittolin. CERN/CMS 20

  21. LHC/CMS-DAQ project timeline LHC/CMS-DAQ project timeline R Research and Development (DRDC) h d D l t (DRDC) 1990 Design of experiment Trigger, Timing and Control distribution (TTC) Readout prototypes (FPGA,PC, IOP-200 MB/s ) 1992 CMS Letter of Intent Networks (ATM, Fiber Channel, xyz..) 1994 Technical Design Report CMS 2-level triggers design 1996 Event Builder Demonstrators FPGA/PC data concentrators 1998 8x8 Fiber channel EVB 32x32 Myrinet EVB 2000 Trigger Technical Design Report 2000 Trigger Technical Design Report 64x64 Ethernet EVB, PC driven 64x64 Ethernet EVB PC driven 2002 DAQ Technical Design Report Final Design Pre-series 2004 2004 64x64 Myrinet/Ethernet 64 64 M i t/Eth t Construction and commissioning 2006 Magnet test Global Run 1024 2 Gb/s D2S Myrinet links and routers 8x80x(80x7) GbEthernet EVB/HLT 8x80x(80x7) GbEthernet EVB/HLT 2008 Circulating beam Global Run 10000 on-line cores 2009 Colliding beams Lesson 1. 12 yeas of R&D (too much?) S. Cittolin. CERN/CMS 21

  22. Computing and communication trends Computing and communication trends Lesson 2. Moore law confirmed S. Cittolin. CERN/CMS 22

  23. Two trajectories Two trajectories 1997 GOOGLE first cluster 1997 GOOGLE first cluster 1997 CMS 4x4 FC-EVB 1997 CMS 4x4 FC EVB 2008 One of Google data centers 10 6 cores 2008 Cessy CMS HLT center 10 4 cores, 2 Tb/s maximum bandwidth S. Cittolin. CERN/CMS 23 23

  24. Global Internet traffic (Cisco forecasts) Global Internet traffic (Cisco forecasts) US Consumer (PB per month) US Consumer (PB per month) 2007 2007 2008 2008 2009 2009 2010 2010 2014 2014 Web, email, transfer 710 999 1336 1785 P2P 1747 2361 3075 3981 Gaming 131 187 252 324 Video Communications 25 37 49 70 VoIP 39 56 72 87 Internet Video to PC 647 1346 2196 3215 Internet Video to TV 99 330 756 1422 Business 1469 2031 2811 3818 Mobile Mobile 26 26 65 65 153 153 345 345 Total global traffic (Pb/M) 4884 7394 10666 14984 Global Internet traffic (Tb/s) 20 30 40 60 Total US traffic (Tb/s) 3 4 6 8 Google US traffic (Tb/s) 0.3 0.7 1.5 3 CMS Maximum bandwidth (Tb/s) CMS Maximum bandwidth (Tb/s) 1 1 2 2 2 2 2 2 >10 10 Lesson 3. Will we buy computing power and network bandwidth? S. Cittolin. CERN/CMS 24

Recommend

Lessons Learned Lessons Learned From From Lessons Learned Lessons Learned From From

Lessons Learned Lessons Learned From From Lessons Learned Lessons Learned From From

Protg 2006, July 26th, Stanford I R I S A C 1 R E C Saint-Cyr Lessons Learned Lessons Learned From From Lessons Learned Lessons Learned From From Ontology Ontology Design Design Ontology Ontology Design Design Jean Andr B

371 views • 12 slides
4 5 6 CSE 142 vs CSE 143 CSE 142 / AP CS A CSE 143 You learned how to write Return of

4 5 6 CSE 142 vs CSE 143 CSE 142 / AP CS A CSE 143 You learned how to write Return of

4 5 6 CSE 142 vs CSE 143 CSE 142 / AP CS A CSE 143 You learned how to write Return of the objects programs and You learned to solve decompose large more complex tasks problems with: efficiently Print statements Data

705 views • 22 slides
WHAT HAVE WHAT HAVE I LEARNED I LEARNED AT THE AT THE DISC DISC SUMMER SCHOOL? SUMMER

WHAT HAVE WHAT HAVE I LEARNED I LEARNED AT THE AT THE DISC DISC SUMMER SCHOOL? SUMMER

WHAT HAVE WHAT HAVE I LEARNED I LEARNED AT THE AT THE DISC DISC SUMMER SCHOOL? SUMMER SCHOOL? Christos G. Cassandras CODES Lab. - Boston University WHY HYBRID SYSTEMS ? COMPLEXITY ( PHYSICAL , BUT MOSTLY OPERATIONAL/ORGANIZATIONAL )

691 views • 11 slides
What We Have Learned from the Pandemic NURSING SIMULATION: What We Have Learned from the

What We Have Learned from the Pandemic NURSING SIMULATION: What We Have Learned from the

July 17, 2020 Nursing Simulation: What We Have Learned from the Pandemic NURSING SIMULATION: What We Have Learned from the Pandemic PAMELA R. JEFFRIES CRYSTEL L. FARINA PAUL LUND PAMELA SLAVEN-LEE George Washington University George

790 views • 21 slides
Object-Oriented Design What Do We Mean by OO Design? Remember how we learned about functions?

Object-Oriented Design What Do We Mean by OO Design? Remember how we learned about functions?

Module 21 Object-Oriented Design What Do We Mean by OO Design? Remember how we learned about functions? First learned to call functions made for us Then learned to define our functions Finally learned to properly design functions

471 views • 30 slides
has learned Col Rakesh Jetly OMM,CD,MD,FRCPC. Lots of Ground to Cover!! Why am I here?

has learned Col Rakesh Jetly OMM,CD,MD,FRCPC. Lots of Ground to Cover!! Why am I here?

Mental Health In the Workplace: What CAF has learned Col Rakesh Jetly OMM,CD,MD,FRCPC. Lots of Ground to Cover!! Why am I here? Lessons learned from Military medicine CAF in Kandahar MH Lessons Learned MH in The

1.18k views • 80 slides
Electrical Team Marina Beshai, Leslie Kim, and Sara Sacks Overview of the Semester: Learned

Electrical Team Marina Beshai, Leslie Kim, and Sara Sacks Overview of the Semester: Learned

Electrical Team Marina Beshai, Leslie Kim, and Sara Sacks Overview of the Semester: Learned how to test the point system. Learned how to use the oscilloscope. Started the portable power generator. Solder: Learned how to

252 views • 14 slides
What have we learned after 10 years and 120,000 cryoablations? Richard Schilling conflicts -

What have we learned after 10 years and 120,000 cryoablations? Richard Schilling conflicts -

What have we learned after 10 years and 120,000 cryoablations? Richard Schilling conflicts - speaker fees and research grants Medtronic, Biosense Webster Talk outline Brief procedure description What have I learned anecdote, personal

575 views • 23 slides
Institutionalizing Lessons Learned October 25, 2006 Loren Plisco Region II Background

Institutionalizing Lessons Learned October 25, 2006 Loren Plisco Region II Background

Institutionalizing Lessons Learned October 25, 2006 Loren Plisco Region II Background SEP 2002 Davis-Besse Lessons Learned Task Force AUG 2004 - Effectiveness Review Lessons Learned Task Force JAN 2005 - EDO Charters

269 views • 22 slides
OSHA Lessons Learned Adam Fries OSHA Compliance Officer February 13, 2018 OSHA Lessons Learned

OSHA Lessons Learned Adam Fries OSHA Compliance Officer February 13, 2018 OSHA Lessons Learned

OSHA Lessons Learned Adam Fries OSHA Compliance Officer February 13, 2018 OSHA Lessons Learned Jobsite Safety and Health Inspections/Audits better known as; PAPER WORK You hate it, Your safety director requires it, OSHA loves it, Does it

926 views • 35 slides
3/8/2019 Epidemiology, Risk Factors, and Outcomes of Pediatric PVD: LESSONS learned from the

3/8/2019 Epidemiology, Risk Factors, and Outcomes of Pediatric PVD: LESSONS learned from the

3/8/2019 Epidemiology, Risk Factors, and Outcomes of Pediatric PVD: LESSONS learned from the Spanish Registry: Lessons Learned from the Registries o Is It possible (and worthy) to do a national registry? Lessons learned from the o

1.16k views • 43 slides
DEBUGGING LESSONS LEARNED WHILE DEBUGGING LESSONS LEARNED WHILE FIXING NETBSD FIXING NETBSD

DEBUGGING LESSONS LEARNED WHILE DEBUGGING LESSONS LEARNED WHILE FIXING NETBSD FIXING NETBSD

DEBUGGING LESSONS LEARNED WHILE DEBUGGING LESSONS LEARNED WHILE FIXING NETBSD FIXING NETBSD ABOUT ME ABOUT ME maya@NetBSD.org coypu@sdf.org NetBSD/pkgsrc for the last 3 years THIS TALK THIS TALK Mix of a bunch of bugs Not solo work

695 views • 31 slides
Lessons Learned A Value Added Product of the Project Life Cycle R Gilman April 19, 2006 Agenda

Lessons Learned A Value Added Product of the Project Life Cycle R Gilman April 19, 2006 Agenda

Lessons Learned A Value Added Product of the Project Life Cycle R Gilman April 19, 2006 Agenda Introduction What are Lessons Learned? Value to the Project Process and Organization Keanes Approach to Lessons Learned Keys to

261 views • 24 slides
LEARNED SO FAR C A R M E N & N I C O L E & A N I TA D D C I N T E R N S 2 0 1 7

LEARNED SO FAR C A R M E N & N I C O L E & A N I TA D D C I N T E R N S 2 0 1 7

WHAT WEVE LEARNED SO FAR C A R M E N & N I C O L E & A N I TA D D C I N T E R N S 2 0 1 7 THE HISTORY OF DISABILITY Di Disability sability Hi Hist story ory st start rted ed in n 40 400 0 B.C. Disability

439 views • 15 slides
{ We are at the end of our first cycle of Learning Outcomes Assessment What has COA learned:

{ We are at the end of our first cycle of Learning Outcomes Assessment What has COA learned:

{ We are at the end of our first cycle of Learning Outcomes Assessment What has COA learned: Weve learned that Learning Assessment is not going away. Weve learned that Learning Assessment can provide valuable information.

313 views • 15 slides
The Learned Intermediary Doctrine: A Retrospective Review to the Present Jennifer A. Guidea,

The Learned Intermediary Doctrine: A Retrospective Review to the Present Jennifer A. Guidea,

The Learned Intermediary Doctrine: A Retrospective Review to the Present Jennifer A. Guidea, Esq. Shana E. Russo, Esq. Reed Smith Life Sciences Industry Group 1 Background and General Application of the Learned Intermediary Doctrine 2

519 views • 50 slides
Four Major Events in Our History and the Lessons Learned Coleen Reiswig, Isobel McDonald, Ted

Four Major Events in Our History and the Lessons Learned Coleen Reiswig, Isobel McDonald, Ted

Four Major Events in Our History and the Lessons Learned Coleen Reiswig, Isobel McDonald, Ted Pincock, Carolyn Bouchard Joanne Archer Outline: 1) Common Themes 2) The Event and lessons learned: Spanish Influenza 1918 SARS

417 views • 38 slides
Lessons Learned from Japans NPL Experience Ladies and gentlemen, today I would like to discuss

Lessons Learned from Japans NPL Experience Ladies and gentlemen, today I would like to discuss

Beijing International Finance Forum Chairman Junichi Ujiie 2004/05/19 Lessons Learned from Japans NPL Experience Ladies and gentlemen, today I would like to discuss the lessons to be learned from Japans NPL problems: first, the disposal

344 views • 3 slides
Some lessons learned from Team Science Some lessons learned from Team Science Lewis Cantley Weill

Some lessons learned from Team Science Some lessons learned from Team Science Lewis Cantley Weill

Some lessons learned from Team Science Some lessons learned from Team Science Lewis Cantley Weill Cornell Medical College, New York Presbyterian Hospital Past participation in Team Science Period Period Type of grant Type of grant Role in Grant

380 views • 20 slides
MDG-SDG TRANSITION IN IVORY COAST: LESSONS LEARNED AND IMPLICATIONS ON CURRENT PUBLIC POLICIES

MDG-SDG TRANSITION IN IVORY COAST: LESSONS LEARNED AND IMPLICATIONS ON CURRENT PUBLIC POLICIES

MDG-SDG TRANSITION IN IVORY COAST: LESSONS LEARNED AND IMPLICATIONS ON CURRENT PUBLIC POLICIES SAKO G. . Oumar CONTENTS I. Context II. Assessment and lessons learned from implemention of MDGs III. Domestication/contextaulisation of SDGs

151 views • 12 slides
VALUE: Lessons Learned Kate McConnell & Erin Horan Association of American Colleges and

VALUE: Lessons Learned Kate McConnell & Erin Horan Association of American Colleges and

VALUE: Lessons Learned Kate McConnell & Erin Horan Association of American Colleges and Universities Todays session Overview of the MSC Scoring and reporting Lessons learned as related to validity Unintended consequences?

661 views • 40 slides
Final Exam Prep Lecture ME EN 412 Andrew Ning aning@byu.edu Outline What Have We Learned

Final Exam Prep Lecture ME EN 412 Andrew Ning aning@byu.edu Outline What Have We Learned

Final Exam Prep Lecture ME EN 412 Andrew Ning aning@byu.edu Outline What Have We Learned Topics and Problems Examples What Have We Learned Topics: Dimensional Analysis, Balance Laws Potential Flow, Boundary Layers, Lift and Drag

483 views • 13 slides
FACILITATED BY: Robin Booth, CPA 2 Training Topics Lessons Learned Best Practices

FACILITATED BY: Robin Booth, CPA 2 Training Topics Lessons Learned Best Practices

Office of Housing Counseling Best 9/20/2016 Practices/Lessons Learned - LHCAs U.S. Department of Housing and Urban Development Office of Housing Counseling Best Practices/Lessons Learned LHCAs September 20, 2016 12:00 PM Eastern Standard

569 views • 9 slides
Lessons learned - viewpoints on increasing analytical capabilities Elizabeth Riczko, FCAS,

Lessons learned - viewpoints on increasing analytical capabilities Elizabeth Riczko, FCAS,

Lessons Learned 3/22/2011 Lessons learned - viewpoints on increasing analytical capabilities Elizabeth Riczko, FCAS, MAAA, CPCU 1 What are we doing? 2 It starts with a goal NOT a method A business question Ask why (and repeat)

338 views • 10 slides

More recommend