HCAL Back End Requirements and Architecture A. Belloni University - PowerPoint PPT Presentation

HCAL Back End Requirements and Architecture A. Belloni – University of Maryland

Biographical Notes  Assistant Professor, University of Maryland  2007 Ph.D. – Massachusetts Institute of Technology  Relevant positions  2014-present: US CMS L4 manager, Phase-I Upgrade: HB/HE/HF ngCCM  2015-present: HCAL Test Beam Run Coordinator  2016-present: US CMS L3 manager, Phase-II Upgrade: HCAL Barrel  Former experience: ATLAS Muon MDT chambers, DAQ, installation, commissioning A. Belloni :: HCAL BE 8/29/2017 2

Outline  Requirements for HL-LHC  Current configuration of Hadronic Barrel Back-end Electronics (HB BE)  Almost current: expected configuration at end of Phase-I Upgrade  Proposed configuration for HB BE during HL-LHC  Including interfaces with front-end and DAQ  HCAL-specific test path A. Belloni :: HCAL BE 8/29/2017 3

HB BE Electronics Upgrade Overview L1 Trigger 2.4Gbps Clock Control DAQ Powerful FPGA Feature extraction, Trigger primitive 5.0Gbps formation data links TCDS++ HB RBX Key reason for upgrade Sustain L1 trigger rate up to 750kHz A. Belloni :: HCAL BE 8/29/2017 4

Legacy HB BE Configuration  Design guidelines  uTCA-based system: widely used in CMS, easy to maintain, economical, compact  FE has no buffers for data, BE does processing for trigger and DAQ: flexible system – algorithms run in low-radiation environment A. Belloni :: HCAL BE 8/29/2017 5

Legacy BE Key components  uHTR  Receives data from front-end to uHTR AMC13 prepare them for trigger, luminosity, data acquisition system  AMC13  First and customized crate management and data hub unit o Used also in trigger system and in clock/timing system  Sends data from crate to DAQ  Commercial parts  uTCA crates  MCH o Second crate management and data hub  Power modules and AC/DC MCH Power converters A. Belloni :: HCAL BE 8/29/2017 6

Legacy uHTR Design Notes  Functions  Store data and transfer to DAQ via AMC13 on trigger  Produce trigger primitives  Luminosity info to CMS and LHC LumiDAQ (HF only)  Key design features Lessons for design of Phase-II BE  FPGA provides flexibility; firmware in source control system  Modular design with mezzanines (power, control…) makes QA and maintenance easier A. Belloni :: HCAL BE 8/29/2017 7

Phase-II FE Overview FEE Card (16 channels) FEE Module (64 channels) Replaced by BCP  No upgrades foreseen for FE in ATCA crate  Legacy system organized in 36 readout boxes (RBX), each of which controls and reads out a 20-degree f wedge in one hemisphere o Total number of readout channels per RBX: 252  Clock and control via ngCCM board o One board per RBX A. Belloni :: HCAL BE 8/29/2017 8

Phase-II FE Summary  Data path  No changes with respect to legacy system; same transverse and longitudinal segmentation, same number of channels  15 h towers with 4 longitudinal depths and 1 h tower with 3 depths in each 5-degree f wedge: 252 channels per RBX; 9072 channels for whole HB  Data rate: 5Gbps  Clock and control path  Four ngCCM with two bi-directional links per RBX o Catastrophic failure of ngCCM board loses 1/4-RBX (~0.7% of HB)  Link rate: 2.4Gbps  Trigger path  Trigger primitives will remain similar to Phase-I primitives  2304 trigger towers, each sending 16-bit trigger primitive o 10 bits: energy sum of entire trigger tower (all longitudinal depths) plus 6 feature bits  Algorithms to define the feature bits are being designed A. Belloni :: HCAL BE 8/29/2017 9

Phase-II BE Design and Requirements  BE needs to sustain trigger rate of 750kHz, and latency of 12.5 m s, required by HL-LHC trigger system  uHTR does not have sufficient bandwidth, and will be replaced by board in ATCA standard  HB will use the same BCP board being designed for EB BE  Value engineering example: “[using the same board] optimizes the usage of development and production resources, enables the sharing of spares, achieves homogeneity in the trigger primitive generation, and facilitates long-term operations and maintenance” (from the Phase -II HCAL TDR)  Firmware will be adapted to work in HB environment A. Belloni :: HCAL BE 8/29/2017 10

Phase-II HB Electronics Schema x 16 2 links QIE11 256 channels Versatile 16 channels QIE cards 32 optical links 1 RBX (4 RMs cards or 16 QIE cards ) @ 5.0 Gb/s - Two UltraScale FPGA - ATCA blade Trigger, DAQ, FE control  EB and HB use common ATCA platform  16Gbps optical links to DAQ and Trigger A. Belloni :: HCAL BE 8/29/2017 11

Readout Schema  Readout unit is 20-degree f wedge in one hemisphere  Corresponds to one RBX  Contains 16x4 h - f trigger towers o 0<| h |<1.392  Each region is served by one FPGA in an ATCA blade  Two FPGAs per blade  HB BE needs a total of 36 FPGAs A. Belloni :: HCAL BE 8/29/2017 12

BE-to-FE Interface 16 32 31 Total number of 30 15 29 FE to BE links: 1440 28 14 27 26 25 13 24 23 12 Number of links 22 21 11 20 Per BE FPGA is 40 + 19 18 10 17 16 9 η 15 14 Total number of 13 8 Tower 4 up + 4 down 12 HB Back-End FPGAs 36 11 7 - 10 9 6 8 One FPGA 7 6 5 5 64 Towers 4 4 3 2 3 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 2 Tower φ 20-degree 1 Control wedge /Status 1 2 3 4 16 η segments x 2@5.0Gb/s + 4 bi-directional @2.4Gb/s = 40 links (32+4 upstream & 4 downstream) A. Belloni :: HCAL BE 8/29/2017 13

BE-to-Trigger Interface 16 32 31 Number of links 30 15 29 Total number of per BE card is 8 28 14 27 BE to L1T links: 228 26 25 13 24 Tower 23 12 22 21 11 20 + One FPGA 19 18 10 17 64 Towers 16 9 η 15 14 Total number of 13 8 12 HB Back-End FPGAs 36 11 7 - 10 9 6 8 7 6 5 5 4 4 3 16Gb/s link 2 3 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 2 φ Tower 16Gb/s link 20-degree 1 wedge 1 2 3 4 • Assuming TPs with 16 bits x 4 depths 8 links x 16Gb/s • Each optical link includes TPs from 2 η ( 8 towers) A. Belloni :: HCAL BE 8/29/2017 14

BE-to-DAQ Interface  Inputs  5.0Gbps (8b10b) upstream payload: 88 bits  2x88 bits per 4 towers: 176 bits  16 h towers: 2816 bits  Input readout size (assume 10 BXs): 28160 bits  Outputs  64 trigger towers x 64 bits (trigger primitive size with 4 oversampling): 4096  Output readout size, assuming 10 BX: 40960 bits  Readout window  28160 + 40960 = 69120 bits (event size ~ 8.5KB)  DAQ payload at 750kHz trigger rate  69120 bits x 750kHz = 52Gbps  4 x 16Gbps per FPGA needed to send event to DAQ  Total number of DAQ links: 144 A. Belloni :: HCAL BE 8/29/2017 15

The BCP board (ATCA blade format)  Based on FPGA UltraScale package: B2104  46 transceivers needed  36 inputs  32 @ 5.0Gbps: upstream links from FE (QIE cards)  4 @ 2.4Gbps: status links from RBX (ngCCM)  16 outputs  8 @ 16Gbps: trigger primitives  4 @ 2.4Gbps: control links to RBX (ngCCM)  4 @ 16Gbps: DAQ  2 control links  TTC, TTS, DAQ flow control A. Belloni :: HCAL BE 8/29/2017 16

HB-Specific Test Path  Build readout chain with HB FE and CTP7 board in uTCA crate  Plan to follow schedule of EB electronics demo  Replace CTP7 with ATCA blade  First test of complete Phase-II readout chain  Perform slice test at CERN TB  Full chain, including actual scintillators and photosensors  HB-specific: firmware  Plan to use same back-end hardware as EB; need to develop firmware to talk to HB front-end A. Belloni :: HCAL BE 8/29/2017 17

Test of Readout Chain  EB demo exists in CERN Lab  May need HB adapter boards between FE and CTP7 1 GBT link (control, bc0, clock) bc 0 A μ TCA Crate M LHC clock C C 1 T 3 32 GBT links P M 7 (QIE data) C H HB RBX spy QIE data Host PC A. Belloni :: HCAL BE 8/29/2017 18

Slice Test  A 20-degree HB wedge is installed along the H2 beam line  CMS HCAL maintains a control room with a complete slice of the DAQ and clock-and-control systems  Plan to install an ATCA crate, with two BCP blades, to test their performance and firmware  Asynchronous beam; pions, electrons, muons in the 20-300GeV range A. Belloni :: HCAL BE 8/29/2017 19

Summary  Required upgrade to satisfy HL-LHC data taking conditions  HB BE foreseen to use the same hardware as EB BE  Improve usage of production and development resources, facilitate long-term operations and maintenance  Project costs and deliverables scale with number of channel in EB vs HB  216 BCP for EB vs 36 for HB  Reserve resources to work on HB-specific firmware  Quality assurance follows recommended guidelines  Foresee production of prototype and pre-production boards, with resources allocated to promptly implement design updates  CTP7, then ATCA demonstrator setup  Slice test at CERN test beam A. Belloni :: HCAL BE 8/29/2017 20