HCAL Trigger Readout HCAL Trigger Readout HCAL Trigger Readout HTR Status and Clocking Issues D. Baden, T. Grassi http://www.physics.umd.edu/hep/esr_dec_2002.pdf 1 CMS ESR December 2002
FE/DAQ Electronics FE/DAQ Electronics FE/DAQ Electronics S-Link: 64 bits @ 25 MHz Trigger Primitives READ-OUT Crate S C D H H H CAL Rack CPU 12 HTRs per B L C T T T REGIONAL S K C R R R Readout Crate, TRIGGER 2 DCC 16 bits TTC @ 80 MHz 32 bits FRONT-END @ 40 MHz RBX HPD Shield Readout Box Wall QIE CCA GOL (On detector) QIE QIE CCA Fibers at 1.6 Gb/s QIE GOL 3 QIE-channels per fiber QIE CCA QIE FE MODULE 2 CMS ESR December 2002
HTR Principal Functions HTR Principal Functions HTR Principal Functions 1. Receive front-end data for physics running • Synchronize optical links • Data validation and linearization • Form TPG’s and transmit to Level 1 at 40 MHz • Pipeline data, wait for Level 1 accept • Upon receiving L1A: • Zero suppress, format, transmit to the concentrator (no filtering) • Handle DAQ synchronization issues (if any) 2. Calibration processing and buffering of: • Radioactive source calibration data • Laser/LED calibration data 3. Support a VME data spy monitoring 3 CMS ESR December 2002
Readout VME Crate Readout VME Crate Readout VME Crate “BIT3” board Front End Electronics – Slow monitoring over VME TTC fiber – Commercial VME/PCI Interface to CPU Fiber 1.6 Gb/s FanOut board R ... – Takes TTC stream in F – Clone and Fanout timing signals a H D H H H B n T C T T T I O HTR ( HCAL T rigger and R eadout ) board R C R R VME CRATE T u D – Spy output over VME 3 t C – FE-Fiber input C – TPG output (SLBs) to CRT – DAQ/TP Data output to DCC DCC ( D ata C oncentrator C ard) board 20m Copper DAQ – Input from HTRs 1.2 Gb/s – Spy output – Output to DAQ Calorimeter Regional Trigger 4 CMS ESR December 2002
“Old” HTR Design Old” HTR Design “ “Old” HTR Design (Summer 2002) (Summer 2002) (Summer 2002) • Board organized around 2 identical sets of circuitry: • Optical inputs • 1.6 GHz, 8B/10B frames, 3ch/link • Dual LC detectors and drivers • TI TLK2501 Deserializers • Crystal RefClk • TTC 80MHz backup • Xilinx Virtex FPGA XCV1000E • 24 channels each • TPG signals • Sent to SLB over backplane, LVDS • SLBs mounted 6 to a transition board • Level 1 accept output to DCC • LVDS output OLD DESIGN • VME • Altera FPGA and firmware 5 CMS ESR December 2002
HTR Functional Experience HTR Functional Experience HTR Functional Experience • What was tested: • VME fully tested and working • Some changes necessary to conform to CMS VME standards • Optical links and synchronization • No indication of any problems. Big success here – was a real worry • LVDS to DCC • Tested, working (Will change cable/connector to Cat 6/RJ45) • Fanout of timing signals on two Cat5 cables • Plan to change to a single Cat6 or Cat7 cable (very low cross-talk) • Firmware – full tests of: • Pipeline and L1A triggering capability • In-line histogramming for source calibration • TTCrx • Not working at all (4 bad on 4 tested). • What was not tested: Anything to do with TPG 6 CMS ESR December 2002
HTR Board Experience HTR Board Experience HTR Board Experience • Produced ~12 boards • Several bare boards were delivered warped • Many opens under FPGA after assembly (~9 boards) • Some fixed after reflow (a few) • Some worse after reflow (shorts) • X-rayed a few boards, sometimes inconclusive • Some opens on VME side • Non BGA FPGA, indicates bad vias • Few other various open circuits • Finally got ~8 boards to “work” • Questionable reliability 7 CMS ESR December 2002
Modifications Modifications Modifications • Change board from using white-tin to gold traces • This process was sold to us by the board maker. Our mistake. • Used only for very high volume, cost competitive products, very difficult and expensive to control. • Gold is flatter and not very much more expensive (~$50/board), better for FPGAs • Change assembly house • Insufficient Quality Control on current assembler – they are fired. • We visited 2 high-end assemblers • Modern Machines • Step up and step down oven temp control. • In-line X-ray for BGA QC • Manufacturability Review • Add stiffeners to HTRS • Flexability of 9U VME boards was underestimated • Worry: fine-line BGA (FBGA) can pop connections 8 CMS ESR December 2002
Modifications (cont) Modifications (cont) Modifications (cont) • Change from FBGA to BGA • FBGA 1.0mm pitch, change to BGA 1.27mm pitch • No additional expense, plenty of available real estate, no need to push • We are just being very conservative here • JTAG capabilities added • Will help with debugging • By making these changes… • We have profited from the summer • We have reduced our production risk considerably 9 CMS ESR December 2002
HTR Design Changes HTR Design Changes HTR Design Changes • SLB transition board issues: • Worries about so many LVDS signals over backplane for old design • Routing is too complicated • Many signals going to same backplane location • Requires multi-layer routing with many vias • TPG cables very thick • Mechanical issues are very worrisome • SLB changes needed (e.g. height reduced after ECAL redesign…) • Solution: move SLB’s to HTR motherboard • Benefits: • Mechanically attach SLB’s to HTR front panel for mechanical stability • Eases routing requirements, reduces board and assembly risks, cheaper too • Change from Xilinx VirtexE to Virtex2 • More resources, block ram, hardware multipliers • Big cost reduction (save $180k) • More modern chip for long-term maintenance • Clock synchronization • Decouple “80MHz” crystal from FPGA system clock • Will allow us to use crystal to maintain synchronization of serdes • This gives us 2 solutions for our “40ps” jitter requirement issue 10 CMS ESR December 2002
New HTR Conceptual Design New HTR Conceptual Design New HTR Conceptual Design Fibers 8-way TI TI 8-way TI P1 LC to DCC LVDS LC FPGA TI LVDS VME Xilinx FPGA XC2V LC TI SLB LC to Level 1 Cal Trigger TI P2 SLB TI TI SLB TI TI TI LC SLB LC FPGA TI Xilinx SLB No P3! XC2V LC TI LC SLB TI TI TI Fanout from TTC RJ45 11 CMS ESR December 2002
Clocking Schematic Clocking Schematic Clocking Schematic • Start with Fanout card • TTCrx Maryland mezzanine card or CERN TTCrm daughterboard • QPLL • Fanout on Cat6/7 quad twisted pair TTC, BC0, 40MHz, 80MHz • In HTR: • Send TTC signal to TTCrx mezzanine board, access to all TTC signals • Send 80MHz clean clock (cleaned by QPLL) to mux • Select 80MHz clean clock OR crystal to TI deserializers Cat 6/7 quad cable (allows PECL) TTC Fanout Board TTCrx 80 MHz F SLB LVPECL Crystal TTC 80MHz 1 to 8 SLB P QPLL BC0 Fanout TI 80MHz SLB (16) BC0 G 40MHz BC0 1 to 8 SLB Fanout A SLB 80 MHz 1 to 8 SLB 40MHz Fanout 40 MHz TTC mezz TTC broadcast bus TTC Single width VME 12 CMS ESR December 2002
HCAL TRIDas TRIDas Clock Scheme Clock Scheme HCAL HCAL TRIDas Clock Scheme Fanout Card TTC BC0 Cat6/7 RJ45 TTCrx QPLL 4 twisted pair… CC40 CC80 (‘CC’ means Clean Clock) TTC TTC broadcast, L1A, BCR, EVR, CLK40 RJ45 TTCMezz CC80 Xilinx CC40 HTR Board BC0 SLB 13 CMS ESR December 2002
Fanout – – HTR scheme HTR scheme Fanout Fanout – HTR scheme Notes: SLBs require fanout of CLK40, BC0. FE-link possibly requires CLK80. PECL fanout was tested in TB2002. One Cat6E cable (low x-talk) replaces the 2 Cat5 cables used in TB2002. TTC and BC0 remain LVDS as in Weiming’s board. HTR needs Broadcast bus, BCntRes and L1A: from TTCrx if we get it to work, otherwise we have to fan them out. TTC fiber Fanout O/E TTC buffer TTC TTC LVDS TTC IN DS90LV001 daughter IN_b card Low-jitter ~Fifteen Brdcst<7:2>, Fanout x 15 RJ45 e.g. DS90LV110 BrcstStr, L1A, RJ45 connectors BCntRes TTC .. LVDS to xilinx Diff. to 6 SLBs .. RX_BC0 LVDS Brdcst<7:2>, Fanout and SLBs Single-end to 2 xilinx LVDS .. BrcstStr x 8 .. BC0 TTCrx Cat6E .. FPGA or Cat7 (or .. .. cable Q1 daughter Q2 card) CLK40 Q3 .. To 6 3.3V-PECL CLK40 Q4 SLBs PECL 2 Test .. LVDS Q5 fanout Points for .. Q6 8 clks CLK40 Diff. to 2 Q7 to TLKs and Xilinx QPLL CLK80 Q8 PECL + termin. BC0 LVDS MC100LVE310 fanout PCK953 3.3V PECL LVPECL- NB100LVEP221 to-LVTTL AN1568/D Fig 11 is LVDS compatible Fanout MC100LVEL37 Onsemi.com CLK80 (top layer) 3.3V-PECL …….. …….. CK 15 Cables & PCK953 CK 80.0789 MHz Fanout Connectors LVPECL- Brdcst<7:2>, …….. …….. 3.3V crystal CK/2 to-LVTTL x 15 tbd BrcstStr, …….. Diff. PECL …….. CK/2 Fanout BCntRes, CMOS (top layer) LVDS 15 connectors L1A Fanout or diff on bottom 8 clks to TLKs + TPs PECL layer ? Board HTR 9U Front-panel space = 325 mm ; => space per connector ~ 21.5 mm Tullio Grassi <tullio@physics.umd.edu> 14 CMS ESR December 2002
TTCrx Mezzanine card Mezzanine card TTCrx TTCrx Mezzanine card • Very simple card: • 2 PMC connectors • TTCrx chip • TTC signal driver on motherboard • Will be sent out for prototype next week • Used by HTR, DCC, and Fanout cards 15 CMS ESR December 2002
Recommend
More recommend
