pxd daq
play

PXD DAQ S. Lange (Giessen) for PXD DAQ team Hardware and firmware - PowerPoint PPT Presentation

Aug 17, 2022 •334 likes •1.06k views

This talk is a preliminary version. Apologies. Final version will be uploaded after the CHARM'2013 conference (08/31-09/04) at Manchester. PXD DAQ S. Lange (Giessen) for PXD DAQ team Hardware and firmware by IHEP Beijing, Bonn, Giessen,

  1. This talk is a preliminary version. Apologies. Final version will be uploaded after the CHARM'2013 conference (08/31-09/04) at Manchester. PXD DAQ S. Lange (Giessen) for PXD DAQ team Hardware and firmware by IHEP Beijing, Bonn, Giessen, Kavli IPMU, KEK, TU Munich Additional software and test MC data by DESY, KIT Karlsruhe, Prague, MPI Munich Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 1

  2. Belle II PXD DAQ DHHC „ONSEN“ (Online Selector Node) because data reduction factor ¸ 10 Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 2

  3. Data Reduction by Factor ≥ 10 ROI (Regions-of-Interest) from 1. DATCON SVD only track finder 2. HLT SVD + CDC track finder followed by logical OR. Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 3

  4. Status of DHH Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 4

  5. DHH prototype  FPGA: Virtex6 VLX75T  128 MB DDR2 memory for a MicroBlaze softcore CPU  SODIMM socket as interface  3x RJ45 for  ethernet,  timing (interface to FTSW board)  JTAG (for flash memory access on FTSW)  two SFP+ 6 Gbps optical transceivers as interface to ONSEN system  SFP for optical connection to DHHC as controller interface,  two Infiniband 4× connectors for receiving data from DHP

  6. New Design: DHH ATCA module not a 1:1 mapping anymore, supports load balancing Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ

  7. DHH/DHHC final ATCA-based system Design • AMC module, ATCA standard • Module programmed as DHH or DHHC • Load balancing 30% peak data rate reduction • Subevent building • VIRTEX6 XC6VLX130T FPGA • 2GB DDR3 • DHHC data throuput : 2.5 GB/s • First modules have been succesfully tested Production • 50 modules to be producced • 12 more modules are in production and to be delivered in September ATCA Carrier board • Schematic ready • PCB layout in progress Igor Konorov, Dima Levit

  8. Data processing in DHH

  9. Status of ONSEN ATCA based FPGA Compute Node Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 9

  10. ATCA Compute Node ATCA Compute Node ATCA Compute Node Version #1, 01/2008 Version #2.1, 12/2008 Version #2.1, 12/2013 Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 10

  11. Compute Node Version #3 Carrier Board, xTCA compliant Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 11

  12. New (2012): ATCA Compute Node Version #3 Daughter Board AMC (uTCA) Power Supply Board Carrier Board xTCA Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 12

  13. 2 x 2 GB DDR2 AMC Version 1.0 Problem: DDR2 RAM only operable with 200 MHz + 177 MHz AMC Version 2.0 Problem: AMC-AMC connection some RocketIO pins were routed to LVDS pins Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 13

  14. AMC v3.2 arrived in Giessen March 12, 2013 used during DESY test May13 64 MB Flash 4 x optical 6.5 Gbps GbE (RJ45) UART-to-USB Connector JTAG for MMC DC-DC converters (IPMI) (12V → 3.3 V) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 14

  15. Giessen Setup 5 ATCA Shelfs Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 15

  16. Problem with the Carrier Board  Virtex-4 switch FPGA on carrier board is connected to each AMC FPGA with 8 LVDS pairs (4 × duplex)  Some links do not work, because low-capacitance pins (no LVDS drivers) were used  Such pins were also used for output to the clock fan-out chip  The remaining links work, if one of the data links is used for the clock → makes carrier board redesign necessary → Not crucial for DESY Test (as switch FPGA not needed) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 16

  17. Status of DATCON Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 17

  18. DATCON Setup for Vienna Test June13 Michael Schnell (Bonn)

  19. Status at Bonn - 2 AMC cards v2 at Bonn, 1 nTCA shelf - Successful test of communication between FADC, FTB and DATCON with ~12 million events in Vienna - Implementation of lookup table and software for sending 'real' (simulated) strip data from the FTB - Full preprocessing chain in DATCON done, including - Decoding of FADC protocol - Strip clustering with simple center of gravity - Coordinate translation - Internal protocol for control and inter-AMC communication (e.g. transmission of coordinate data, update coord translation table...) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 19

  20. Status at Bonn - Simple tracking unit (only straight tracks) for the Giessen-Bonn test (planned for Sep 23-27, 2013) - Track extrapolation to PXD and ROI creation TODO: - ROI transmission module to ONSEN - Bit error test with new clock input (GREFCLK) - Backplane bit error tests - DDR2 memory access for testbeam at DESY Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 20

  21. Planned DATCON Setup for DESY Test Jan14 Michael Schnell (Bonn)

  22. Firmware Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 22

  23. Buffer management for 4 GB of DDR2 RAM (buffer for 5 seconds HLT latency) implemented in VHDL B. Spruck (Giessen) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 23

  24. ATCA Firmware ONSEN Firmware 1526 MB/s achieved AURORA 8B/10B 540 MB/s required 625 MB/s TCP/IP 95 MB/s only one receiver UDP 60 MB/s multiple receivers 271.000 pointers/s achieved 3 x 30.000 pointers/s required Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 24

  25. Status of ROI Firmware on ONSEN Reminder: ROI core receives ROIs from - DATCON - HLT then performes logical OR. ROI selector is implemented PARALLEL Format unpacker/decoder for 3 formats: DHH (DHP subformat, DCE subformat, RAW subformat) 16-bit aligned: 1 bit of row address in encoded in column addr Frequency must be doubled because of 16 bit → 32 bit unpacking Long-term stability 9 random events standalone FPGA core run for ~10 3 MC events (with random ROIs) and for some 10 Future: pixel recovery Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 25

  26. same DHH ID Integration of ROI Selector into Firmware different chip ID Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 26

  27. Integration of ROI Selector into Firmware s ome signals had to be delayed by 1-2 clock cycles for synchronisation Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 27

  28. Status of ROI core implementation for Jan14 DESY test: 41 hour test with 2xNPI(data, ROI) + ROI selector core + siTCP successful (zero errors observed) „debug mode“ switched on: ROIs are also sent out Integration into „full core“ (complete Onsen firmware) → ongoing work Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 28

  29. basf2 interface Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 29

  30. Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 30

  31. ≥9 Mill. pixel hits of „cold“ pixels Landau fit to a „cold“ pixel from DESY data in basf2 reformatted (new Jan14 format) PC-PC send/receive (socket) Bachelor Thesis → basf2 Klemens Lautenbach (Giessen) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 31

  32. Results from May13 DESY test beam Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 32

  33. Change between readout chains by re-plug infiniband cable Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 33

  34. T est beam in May with DHH prototype  1xDHP  1xDCD  1xFTSW  DHP interface 1.6 Gbps  Onsen Interface 3.125 Gbps Igor Konorov, Dima Levit

  35. T est beam in May with DHH prototype DHH Problem observed in the May 13 DESY test: scrambled data when DHP/DCD programmed by DHH Working scheme : programming by DHH EMULATOR and reading by DHH Reason : not correct programming sequence and of DCD by DHH Problem understood and being solved. Igor Konorov, Dima Levit

  36. ONSEN Setup in May13  1 uTCA Shelf (+ 1 spare)  1 AMC card (+ 1 spare) This is the new AMC card v3.2 which arrived in Giessen in March 2013.  No xTCA carrier board  Aurora 3.125 Gbps on optical link (not highspeed, because we didn't know beforehand if we would have the speedgrade -11 FPGAs)  No ROI selector Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 36

  37. Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 37

  38. Differences between DHH+Onsen and DHH-Emulator System  optical link, L= 30 m optical cable  Buffer management was used 256 kB buffer size x 4096 buffers = 1 GB  DATA WAS DUPLICATED - Writing data file - on ONSEN readout PC, - on BonnDAQ - on EUDAQ (at same time) Belle II Review, Sep 9-10, 2013 | S. Lange (Giessen) PXD DAQ 38

Recommend

More recommend