Data Acquisition System for the PENeLOPE Experiment using the Unified Communication Framework Dominic Gaisbauer , Igor Konorov, Dymtro Levit, Prof. Dr. Stephan Paul Technische Universität München Institute for Hadronic Structure and Fundamental Symmetries Novosibirsk, March 3rd, 2017
PENeLOPE • P recision E xperiment on Ne utron L ifetime absorber O perating with P roton E xtraction movement • Will be located at the Forschungs- mechanism Neutronenquelle Heinz Maier-Leibnitz (FRM II) • Magneto-gravitational trap for ultra-cold proton detector neutrons 2.5 m • Aiming for a precision of ± 0.1 s • Measuring protons and neutrons outer pressure vessel helium vessel storage walls (electropolished) Dominic Gaisbauer (TUM) | Novosibirsk 2017 2
PENeLOPE - Proton Detector Requirements • Protons are guided via magnetic and accelerated by electrical field • Complete detector and electronics on -30 kV • Detector at 77 K • Electronics at 300 K • Active area of 0.23 m 2 • Peak event rate including margin: 130.000 events/s + bg ≈ data rate: 500 Mbit/s • Background is < 1/Ch/s Dominic Gaisbauer (TUM) | Novosibirsk 2017 3
PENeLOPE - Hamamatsu S11048 APD 6.8x14 mm 2 active area • 18x9 mm 2 size • • 4.4 %/V gain sensitivity to voltage at a gain of 100 • Operational voltage of 365 V to 440 V • No epoxy window since low energy protons would be absorbed • Commissioning tests with samples done • Beam test with final detector mid of 2017 30 keV proton spectrum Dominic Gaisbauer (TUM) | Novosibirsk 2017 4
PENeLOPE - Proton Detector Readout Concept UCF Inside of Outside of 24 PreAmp, Cryostat Cryostat APDs Shaper, ADC SDU01 4x Signal ... 30 kV high Ground Processing 24 PreAmp, Voltage Potential Slow APDs Shaper, ADC Control PC 14x Passive IPBus NAC ... Optical Interface UDP 24 PreAmp, Splitter APDs Shaper, ADC SDU14 DAQ PC 4x Signal ... 24 PreAmp, Processing APDs Shaper, ADC Slow • 500 Mbit/s data rate Control Bias • 1496 channels Dominic Gaisbauer (TUM) | Novosibirsk 2017 5
PENeLOPE - Proton Detector Readout Concept SEP UCF Inside of Outside of 24 PreAmp, Cryostat Cryostat APDs Shaper, ADC SDU01 4x Signal ... 30 kV high Ground Processing 24 PreAmp, Voltage Potential Slow APDs Shaper, ADC Control PC 14x Passive IPBus NAC ... Optical Interface UDP 24 PreAmp, Splitter APDs Shaper, ADC SDU14 DAQ PC 4x Signal ... 24 PreAmp, Processing APDs Shaper, ADC Slow Control Bias Dominic Gaisbauer (TUM) | Novosibirsk 2017 6
Unified Communication Framework (UCF) • Originates from the SODA time distribution system developed for the PANDA experiment • Single high-speed serial link for data, slow control, trigger, and timing information implemented on FPGAs • Up to 64 different communication channels (e.g. timing, slow control, Data, JTAG, I2C, SPI, TCP, UDP…) • Fixed latency for one channel • Priority handling for all channels • Self recoverable after connection losses • Independent from physical layer TCS Data Data Data Front-end Electronics/ Detector/Sensor FPGA Stage PC/DAQ FPGA Stage IPBus IPBus Dominic Gaisbauer (TUM) | Novosibirsk 2017 7
UCF – Example Topologies • Point-to-Point topology: • Multiple or single 1:1 connections • Experiments with high data rates, … • Bidirectional on all channels Data Data TCS Front-end Slave Master Slow Control DAQ-PC Data TCS TCS Front-end Slave Master TCS Slow Control … Data Slow Control PC TCS Front-end Slave Master Slow Control IPBus Dominic Gaisbauer (TUM) | Novosibirsk 2017 8
UCF – Example Topologies • Star-like topology: • Single 1:n connections • Experiments with low data rates, time distribution systems … • Slaves share link in time division manner • Bidirectional on all channels Data TCS Data Front-end 0 Slave DAQ-PC Slow Control Data TCS TCS Front-end 1 Slave TCS Splitter Master Slow Control Data IPBus TCS Slow Front-end Slave Slow Control Control PC 255 Dominic Gaisbauer (TUM) | Novosibirsk 2017 9
UCF – Example Topologies • Hybrid topology: • Combination of point-to-point and star-like topologies • Bidirectional on all channels Data TCS Data Front-end 0 Slave Slow Control Splitter Master DAQ-PC Data TCS Front-end Slave 255 Slow Control … TCS TCS Data TCS Front-end 0 Slave Slow Control Slow Splitter Master Control PC Data TCS Front-end Slave IPBus 255 Slow Control Dominic Gaisbauer (TUM) | Novosibirsk 2017 10
UCF – Low Layer Protocol • Backbone of UCF • Handles communication and initialization • 8b/10b encoding scheme • 10b K-characters for control and synchronization • Protocol frames consist always of several character sequence: • Start of frame • Type of the message (either specific destination or broadcast) • Protocol identifier • Payload • Remainder defining the valid bytes in the last transmission • End of frame SOF TYPE ID PAYLOAD PAYLOAD ... PAYLOAD REM EOF Dominic Gaisbauer (TUM) | Novosibirsk 2017 11
UCF – Initialization • Fixed phase synchronization by sequence of two defined K-characters ( x”BCDC”) • Synchronization character will be send for specific time to let the slaves synchronize • Attached parties are scanned by sending an initialization frame containing different DNAs and waiting for response • DNA can be taken as the serial number of an FPGA • Unique ID and IP assignment for all connection parties ... BCDC SOF ID DNA ... DNA EOF BCDC ... Dominic Gaisbauer (TUM) | Novosibirsk 2017 12
UCF – Priority Handling • All 64 communication channels have different priorities • Protocol 0 has the highest and then it cascades down to the protocol 63 which has the lowest priority • Frames with higher priority can always interrupt lower priority frames • Maintains fixed latency for the timing channel SOF TYPE ID x” 0001 ” x” 0203 ” SOF TYPE ID x” 0001 ” x” 0203 ” x” 0405 ” x” 0600 ” REM EOF x” 0405 ” x” 0000 ” REM EOF Dominic Gaisbauer (TUM) | Novosibirsk 2017 13
UCF – User Interface and Configuration • All channels are addressed via the standardized ARM AMBA AXI4 Stream interface • Leads to easy interfacing with other IP-Cores • Configuration of all parameters with a generic directly in the top module instantiation: • Link speed • Topology • Device type (Spartan6, Virtex6, Artix7) • …. Dominic Gaisbauer (TUM) | Novosibirsk 2017 14
UCF – Tests and Measurements • Point-to-Point topology with 1 slave and 1 master • 2.5 Gbit/s link speed • Virtex 6 as slave and master • Recovered clock jitter ( σ ) of 23 ps • Long term test with 4 different protocols and link utilization of 99 % over two weeks Dominic Gaisbauer (TUM) | Novosibirsk 2017 15
UCF – Tests and Measurements • Star-like topology with 12 slaves and 1 master • 1.25 Gbit/s link speed • Spartan 6 FPGA as slave and Virtex 6 as master • Switching time of 16 µs (includes character transmission and synchronization) • Long term stability test with 99 % link utilization over two weeks • JTAG over UCF • IPBus over UCF Tran ansm smissi ssion on Effici cien ency cy Time e [µs] s] [%] %] 25000 99,93 10000 99,84 1000 98,42 500 96,90 100 86,20 Dominic Gaisbauer (TUM) | Novosibirsk 2017 16
Conclusion • Developed IP-Core providing unified communication of up to 64 channels via a single optical link • Fixed latency for one channel (23 ps clock jitter) • Standardized ARM AMBA AXI4 Stream interface for user • Multiple 1:n and 1:1 connections possible • Typically 98 to 99 % link utilization efficiency for star-like topologies (16 µs switching time) • JTAG over UCF • IPBus over UCF Dominic Gaisbauer (TUM) | Novosibirsk 2017 17
Thank you for your attention
Recommend
More recommend
