ONSEN ONSEN HLT/DatCON Merger and HLT/DatCON Merger and Trigger/ROI Distributor Trigger/ROI Distributor Björn Spruck Talk held somewhere Once upon a time after 30.4.2014 30.4.14, p. 1
Belle II PXD DAQ Belle II PXD DAQ (PC Farm) (PC Farm) (PC Farm) 30.4.14, p. 2
DHH Load Balancing DHH Load Balancing Full Subevent (5 DHH) to one output. Selected by trigger number! 30.4.14, p. 3
Event Wise View Event Wise View ONSEN *4 new 1 2 3 4 1 2 3 4 DHHC 5 6 7 8 5 6 7 8 .. .. .. .. .. .. .. .. 1 2 3 4 1 2 3 4 5 6 7 8 5 6 7 8 Each DHHC sees .. .. .. .. .. .. .. .. 1 2 3 4 1 2 3 4 1/8 of the whole PXD 5 6 7 8 5 6 7 8 .. .. .. .. .. .. .. .. 1 2 3 4 1 2 3 4 5 6 7 8 5 6 7 8 .. .. .. .. .. .. .. .. 4*4 DHH 4*4 DHH every 4th event every 4th event => 1/2 PXD => 1/2 PXD f.e. upstream part f.e. downstream part These are the = Carrier with 4 AMCs Event Numbers! = 4 AMCs served by one DHHC 30.4.14, p. 4
Load Balancing / Multiplexing by Event Number Load Balancing / Multiplexing by Event Number Be flexible, don't use “just” the lower two bits as selection. The following was decided by Igor, Dima and Björn: We use a lookup table with 64 entries. 6 lower bits of trigger number as index in table which selects then one of the four outputs Table can be changed “on the fly“ by Slow Control Easily adaptable if we cannot use all four outputs in a test Deactivate one output w/o loosing data Generate lower or higher data rates on one output Because of the delayed HLT trigger, modifying the table is only possible during a stopped DAQ (e.g. HLT queue/fifos must be completely empty!) 30.4.14, p. 5
Closeup of Merger Closeup of Merger D a t C o n H L T xFP AMC Me r g e r Carrier Board #0 D i s t r i b u t e s H L T T r i g g e r b y E v e n t N u m b e r i n e x a c t l y t h e s a m e w a y a s t h e D H H C d o e s i t . C o n fi g u r a b l e ! D i s t r i b u t o r D u p l i c a t e s t h e e a c h d a t a t o t w o o u t p u t s . ( e . g . f o r w a r d / b a c k w a r d ) S e l e c t s f w / b w R O I s w i t h i n d a t a ( o p t i o n a l ) Carrier Board Carrier Board Carrier Board Carrier Board #1 #2 #3 #4 Carrier Board Carrier Board Carrier Board Carrier Board #5 #6 #7 #8 30.4.14, p. 6
Results Results Implemented in VHDL including PLB registers for LUT configuration Tested with 1GEvents (32GB) with different LUT setup Checksums, events number selection and continuity are O.K. SITCP in → Distributor → (SITCP out 1, nirvana, nirvana, SITCP out 2) on an AMC board (Virtex 5). But Not checked with four real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA) 30.4.14, p. 7
Forward/Backward ROI Sorting Forward/Backward ROI Sorting ROIs for forward/backward part of PXD are distinguishable by their DHH-ID We can reduce the HLT data which is send to each node by sending only ROI data which the node needs Remark: Data rate is already low, just double the output (aka DESY test) will work, too. But: We would save some memory bandwidth on the Selector nodes. Results: Works. Checked with 1,6 GEvts Input (=570 GB) → 2*400 MEvts Output (=2*70 GB), Rate ~112 MB/s Input Checksum is recalculated as HLT/ROI content of package changes by selecting FW/BW ROIs only. But: Not checked with eight real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA) 30.4.14, p. 8
Closeup of Other Carriers Closeup of Other Carriers Carrier Board #0 Carrier Board Carrier Board Carrier Board Carrier Board #5 #6 #7 #8 D i s t r i b u t o r Carrier Board Carrier Board Carrier Board #2 #3 #4 Carrier Board D o u b l e r #1 D o u b l e r ^ 2 D o u b l e r D o u b l e r AMC #1 AMC #2 AMC #3 AMC #4 R e ma r k : A l l A MC i n o n e C a r r i e r p r o c e s s s u b - e v e n t s o f t h e s a me E v e n t N u mb e r . T h u s n o f u r t h e r s e l e c t i o n n e c e s s a r y n e e d e d . T h e d a t a j u s t h a s t o b e c l o n e s t o f o u r o u t p u t s ( o r d o u b l e d t w o t i me s ; d o u b l e d l i k e i n “ D E S Y t e s t ” ) O n e c o u l d s e l e c t f u r t h e r o n t h e D H H I D s , b u t t h a t s n o t n e e d e d , d a t a r a t e i s a l r e a d y l o w . ( n e v e r t h e l e s s , i t w o u l d s a v e me mo r y b a n d w i d t h . . . ) 30.4.14, p. 9
Recommend
More recommend
