Nikhef plans (and some comments from Jos Vermeulen) Frank - PowerPoint PPT Presentation

Nikhef plans (and some comments from Jos Vermeulen) Frank Filthaut, Paul de Jong, Milo Vermeulen

Proposed hardware setup 1 FELIX PC (SuperMicro) single CPU (3.6 GHz E5-1650v4 or 3.5 GHz E5-1650v2), 6 cores, hyperthreading; 16 or 32 GB memory (2 hosts available; upgrade from 16 GB to 32 GB possible) 8-lane PCIe Gen3 ➠ measured max. throughput > 6 GB/s HTG710 board, 6 input links ➠ input rate 57.6 Gb/s (payload: 45 Gb/s) emulate 6 WIB links using VC709 (optical splitter to create 6 links from 4) dual port 40 Gb/s Ethernet (or 56 Gb/s Infiniband) NIC (Mellanox), 8-lane PCIe Gen3 (3 available) 1 event builder PC data sink BoardReader PC (if needed) All hardware available (Nikhef DAQ test lab) 2

Block view VC709 HTG710 
 CPU CPU / CPU UDP TCP/IP 
 artDAQ custom WIB link 
 Board 
 / … Event 
 FELIX full 
 emulator Reader Builder 
 mode trigger FELIX PC PC ProtoDUNE triggered mode: house Board Reader on the FELIX PC (suggestion by Giovanna) handles both trigger (timestamp) selection and subsequent compression to be seen whether this is feasible (especially compression likely to be CPU intensive) Fall-back options: 1: additional computing power (separate host) for compression (or perhaps — but unlikely — even for timestamp selection) 2: determine maximum number of WIB links / FELIX PC (if not 6) 3

Block view VC709 HTG710 
 CPU CPU / CPU UDP TCP/IP 
 artDAQ custom WIB link 
 Board 
 / … Event 
 FELIX full 
 emulator Reader Builder 
 mode FELIX PC PC CPU Board 
 Reader Board 
 DUNE trigger-less mode: Reader PC house Board Reader on the FELIX PC and/or offload to additional computing power compression only; exact setup to be determined from throughput tests (should get a good idea from compression in triggered setup) 4

Progress Minimal hardware setup exists relying heavily on / using setup of core FELIX firmware team at Nikhef additional hardware can be bought quickly and at modest cost BoardReader: Milo (new PhD student) to begin ~ now experience with C++ but needs to get acquainted with the context Full mode emulator: firmware programming to begin soon driven also by ATLAS needs René Habraken (new engineer) 5

Comments (from Jos) on Eric’s slides p4 (throughput): suggest that the issue may not be with the number of lanes but with the use of the DMA controller p10: with ≤ 4 input links, cheaper VC709 should suffice in lieu of HTG710 also a target for FELIX firmware development p13: suggest cheaper motherboard / CPUs Supermicro X10SRA-F motherboard with 3.6 GHz E5-1650V4 CPU and 32 GB DDR4 2400 MHz ECC reg memory in 4 U Supermicro rack-mountable chassis (743T -665B) (in Amsterdam 1640 Euro ex. VAT, including 256 GB SSD). The slots can be configured as two 16-lane and one 8-lane Gen3 PCIe slot or as one 16-lane and four 8-lane slots (8-lane cards can be used in 16-lane slots) used both at Nikhef and at CERN 6

Throughput (from Jos) PCIe&throughput& One"card" Two"cards" ! Measurements&where&done&using&a& simple&counter&filling&the&PCIe&FIFO & ! For&test&purposes&only&:&single&shot&DMA& transfers & ! For&normal&operabon:&conbnuous&DMA& transfer&is&foreseen& JV,&Münich&Muon&Week,&15COctC2015& 18& 7