DSP at SAO and a Study of Next Generation ALMA Digital Processing Andr´ e Young Harvard-Smithsonian Center for Astrophysics ALMA Future Science Development Program Workshop Charlottesville, VA 24 August 2016
Overview This talk is about a study of next generation ALMA digital processing... Credit: Jonathan Weintroub Credit: EFE/Ariel Marinkovic ...but we begin the recent correlator upgrade at the SMA: high-altitude site, challenging environment to keep equipment cool power is expensive, reduce consumption as far possible keep up with rapidly evolving technical landscape, while still supporting / running legacy systems in parallel 2
The Submillimeter Array Correlator Upgrade Original ASIC correlator built on 90s tech 90 correlator boards: Process up to 2 GHz per Rx 2 GHz IF split into six 328 MHz blocks, each block into four 82 MHz chunks 812.5 kHz finest uniform spectral resolution without loss of bandwidth MIT Haystack MarkIV (various modes to allow finer resolution correlator board, 32 non-uniformly, or by trading bandwidth) XF-corr ASICs 3
The Submillimeter Array Correlator Upgrade SWARM: SMA Wideband Astronomical ROACH2 Machine 8x 1U rack components (1 quadrant): Process 2 GHz per Rx full usable 2 GHz in single baseband 2 channels per ROACH2 (2x 5Gsps ADC) 140 kHz spectral resolution, uniform across the entire band ROACH2, built around 1 full Stokes without loss of resolution Xilinx Virtex-6 FPGA 4
The Submillimeter Array Correlator Upgrade Forest of lines in Orion BN/KL, two quadrants of SWARM. Data from 14 Aug 2016. (Primiani+ 2016, in prep.) 5
The Submillimeter Array Correlator Upgrade SMA correlator room digital racks (left) and analog racks (right) ASIC correlator essentially replaced with single SWARM quadrant digital equipment space reduced by a factor of 6 wideband digital system greatly reduces analog system 1x 2 GHz SWARM baseband = 24x 82 MHz ASIC basebands ⇒ analog equipment space reduced by factor ∼ 8 large reduction in power consumption from 25 kW (excl cooling) to 2 kW (incl cooling) Full SWARM system, 8 GHz per Rx expected around end 2016 6
SWARM: FPGA Logic SWARM built on the CASPER packetized correlator architecture, within each ROACH2 F-engine, 16384 channels per Rx from single antenna X-engine, 2048 channels per Rx for all baselines 10GbE crossbar switch for F-to-X data distribution 7
SWARM: FPGA Logic Very high utilization of FPGA resources full rate system clocked at 286 MHz, staged in 11 th s to meet timing closure: 6/11, 8/11, 10/11 (compatibility with legacy clocking) currently at 11/11 rate, with timing score of ∼ 3000 using -2 speedgrade parts (picoseconds of negative slack required to route signals at given clock rate, operating conditions dependent, helps to run systems as cool as possible) Xilinx Virtex-6 (XC6VSX475T) Occupied slices 97% Slice registers 44% Slice LUTs 82% DSP48Es 44% BRAM 70% IOBs 89% 8
SWARM: Built-in Phased Array & VLBI Capability Combine connected-element array into equivalent larger single dish in-situ phase calibration to correct time- and direction-dependent atmospheric delay signal-/noise-space decomposition of the correlation matrix (largest eigenvalue → eigenvector is the phasing solution) Phasing feedback loop in SWARM Phasing efficiency during EHT observation, 4 April 2016 9
Next Generation ALMA Digital Processing Study Present ALMA correlator technology is ∼ 10 years old. What ALMA digital processing system can be built with an upgrade to current / near-future technology? Smithsonian Astrophysical Observatory Joint ALMA Observatory Universit´ e de Bordeaux National Radio Astronomy Observatory ngALMA correlator & phased array kick-off meeting, May 2016 @ SAO LTR: J. Weintroub, M. Rupen, R. Lacasse, B. Carlson, M. Hecht, J. Hickish, National Research Council Canada R. Wilson, G. Crew, C. Langley, S. Doeleman, AY, L. Blackburn, A. Baudry, A. Saez, S. Ashton, R. Escoffier, L. Greenhill. University of California Berkeley + R. Primiani, D. Herrera, J. Test, K. Young, L. Matthews. MIT Haystack Observatory 10
ngALMA Study: Project Overview and Timeline One-year study Monthly teleconferences Monthly progress reports Bookend in-person meetings (May ’16 @ SAO, Feb ’17 @ NRCC) Mid-term two ‘busy weeks’ @ SAO (11/16 visit A. Saez & D. Herrera) 04/16 05/16 06/16 07/16 08/16 09/16 10/16 11/16 12/16 01/17 02/17 03/17 04/17 Start-of-study Kick-off meeting Science requirements System architecture & breakdown Subsystems analyses Integration Concluding meeting Final reports & project end 11
ngALMA Study: Baseline Science Requirements Identify requirements to enhance existing / enable new science Target requirements compared to existing system: Existing * Proposed 1 Number of antennas 72–80 64 2 Maximum baseline 300 km 30 km 3 Total bandwidth 32 GHz / pol 8 GHz / pol 4 Baseband channel 12–14 GHz 2 GHz 5 Input / correlator sample resolution 4-bit / 4-bit 3-bit / 2- or 4-bit 6 Finest spectral resolution 1 kHz ≥ 3.8 kHz 7 Number of channels ≥ 1e6 / pol 8 Integration / read-out time 1 ms (auto-corr) 1 ms (auto-corr) 16 ms (cross-corr) 16 ms (cross-corr) 9 Independent subbands > 16 10 Subarrays 5–7 11 Phased array beams 2–4 12 Built-in VLBI capability * Escoffier et al 2007 12
ngALMA Study: System Architecture and Work Breakdown Translate science requirements to system specifications, assuming receive raw digitized baseband data as input deliver raw correlator / phased array data as output matched up/down-stream analog/digital capabilities will be available FX-architecture favored (over FXF-, XF-) Project onto technology for start-of-construction by Q2 2022 cost & performance estimates Break down into subsystems (work packages) smaller working groups subsystem specifications investigate platforms & architectures 13
ngALMA Study: System Architecture and Work Breakdown Work packages / subsystems Science require- Define a set of baseline science requirements (two slides ments ago...) F-engine Identify platform, CPU vs GPU vs FPGA vs ASIC Determine architecture for given platform Corner-turn Identify platform, backplane vs ethernet Consider interface to F-/X-engine X-engine Identify platform, CPU vs GPU vs FPGA vs ASIC Determine architecture for given platform Phased Array Beamformer design and integration VLBI capability Staging Logistics of assembly, running in parallel with existing correlator, etc 14
ngALMA Study: Intermediate Outcomes Only so much done in a year... ⇒ Identify prototyping study areas ...with limited person-power ⇒ Enlarge the collaboration Possible areas to fast track to project status include: 1 High-speed ADCs and interfacing 26 GSa/s & Virtex-7 port to SKARAB (a.k.a. ROACH3) [UCB] 2 F-/X-engine subsystems implementation Prototyping on latest high-performance tech (Xilinx Ultrascale+, NVIDIA Tesla P100, 100GbE, ASIC) 3 Proof-of-concept wideband pipeline SMA processes 4x 2 GHz, natural testbed for 1x 8 GHz Phased project towards 16 GHz (ngALMA) Q2 16 Q2 17 Q2 18 Q2 19 Q2 20 Q2 21 Q2 22 ngALMA Prototyping / PoC studies Projected start-of-construction 15
Conclusions Upgrade to SMA correlator yielded great performance benefits Can these benefits be reproduced for a future ALMA system? ...ngALMA digital processing study to answer that, plus Identify areas for fast-track prototyping Promote these to ‘project’ phase with more team members SMA receivers widebanded to deliver 14 GHz SMA a natural testbed of ngALMA digital systems High-altitude site, prototypes can be fielded and commissioned there for later deployment at ALMA 16
Recommend
More recommend
