Camera Visualization System Requirements and Status JTM - March 2017
Visualization Requirements ● Camera image visualization is mainly aimed at diagnosing issues with CCDs during CCD/raft testing, focal plane integration, camera commissioning and operations ● Timescale is now through commissioning and operations ● Capabilities that are somewhat different from visualization for science ○ Near real time image display (as well as access to archive data) ○ Hold a data stream image for detailed analysis (under user control) ○ Able to scale from full-focal plane to individual amplifiers with real-time panning/zooming ■ Able to display with/without overscan ○ Ability to display, select, and analyze image regions by hardware entity (CCD, amplifier, overscan, data, etc .) ○ Perform HW diagnostic functions ( e.g. , pixel noise, CTE, … ) ○ Monitor HW performance (noise, gain, bad pixels). ■ Examples: histogram pixel distributions; Highlight hot/dead pixels on the image ○ Ability to run DM and user-defined algorithms on selected data in real-time ○ Ability to compare current data with archived data ○ Ability to overlay real-time trending quantities ○ Able to run from multi-panel wall mounted display to laptop
Visualization Plans ● After reviewing other tools including a camera specific solution that had been developed previously we decided to adopt IPAC firefly ○ Plan is to use firefly as a toolkit, and to build camera specific functionality on-top/around firefly library ○ Our hope was that by using firefly we would be giving IPAC early feedback on a tool which could be used by other parts of the project ● We have only a limited budget for visualization so have mainly used “contributed” and/or “student” manpower ○ Jon Thaler and John Hart (Illinois) ○ Students (never more that three at a time): Neelan Coleman, Vikas Agartha, Bing-Jul Ho, Dingcheng Yue, Yutong Wang, Joe Pagliuco, Wei Ren, Ting Huang. ○ Stuart Marshall and Tony Johnson (camera integration) ○ Tatiana Goldina (IPAC liaison)
Visualization architecture Diagnostic Cluster (~1 core/CCD) Visualization Server CCS Rafts 1 Visualization Rest API Ingester CCS Rafts 2 DAQ “In-memory” CCS Rafts 3 “2-day-store” Fits File CCS Rafts 4 Python/ Firefly CCS Rafts n DM I&T Data Storage Firefly Web Client Camera Javascript extensions Architecture is design to minimize latency from image acquisition to display.
Current Status ● All camera computing equipment including visualization equipment in installed at SLAC (IR2). ○ We expect to get the first rafts from BNL this month ○ Firefly + camera extensions are installed and being commissioned now. ● We will also deploy firefly at BNL for raft level test stand (TS8) this month
Visualization Capabilities
Camera requirements for DM/IPAC support ● Over past 2 years we have got good support from IPAC team ○ Tatiana attended our weekly meetings with students (typically <30 minutes) and was extremely useful at helping students overcome roadblocks and understand firefly capabilities ○ Occasional working meetings with IPAC team to discuss roadmap ● We have very few requirements beyond what would be required for general LSST visualization ○ Timescale is different -- we plan to start using this now ■ Early users are likely to give plenty of feedback ○ We may (will) encounter some bugs/usability issues before other users ■ Help with getting these fixed, or finding workarounds is required ○ Some items which are not requirements, but which would be nice to have ■ Bottlenecks in current implementation (need to write single giant FITS file) ● We would like a supported API for feeding data to firefly ○ We would provide the camera interface to API ■ Improved python lifecycle (currently each operation requires re-reading FITS file) ● We strongly believe continued cooperation between camera and IPAC/DM gives opportunity for mutual benefits and benefits to LSST ○ We understand DM budget is tight, but do not believe this will require large divergence from what would be required for DM only
Backup links ● Live camera/firefly demo: http://lsst.cs.illinois.edu:8080/static/index.html ● Camera Visualization Concept of Operations: https://community.lsst.org/uploads/default/original/1X/8ae07c6cad063538548 b945a00060250f029985a.docx
Recommend
More recommend
