DM System Design and LDM-148 KIAN-TAT LIM, DM SOFTWARE ARCHITECT - PowerPoint PPT Presentation

DM System Design and LDM-148 KIAN-TAT LIM, DM SOFTWARE ARCHITECT

What Are We Building? Title Text A system composed of services that need to be operated. • Note that not all services built by DM or using DM software are operated by the Data Products and Processing Department; some are operated by the Science Operations Department and some by the Observatory Operations Department. Running on software that needs to be released as a product as well as data center infrastructure that needs to be maintained. Everything that follows is "to the best of my current understanding"; it may be wrong and should be challenged if wrong now or in the future. 2 Joint Technical Meeting - 2017-03-06 - Glendale

Service and Payload Components Community Alert Brokers Alert Users Data Management System Level 1 Domain Level 2 Domain Level 2 Monthly Quality Alert Prompt Alert Mini- Calibration Control Broker Processing Broker Payload Observatory Feed Annual Telemetry Calibration Calibration Gateway QA Payload Payload OCS Daily Cal. Template DRP Driven Update Alert Generation Payload Batch Payload Production Payload Image and Payload EFD Archiving Level 1 Calibration Data Offline DayMOPS Quality Products Release Processing Payload Control Production Production Data Backbone Data Access Centers (x2) Analysis and Developer Support Science Science Platform Platform Science Data Bulk Data (Commiss.) (Sci. Verif.) Platform Access Distribution (DAC) Web Svcs Developer Integration Services & Test Other Data EPO Partners Users 3 Domains are not exactly enclaves or centers

Software "Stack" obs_lsstSim obs_decam obs_subaru obs_cfht Plugins meas_extensions_ meas_extensions_ meas_modelfit shapeHSM convolved meas_algorithms meas_deblender meas_astrom pipe_tasks ip_diffim jointcal obs_base meas_base ip_isr Frameworks pipe_base coadd_utils afw cameraGeom coord detection display geom image math table and Utilities Middleware pex_exceptions, pex_config base, utils daf_base log 4 Middleware/utilities, frameworks, and "plugins" (or independent computation packages) "obs_" packages are more middleware configuration than computation

Middleware and Infrastructure Control and Orchestration Command Line SUIT Activator Orchestration Activator Activator Task Framework Activator (base) Files Data Backbone SuperTask Data Butler Qserv Task RDBMS MultiNodeTask MultiCoreTask Config API Logging API API API Provisioning & Authentication Resource Monitoring Science Algorithms & Authorization Management 5 How do Stack apps become services? Task framework, control and orchestration, other key infrastructure

Future of LDM-148 Brief summaries of the functions of each component Interfaces between components and outside the subsystem Links to requirements in both the DM System Requirements (LSE-61) and Interface Control Documents (ICDs) Partial draft available now at https://ldm-148.lsst.io/v/draft Components complete by end of March Interfaces and linkages complete by DM Review dry run 6 Joint Technical Meeting - 2017-03-06 - Glendale