of large-scale facilities using software development and scientific - PowerPoint PPT Presentation

Maximising the scientific impact of large-scale facilities using software development and scientific computing. Jon Taylor

Outline: Scientific impact from Neutrons Role of software & Development Some key projects Considerations @ day 1 2

3

4

5

8

11

ANGSTROMS AND PICOSECONDS • electronic and magnetic structure PHYSICS • strongly correlated electron systems NANOMETRES AND NANOSECONDS • chemical structure and dynamics CHEMISTRY • structure/property relationships MICRONS AND MICROSECONDS • interfaces, micelles, etc. SOFT MATTER • protein structure and dynamics & BIOLOGY METRES AND MONTHS • engineering measurements, archeometry ENGINEERING & • tomography, residual stress, etc. MATERIALS SCIENCE 12

20

• High profile publications What is “Impact” • Shorter lead time to publish 22

• Shorter lead time to publish • Minimize barriers to publication • Maximise efficient use of beam time • Make life easier for users and staff • Remove single point failures 23

• High profile publications • Access to simulation software. • Help to use simulation software. SC generates a factor 3 in neutron citations correlates to a factor 2 increase in H 24

develop beam lines In order to grow our user community we must be able to service users who have no prior experience of “Proposal to publication support for scientific computing run experiments without support. ether neutron scattering, or scientific computation to maximise the impact of the science programme that does not requires a priori knowledge” Thanks I promise to If you need help get in touch. nature next 6 months 25

Pre Beamtime Beamtime Post Beamtime Data collection needs to be Flexible Simulation Experiment Data Data Data Proposal Data reduction needs to be Flexible Modelling Planning Collection Reduction Visualisation Interpretation Virtual experiment aided Live data view Analysis and interpretation are a barrier to publication planning Data driven collection Data collection needs to be “informed” Common data backplane Simplified remote data access cloud technology Common experiment and simulation data

Relevant Technologies & frameworks All open source :: develop within existing FWs Pandaas Sine2020 27

Acknowledgements Funding ing 28

Full Instrument description Workspace history Multiple workspace types Instrume nt Box controlled rebin on nD data Geometr y Live event view Algorithms event aware Slice viewer for 2D data Python + IPython + Numpy + SciPy ParaView OpenMP Instrument view User extendable 29

H2020 projects • PaNDAs Federated data storage and analysis infrastructure • SINE2020 :Data JRA (Thomas Rod) Automated Workflows and data analysis pipelines 32

SINE2020 World class S cience and I nnovation with N eutrons in E urope in 2020 • EU Horizon 2020 proposal (INFRADEV-4) • Five main partners: Facility Task Main topics: ILL+FZJ+PSI Mantid for continuum sources • Innovation based on ESS SANS (SASView) neutrons experiments FZJ Reflectometry (BornAgain) • Ready for ESS in 2020 ILL Modelling (nMoldyn) • Interoperable software for ISIS QENS (Mantid) data processing PSI Imaging (MuhRec/KipTool)

Interoperable Data Treatment Software for the European User Community 34

Supporting development of existing programs ESS will support (distributed) development of existing programs by hosting software development environments – Web sites, repositories, build servers, etc SASView and McStas will be the first two projects supported in this way

High availability Infrastructure Complexity metric + +

A not extensive list of experimental requirements Experiments Temperature / field / pressure dependent measurements of diffraction INS Data rate reflectometry & compute SANS. requirement Generation of 4D data volumes, step scan / continuous rotation mode Back projection reconstruction imaging Experiment set-up modes Real time Laue mode. Real time motion scan mode. Sample location and centring. Slit scans 39

Towards “Real Time” processing / Visualisation Modelling Considerations:: Data rate Compute availability Architecture Usability :: UX & UI 40

Lattice excitations experiment and empirical modeling Coherent lattice excitations from powders 41

GULP fitting cycle ~60s 42

Generation of 4D data volumes in RRM • Worst case 48 hrs for reduction 1x 200+ Gb file 1x 200+ Gb file or 2+ hrs 2-300 files per ei 2-300 files per ei 60s ~200mb each ~200mb each 43

• ~10 9 n/s/cm 2 • ~600k channels 44

schematic of control at a parallel event based source like ESS Instrument Data Network Machine timing Network Epics Network TRC TRC TRC SE IOCs Motions IOCs Chopper IOCs TRC TRC TRC TRC BDI BDI BDI Rotation BDI Mon Mon Ban Mon Cryo 1 3 k 1 2 Tsa mple Y Reg u Chopper Chopper Chopper Events Events Events 3 2 1 Vac BDI Ban k2 Speed 3 Speed 2 Speed 1 Slit 1 Blade 1 Delay 3 Delay 2 Delay 1 Blade 2 Balde 3 Table / Gonio Phase angle 3 Phase angle 2 Phase angle 1 Blade 4 BDI Park on/off Park on/off Park on/off Hor Gap Ban Ver Gap k 3 Veto set Veto set Veto set T sample T Regulation Veto active Veto active Veto active Phi Chi Theta Gas flow Field

Instrument Data Network Machine timing Network Epics Network Live Data reduction IC Client IC server side Live side visualisation Online Data Pandaas compatible reduction Aggregator type VM infrastructure Streaming file service Stream Management server Online Data reduction Streaming file ESS Instrument Data Stream service 46

Concluding remarks Software is essential for impact Collaborative development maximises impact Existing projects essential for ESS 47