Program of the workshop Wednesday: 9:00-10:00 Introduction to pyFAI – 10:00-10:30 Software installation on laptop computers – 10:30-11:00 Coffee break – 11:00-12:30 Demonstration of the user interfaces – 12:30-14:00 Lunch at the canteen – 14:00-15:00 Spotlight on application: beam-line presentation – 15:00-16:00 Discussions about needs – 16:00-16:30 Coffee break – 16:30-17:30 Tutorials: Jupyter – Thursday: 9:00-10:00 Tutorials: geometry, detector distortion and calibration – 10:00-10:30 Coffee break – 10:30-12:30 Tutorials: multi-geometry, goniometer, inpainting – 12:30-14:00 Lunch at the canteen – 14:00-17:00 Coding camp – PyFAI 07/06/2017 Page 2
Fast azimuthal integration … in Python While speed is only needed at large facilities … … proper calculation is needed for any scientific application PyFAI 07/06/2017 Page 3
PyFAI workshop General introduction to PyFAI PyFAI 07/06/2017 Page 4
Introduction to Azimuthal integration Allows the use of area detectors for ● Small angle scattering – Powder diffraction, PDF, ... – Better harvesting of X-ray photons (large solid angle) ● Azimuthal integration The devil is hidden in the details (of implementation) ● But many other tools exists: PyFAI is: ● - FIT2D Open source – - DataSqueeze Open to contribution – - XRDUA Open to discussion – - Foxtrot Free – - Maud Fast – - GSAS-II PyFAI 07/06/2017 Page 5
PyFAI is fast PyFAI 07/06/2017 Page 6
Concepts in PyFAI Image ● 2D array of pixels as numpy array read using silx, fabio, h5py, ... Azimuthal integrator: core object ● powder diagram using integrate1d ● “cake” image, azimuthally regrouped using integrate2d ● Detector ● Calculates the pixel position (center and corners) ● Calculate the mask ● Contains the flat-field correction. ● Geometry ● Position of the detector from the sample & incoming beam → saved as PONI -file http://www.silx.org/doc/pyFAI/pyFAI.html#experiment-description PyFAI 24/04/2017 Page 7
About the geometry used in PyFAI ● 3 distances in meter: dist, poni 1 , poni 2 PONI- file ● 3 rotation in radians: rot 1 , rot 2 , rot 3 ● wavelength / energy Area From the sample's point of view, detector Looking at the detector : d 1 rot 1 ↑ : move detector to the right rot 2 ↑ : move detector downwards rot 3 ↑ : move detector clockwise d 2 rot 1 S Point Of Normal Incidence : c a t PONI = (P 1 , P 2 , P 3 =0) t e r x 1 e d b e a m d 3 Distance x 2 2θ Detector's origin: Incoming lower left, looking from rot 3 X-Ray beam x 3 the sample Origin : sample position PyFAI 07/06/2017 Page 8
What happens during an integration 1) Get the pixel coordinates from the detector, in meter. There are 3 coordinates par pixel corner, and usually 4 corners per pixel. 1Mpix image → 48 Mbyte ! 2) Offset the detector's origin to the PONI and rotate around the sample 3) Calculate the radial (2 θ ) and azimuthal ( χ ) positions of each corner 4) Assign each pixel to one or multiple bins. If a look-up table is used, just store the fraction of the pixel. Then for each bin sum the content of all contributing pixels. 5) Histogram bin position with associated intensities 6) Histogram bin position with solid angle (and other normalization) 7) Return bin position and the ratio of sum of intensities / sum of Ω PyFAI 24/04/2017 Page 9
Calibration in pyFAI The determination of the geometry is also known as calibration ● The prerequisite is: – detector geometry and mask, ● calibrant (LaB 6 , CeO 2 , AgBh, …) ● wavelength or energy used ● Only the position of the detector and the rotation needs to be refined: – 3 translations: dist, poni 1 and poni 2 ● 3 rotations: rot 1 , rot 2 , rot 3 ● It is divided into 4 major steps: ● 1) Extraction of groups of peaks 2) Identification of peaks and groups of peaks belonging to same ring 3) Least-squares refinement of the geometry parameters on peak position 4) Validation by an human being of the geometry PyFAI assumes this setup does not change during the experiment ● Tutorial: ● http://www.silx.org/doc/pyFAI/usage/cookbook/calibrate.html PyFAI 24/04/2017 Page 10
PyFAI is a library & an application ... ≠ Library Graphical application Easier to use – Re-usable code – Looks better – Needs the definition of an API – Only one application – Faster to develop – Harder for re-use – Easier to test and maintain – PyFAI is itself relying on the Scientific Python stack: ● NumPy – SciPy – Matplotlib – H5Py – Cython – FabIO – + PyQt5 + silx PyFAI 24/04/2017 Page 11
Examples of other application relying on pyFAI NanoPeakCell: Serial crystallography pre-processing ● Nicolas Coquelle, IBS Grenoble – PySaxs: data analysis for SAXS experimental station ● Olivier Tache, CEA Saclay – Dpdak: online data analysis for Saxs data ● Gunthard Benecke, Petra III – Dioptas: offline data analysis for high pressure diffraction ● Clemens Percher, APS → Germany – Bubble: online data analysis for Saxs/Waxs data ● Vadim Diadkin, Dubble & SNBL CRG beamlines – Project for materials and strain analysis ● Jozef Keckes, Loeben university, Austria – xPDFsuite ● Prof. Simon Billinge, U. of Columbia – http://www.silx.org/doc/pyFAI/ecosystem.html PyFAI 24/04/2017 Page 12
User community of pyFAI PyFAI is used in most European and American synchrotons/FELs ● PyFAI mailing list subscribers grouped by country ESRF France Germany Google/hotmail United Kingdom System Spain USA Italy Sweden Netherlands User support is provided via the mailing list: pyFAI@esrf.fr ● Direct contact with authors is discouraged – http://www.silx.org/doc/pyFAI/project.html#getting-help PyFAI 24/04/2017 Page 13
Layers in pyFAI Applications level: ● GUI applications: pyFAI-calib, pyFAI-integrate, diff_map – F Scriptable applications:pyFAI-average, pyFAI-saxs, pyFAI-waxs, diff_tomo, … – l e E x a Python interface: ● i s b e Top level: azimuthal integrator – i Mid level: calibrant, detector, geometry, calibration – l o i f Low level: rebinning/histogramming engines (Cython or OpenCL) – t y u s e Question: how to define the right balance ? ● It is up to you ! PyFAI 07/06/2017 Page 14
Silx-kit The silx ecosystem PyFAI 07/06/2017 Page 15
silx-kit : join efforts, share the maintenance Jérôme Kieffer | data analaysis unit | ESRF 07/11/2016 Page 16
Management of the silx-kit project Public project hosted at github ● https://github.com/silx-kit/silx Continuous testing ● Linux, Windows and macOS Nightly builds ● Debian packages – Weekly meetings ● Quarterly releases ● Code camps before release ● Continuous documentation ● http://www.silx.org/doc/silx/ Jérôme Kieffer | data analaysis unit | ESRF 07/11/2016 Page 17
silx-kit project and the silx library Mainly Pierre Knobel … Mainly Jérôme Kieffer Mainly Thomas Vincent … and Valentin Valls Mainly Henri Payno Jérôme Kieffer | data analaysis unit | ESRF 07/11/2016 Page 18
Dahu: Online data analysis server Dahu is a lightweight plugin based framework... ... technically a JSON-RPC server over Tango written in Python The dahu server executes jobs ● Available on https://github.com/kif/dahu Each job lives in its own thread. – Each jobs execute one plugin – The job is responsible for de/serializing JSON string coming from Tango – Plugins are Python classes or functions ● Plugins are dynamically loaded from python modules – Plugins have a single input and output: simple dictionaries. – Jobs can be re-processed off-line ● Lightweight means limited overhead: ● 1 µs for a dummy plugin execution – 150 µs for a dummy job – <0.3 ms when called from Tango – Jérôme Kieffer | data analaysis unit | ESRF 07/11/2016 Page 19
Applications in pyFAI Jérôme Kieffer | data analaysis unit | ESRF 07/11/2016 Page 20
Description of applications in pyFAI: Preprocessing ● Mask drawing tool ● Calibration ● Integration ● Diffraction mapping ● Other features ● PyFAI 07/06/2017 Page 21
Image pre-processing: pyFAI-average A tool for filtering a stack of images : ● Used to merge multiple input images (can be a multiframe nexus) – Merging methods available: – min, max, mean, std, median, sum, quantiles, cutoff Correct for dark-current & flat-field – Normalize for a monitor value (from headers) – Exports in multiple formats (see FabIO) – Can be used to convert image format (NeXus → TIF) ● http://www.silx.org/doc/pyFAI/man/pyFAI-average.html PyFAI 07/06/2017 Page 22
Mask drawing tool: pyFAI-drawmask Click here when your done http://www.silx.org/doc/pyFAI/man/pyFAI-drawmask.html PyFAI 07/06/2017 Page 23
Recommend
More recommend
