Talk Online Image Alignment using the EDNA Framework

Presented by Jerome Kieffer (ESRF) in Scientific track 2011 on 2011/08/28 from 12:00 to 12:30
Abstract

author:J. Kieffer, O. Svensson, V. A. Solé, B. Fayard

EDNA is a Python framework in for developing plugin-based applications [1] especially for online data analysis of X-ray experiments developed in collaboration between synchrotron facilities [2]

EDNA provides:

Modularity

The components are organized in a logical class hierarchy that makes it straightforward to develop new functional plugins by deriving them from the appropriate parent. Two families of plugins have been designed: the first branch contains the execution plugins that are responsible for the execution of a particular action (e.g. execution of third-party software); the second branch contains the control plugins that are responsible for the data flow (propagation of the data), the workflow (sequential or parallel execution of appropriate execution plugins) and the error-tracking mechanism.

Datamodel facility

The EDNA kernel provides a datamodel toolkit that allows the construction of elaborate data models needed by advanced applications. In addition, it allows the design of unitary data models that can be unit-tested, so that a plugin can be launched and tested independently of any application context. The framework also provides code generation machinery allowing automatic code generation from datamodel to Python code.

Testing framework

A testing framework has been developed and integrated with the kernel in order to test applications and individual components easily and efficiently. In order to ensure the reliability and robustness of the components, the testing framework provides all the necessary utilities to check a class and a plugin either in a unitary manner or by testing its execution in an application context. These families of tests can be automatically launched via test suites, thus allowing a high degree of confidence when implementing new components or re-implementing features of existing components, and permits continuous integration.

FullField X-Ray Absorption Spectroscopy

This framework ensures rapid development and integration of scientific pipelines in which robustness and quality play a fundamental role. After the first prototype, designed for macromolecular crystallography experiments [2] (being in production now at several synchrotrons), EDNA became widely used for all kind of data reduction on synchrotron beamlines. The presentation will focus on the latest development done in EDNA to go towards data analysis using directly python libraries, mainly numpy, scipy and h5py; for doing automatic image alignment for the FullField XANES experiments at ESRF beamline ID21 [3]. This pipeline consists of a set of plugins for image manipulation (dark current and flat field correction), image offset measurement (by FFT correlation) and image shift using the ndimage module of scipy.

References

  1. EDNA site: http://www.edna-site.org
  2. “EDNA: a framework for plugin-based applications applied to X-ray experiment online data analysis.” M.-F. Incardona, G. P. Bourenkov, K. Levik, R. A. Pieritz, A. N. Popov and O. Svensson; J. Synchrotron Rad. (2009). 16, pp 872-879 [ doi:10.1107/S0909049509036681 ]
  3. “Sub-micron hyperspectral X-ray imaging of heterogeneous rocks and geomaterials: applications at the Fe K-edge.” V. De Andrade, J. Susini, M. Salomé, O. Beraldin, C. Rigault, T. Heymes, E. Lewin and O. Vidal; Analytical Chemistry, submitted.