Algorithmic and methodological tools

Publié le 14 janvier 2021 par Mickaël Coriat.

This service task of SNO SKATE includes the development of algorithmic and methodological tools for SKA and its pathfinders, as well as for the European SKA Regional Centre.

The development of algorithmic and methodological tools is a common task for all instruments involved in the service. The scientific success of SKA and its pathfinders depends critically on these developments, which are essential for data calibration, processing of radio interference emissions, wide-field imaging, and detection of transient phenomena. The analysis of SKA’s large surveys will require that some of the
required tools are made available. The French teams have recognised expertise in this field, but they are not the only ones. The credibility of the proposals that will be made for SKA and the SKA Regional Centre will have to be underpinned by validation and use of these tools on pathfinders. Conversely, the scientific exploitation of pathfinders already requires such tools, as we saw recently with the first results of the major survey of the Northern hemisphere by LOFAR : the exploitation of the LOTSS survey is based on direction-dependent calibration and imaging tools developed largely in France.

These tools for the SRC and the major SKA surveys are currently being prepared within the framework of the SKA Science Working Groups and the definition work of the European SRC (AENEAS and its successor), in particular in the form of "Data Challenges".

These considerations lead us to define the following service tasks :

 Development of methods and associated algorithms for data processing and analysis for SKA : deconvolution, direction-dependent and direction-independent calibration, wide-field imaging. These developments can be carried out within the framework of SKA consortia, with delivery in the construction phase, within the framework of Science Working Groups (SWGs) or SKA Regional Centres (SRCs). These tools will be tested, validated and used beforehand on SKA precursors and pathfinders such as LOFAR, NenuFAR, ASKAP, MeerKAT, MWA or HERA ;

 Participation in the definition and production of the scientific and technical data challenges that will be piloted by SKA-O and the SWGs. In particular the development of simulated datasets and tools to simulate them ;

 Development of tools and methods that will be necessary first of all for the scientific exploitation of the data challenges and then for the exploitation of the SKA data afterwards. In particular, algorithms for the automatic detection and classification of sources and transient phenomena. Once tested and validated, these tools will be made available to SKA-O and the community through the SRCs ;

 Development of tools specific to the scientific objectives of SKA’s future major observation programmes : development of pipeline elements for SRCs and large surveys, definition and production of standardised data in Virtual Observatory format (images, spectra, light curves), definition and production of data with added value (e.g. catalogues) ;

 Tests and implementation of these tools through the major observation programmes underway on SKA’s precursors and pathfinders. The techniques and algorithms developed within the framework of this service task and tested in the SKA data challenges and on the SKA precursors will thus become general-purpose tools for SKA in the same way as the calibration and large-field imaging software developed in France are now part of the LOFAR pipeline and are routinely used to process data from the large interferometers in operation (ATCA, VLA, MeerKAT, etc.) ;

 To facilitate the deployment of new algorithms and allow greater flexibility for users, the SRCs will use a software container-based interaction system. An effort will therefore be made within the SKATE service to provide containers including our tools and all the necessary dependencies so that they can be executed on any server ;

 Definition and optimisation of SKA’s VLBI observation mode : simulation and configuration tools, for astrometry and very high-resolution imaging. This activity is part of the SKA VLBI Focus Group. Developments are notably carried out within the framework of the European project JUMPING JIVE, led by the European institute JIV-ERIC (of which France is a member via the CNRS) (LAB/OASU) ;

 Tools for dealing with radio interference (RFI) : the pressure of telecommunications regularly reduces our radio frequency observation capabilities. Tools to help planning observations will have to integrate a priori information (e.g. interfacing with databases or satellite/aviation information sites), or statistical information, collected during past observations to minimise the failure rate of subsequent ones. In some cases, the impact of radio interference can be reduced by methods that make the dataset scientifically usable (e.g. low-level real-time flagging algorithms, or post-processing in the u-v plane or image). The aim will be to further develop these methods for preventing and treating RFI that have already been explored by our community.