Data Centre

Publié le 14 January 2021 par Mickaël Coriat.

This service task of SNO SKATE concerns the distribution of FR606 and NenuFAR data via the LOFAR-NenuFAR data centre as well as the preparation of French participation in the SKA Regional Centers network at both global and European level.

Low-frequency radio astronomy instruments are massive data producers and require significant computational resources; if SKA1 is announced as the first "Exascale telescope", LOFAR’s long-term archive already contains 44 PB, and NenuFAR will produce several PB per year of raw data. For the French community, the Nançay Data Centre (CDN) provides the storage space and the computing (hardware and software) and data distribution means. The CDN thus enables optimal exploitation of the low-frequency radio astronomy instruments to which the community contributes, in particular the instruments located at Nançay, FR606 and NenuFAR (the production of massive data makes it relevant to process and archive them on site). Its primary objective is to support the operation of LOFAR (including the standalone mode of FR606) and NenuFAR. The possible needs for the operation of other SKA precursors such as MeerKAT or ASKAP are considered on a case-by-case basis.

Access to SKA1 data, be it archival data or observational data from PIs or Key Science Projects, is one of the main roles of SKA Regional Centres. Today, French participation in the design of the network of SRCs at the global and European level is based on several aspects: 1/ participation in the working groups set up at the end of 2020 by the SRC Steering Committee (SRCSC) in order to identify technological and methodological solutions for the future network of SRCs at the global level, 2/ technical contribution to the second SKA Data Challenge - SDC2. Through the coordination of SKA-France and GENCI, nodes of the GPU partition of the IDRIS Jean-Zay machine are made available to the community participating in the challenge, and 3/ the expertise acquired through the implementation of the CDN.

In detail, the service tasks are:

  • Maintain the scientific processing software up to date,
  • Set up means of processing and archiving LOFAR data for the French community,
  • Set up means of processing and archiving FR606 data, setting up a database,
  • Design, operation and perpetuation of the processing and archiving of NenuFAR data: nature of archived data, of online data, quality control, etc.,
  • Set up access to NenuFAR data for the French and international community: members of the Key Programmes, and starting 2021, PIs of observation proposals,
  • Archive data and provide access to public data (NenuFAR data will be public after a proprietary period of one year after completion of the programme).
  • Participation in the activities of one or more of the working groups of the SRCSC, which begins its activities in 2021:
    • WG0: SRC Network Architecture
    • WG1: Data Logistics Working Group
    • WG2: Operations Working Group
    • WG3: SW Federated Computing and Data Software Services
    • WG4: SW, Science Archive-VO-FAIR
    • WG5: Compute Working Group
    • WG6: Science User Engagement

Virtual Observatory

In the case of NenuFAR, beamformed data, dynamic spectra and pulsar archives are distributed in FITS and PSRFITS format. Imaging data will be distributed in Measurement Set format (visibilities and cross-correlations) and in FITS format (images). The experience acquired during the first NenuFAR (resp. MeerKAT/ASKAP) Key Programmes will allow a diversification of "science ready" products and an evolution towards Virtual Observatory standards. This reflection is conducted in coordination with the Strasbourg Data Centre, which has the required expertise. Within the SKA context, similar needs are emerging for the development of OV standards for radio astronomy. These reflections will take place with the WG4 of the SRCSC (see above).