Centre

DATA PROCESSIONG TOOLS FOR MIRI

Pipeline

Data processing tools for MIRI data reduction are being developed by STScI on the basis of algorithms provided by the Performance Tests and Calibration Team (PTCT) and by individual institutions belonging to the European Consortium (EC). Inside the EC MICE members were heavily involved in the imager pipeline working group (IPWG),  which task was to define the overall architecture and to produce those algorithms to be coded in Python and included in the general pipeline built at STScI.

Indeed, MICE leads the Imager and Coronagraphs Working Groups : MICE is in particular responsible for the calibration, the mosaicking, the source detection and the photometry.

 

  • The pipeline delivered by STScI has been tested in the course of changing versions (currently 7.3 – October 2019).
  • Programmes for the throw on one or several files in an automated way have been written.
  • The dependence in memory and processor according to the size of the files of entry has been studied.
  •  

The proposed architecture shown below is in particular based on so-called “self-calibrations”. Not all of them will be applied in the general pipeline developed at STScI. A copy of this software will be kept updated at MICE’s premises. It will be the task of MICE’s management to decide when and which modifications should be made to this pipeline.

MIRI CALIMAGE 3

Self-calibration is commonly referred to as taking advantage of the fact that a set of dithered images will have different pixels sampling the same position in the sky. Comparing the measured signal in these pixels can be used to study the performance of the detector and to make a correction of the image, if necessary. Calibration steps in the pipe line can then be thought of as an active rather than a passive correction.

Three primary calibration steps have been identified:

  •  – Baseline/background removal
  •  – Delta cosmic ray rejection
  •  – Solving for delta-flat and delta-dark (note that the suffix delta is used here to not confuse with calibration steps earlier on in the pipeline i.e. in cosmic ray rejection).

Calibration images are made as close to the epoch of the observation as possible. But these calibrations may not be sensitive enough due to the change in the sky background and flat field in time. Self-calibration has the advantage that one is using the data taken at the same epoch to calculate the background and flat field.

  • For MIRI the thermal signature of the telescope and other components (<15μm) is expected to change in time and therefore require self-calibration for correction. MIRI has been designed to take dithered images that are specifically optimized for self-calibration: 12-point Reuleaux pattern and 311-point cycling pattern. 
  •  
  • Note that it may not be necessary to use self-calibration if the thermal backgrounds are more stable than predicted in the worst-case scenarios. It has to be said also that historically, Spitzer processing had a self-calibration option.  
  •  
  • However, a strategy of spatial redundancy and masking worked well in most cases. Notably for Spitzer it was used for some programs such as for observations of exoplanet transits, using a dedicated pipeline: it is part of MICE’s tasks to develop such dedicated pipe lines. 

  •  It is worth noting also that :

    •  
    • – Parallelization is foreseen for the future; it will be necessary for running Astro-Drizzle.
    •  
    • – The distribution of the data by the STScI to the users is not yet well defined: it will most probably be of the ESO/Archiving kind, although with the sending of a CD containing data and an executable version of the pipe line. It remains to be decided how MICE will proceed in that respect when data will be re-processed. 
    •  
    • – MICE will use a VO compatible archiving system for data products coming from our pipeline (and its specific extra steps), as the STScI will do (MAST: Mikulski Archive for Space Telescopes).

  • Programs & Projects Accomplishments

  • The LABCOM INCLASS

  • Meetings and Conferences

  • Tools

  • Documents

  • Detectors

  • Pipeline

  • Data rate

  • High level data analysis tools

  • Simulator

  • Science Data for Webb’s First Images Now Available

    • JWST