IGRINS‐2 Pipeline Data Reduction Products - igrins/plp GitHub Wiki

IGRINS-2 Pipeline Data Reduction Products

Below is a list of the data products from the IGRINS-2 Pipeline along with descriptions.

Main reduced data products

These are the main reduced data products created by the IGRINS-2 Pipeline, and found in the Gemini Archive. There are three main reduced data product FITS files: the spec.fits, spec2d.fits, and spec_a0v.fits files. The primary FITS header is stored in Extension 0 and contains most of the important header keywords. Subsequent FITS extensions store the data. The descriptive name of each extension is found in the EXTDESC header keyword.

  • plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.

    • spec.fits File that stores the extracted 1D spectrum. Targets reduced with "Stellar" recipes will be optimally extracted and "Extended" recipes will be sum extracted. Columns are x pixel position on the detector and rows are echelle orders. Pixel values are as follows.

      • Ext 1 - Extracted 1D spectra. The pixel values are detector counts.
      • Ext 2 - Extracted 1D variance of target extracted spectrum in Ext 1.
      • Ext 3 - The night's wavelength solution in microns determined from the OH sky emission lines.

    • spec2d.fits 2D spectrum stored as a data-cube of the rectified echelle orders. For Ext 1 and Ext 2, the X-axis is x pixel across the detector, y-axis is the spatial axis along the IGRINS slit, and the z-axis is the echelle order

      • Ext 1 - Rectified 2D spectrum. The pixel values are detector counts.
      • Ext 2 - 2D variance of 2D target spectrum in Ext 1.
      • Ext 3 - The night's wavelength solution in microns determined from the OH sky emission lines. Same as in the spec.fits file where columns are the x pixel position on the detector and rows are the echelle orders.

    • spec_a0v.fits File that stores many aspects of the 1D spectra in the following extensions. This file provides a spectrum that is crudely relative flux calibrated and telluric corrected. The flux calibration and telluric correction uses the target's associated standard star and a synthetic spectrum of Vega. This is meant to be a "quick look" calibration and might be sufficient calibration for some science cases, but we generally recommend you do your own flux calibration and telluric correction. Columns are x pixel position on the detector and rows are echelle orders.

      • Ext 1 - SPEC_DIVIDE_AOV - Target spectrum divided by its associated A0V standard star and multiplied by a synthetic A0V spectrum of Vega. This spectrum is roughly telluric corrected and relative flux calibrated. Good for quicklook purposes or if careful telluric correction and flux calibration is not needed. If you require a careful telluric correction and flux calibration, we recommend using the data from .spec.fits files for the target and associated A0V standard star along with a synthetic spectrum generated from stellar atmosphere models that matches the stellar parameters of the observed standard star.

      • Ext 2 - SPEC_DIVIDE_AOV_VARIANCE - Variance of Ext 1 propagated from the variance in the target and A0V spectra.

      • Ext 3 - WAVELENGTH - The wavelength solution in microns as determined from the OH sky emission lines in the sky frame taken for this night. Same as plp/calib/primary/{UTDATE}/SKY_N{UTDATE}S{OBSNO}_{BAND}.wvsol_v1.fits.

      • Ext 4 - TGT_SPEC - Target spectrum used in Ext 1. Same as plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.spec.fits.

      • Ext 5 - TGT_SPEC_VARIANCE - Variance of target spectrum in Ext 4. Same as plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.variance.fits.

      • Ext 6 - A0V_SPEC - Standard A0V star spectrum which the target spectrum is divided by in Ext 1. The A0V spectrum used is set by the frame number given in GROUP2 for a target in the recipe log. If no star is specified in GROUP2, the PLP will grab the nearest A0V standard spectrum to use. Same as plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.spec.fits.

      • Ext 5 - A0V_SPEC_VARIANCE - Variance of standard A0V spectrum in Ext 6. Same as plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.variance.fits.

      • Ext 8 - VEGA_SPEC - Synthetic spectrum of Vega of which is multiplied by the target divided by standard A0V star spectrum in Ext 1. The vegallpr25.50000resam5 model by R. Kurucz http://kurucz.harvard.edu/stars.html.

      • Ext 9 - SPEC_DIVIDE_CONT - Same as Ext 1 but divided by the estimated continuum of the A0V standard star instead of the A0V star spectrum itself. This leaves the target spectrum roughly relative flux calibrated but NOT telluric corrected. With some math with other extensions in the spec_a0v.fits file, can be used to recover the fitted continuum to the A0V standard star, just the telluric lines, etc.

      • Ext 10 - SPEC_DIVIDE_CONT_VARIANCE - Variance for Ext 9.

      • Ext 11 - MASK - Mask for pixels used in A0V star continuum fit. A value of 1 means those pixels are below the threshold and not used for the continuum fit.

      • Ext 12 - ORDER_MATCH_COR - [TO BE UPDATED]

Additional Data Reduction Products [TO BE UPDATED]

  • plp/calib/primary/{UTDATE}/SKY_N{UTDATE}S{OBSNO}_{BAND}.

    • wvsol_v1.fits - The night's wavelength solution determined from the OH sky emission lines. Columns are x pixel position on the detector and rows are echelle orders. The pixel values are wavelength in microns.

The PLP generates many additional data products that might be useful for advance users or for debugging.

  • plp/outdata/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.

    • sn.fits Signal-to-noise per resolution element (not pixel!) for the extracted 1D spectra. We recommended the user use .variance.fits for the uncertainty. Columns are x pixel position on the detector and rows are echelle orders.

    • spec_flattened.fits - An A0V standard star spectrum (same as found in the spec.fits) but divided by an estimate of its own continuum. Useful for getting the telluric absorption. Columns are x pixel position on the detector and rows are echelle orders.

    • wvlcor_image.fits Multi-extension fits file of the 2D echellogram which has been distortion corrected. In practice, this fits file is useful for viewing the stacked 2D echellogram for a target. The distortion correction straightens out shifts in wavelength along the length of the slit for each order, and is usually small. See the explanation of distortion correction.

      • Ext 1 - IMAGE - Distortion corrected 2D echellogram. Units are counts. Not that some inter-order pixels are masked.
      • Ext 2 - VARIANCE - Variance on Ext 1. Units are the square of the 1 sigma uncertainty in the counts.
      • Ext 3 - PROFILE_MAP - Map showing the x-axis (wavelength or pixel?) shift used in the distortion correction.
      • Ext 4 - MASK - Mask for pixels used in distortion correction.

    • debug_image.fits Two extension fits file of targets extracted with a "stellar" recipe (e.g. stellar-ab and a0v-ab) showing the following stacked the 2D echellogram

      • Ext 1 - Stacked 2D (distortion corrected?) echellogram of the target. Similar to wavecor_image.fits.
      • Ext 2 - "Synthetic" stacked 2D echellogram of the target where the extracted 1D spectrum is multiplied by the measured slit profile and then painted back onto an array the size of the detector. Possibly useful for debugging purposes to see uncorrected comics, detector defects, etc.

    • slit_profile.json - Average measured slit profile for a stellar target AB nodded on slit stored as two arrays in a json file. The array labeled "profile_x" is the fractional distance across the slit, and "profile_y" is the slit profile of the target. This is the slit profile used for optimal extraction and is discussed further here.

    • flat_off.fits - Stack of Flat Off exposures.

    • flat_on.fits - Stack of Flat On exposures.

    • flat_normed.fits - Flat On stack normalized to its 99th percentile.

    • flat_on.json - Debugging output that reports the FWHM of the Flat On background distribution (distribution of background values for all pixels). Not particularly useful for the user.

  • plp/outdata/{UTDATE}/

    • flexure_H.csv and flexure_K.csv - Files that report the measured flexure along the x-axis in pixels per band used in the flexure correction. The first column is the band H or K, the second column is the individual exposure frame number, and the third column is the measured flexure in pixels. The flexure is measured by FFT cross correlation of the x position of the OH sky emission lines in an exposure with the SKY frame. Flexure in the K band is typically ~10x larger than the H band, and the K-band flexure can be up to a little over a pixel in extreme cases. The reduced data are corrected for the flexure reported in these .csv files.

    • telluric_shift_H.csv and telluric_shift_K.csv - Files that report a measured shift along the x-axis in pixels per band that might arise from point continuum source targets (e.g. stars) not being centered on the slit through their entire observation. This can occur during poor guiding or in high winds shaking the telescope. This is done by cross correlating a few orders with a lot of telluric absorption lines using the same methods we use to measure flexure. The issue is rare but these files can be checked if you notice bad telluric correction residuals.

    • a0v.db - Database used by PLP to save which files correspond to A0V standard stars.

  • plp/calib/primary/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.

    • wvlsol_v1.json - JSON file that stores the wavelength solution in microns for the x-pixel position on the detector in each order. Same information as wvlsol_v1.fits.

    • wavelength_map.fits - 2D echellogram wavelength solution. Gives the wavelength in microns for each pixel on the detector where an order falls. Possibly useful for forward modeling 2D spectra.

    • fitted_pixels.json - Stores a table about the OH lines and the fit for the wavelength solution. Each entry in the table appears to be for an OH line used for the fit. The table has the following columns: ['slit_center', 'kind', 'order', 'wavelength', 'd_cent', 'cent_pixel0', 'pixels', 'params']

    • order_map_masked.fits - 2D echellogram map of the individual order numbers showing which pixels correspond to which order. Inter-order pixels are given a value of zero.

    • slitoffsset_map.fits - 2D echellogram map of the pixel shift in the x direction of the slit compared to the slit center along a column in an order. This is the "tilt" of the slit on the echellogram.

    • slitpos_map.fits - 2D echellogram map of the distance along the spatial axis of the slit, from 0 to 1 for each order.

    • volumefit_coeffs.json - JSON file that stores information about the distortion solution (slit position and offset on the echellogram) in a table with the following columns: ['pixel', 'order', 'slit', 'coeffs']

  • plp/calib/primary/{UTDATE}/FLAT_N{UTDATE}S{OBSNO}_{BAND}.

    • bias_mask.fits - 2D echellogram flat threshold and order mask. Regions in orders on the detector where the flat is bright above a threshold. Useful for isolating order or inter-order pixels on the 2D echellogram where data actually are.

    • flat_off_bg.fits - 2D echellogram estimate of the background in the Flat Off exposures.

    • hotpix_mask.fits - 2D echellogram mask for hot (bright) pixels on the detectors. This mask is applied to almost all data. Determined by finding bright pixels in the Flat Off exposures.

    • centroid_solutions.json - JSON file storing polynomial solutions to the bottom and top trace edge trace centroids of the orders from the flat field. The centroids are determined from edge finding using the derivative of the flat.

    • centroids.json - JSON file storing the top and bottom centroids of each order determined from the derivative of the flat. These centroids are used to calculate the polynomial solution to determine the traces that define the edges of the orders. The centroid_solutions.json file reports the polynomial fits to the values stored here.

    • deadpix_mask.fits - 2D echellogram mask that identifies locations of dead pixels from the Flats. The dead pixels are determined from the Flats where the normalized flux is below a threshold of 0.6 or strays significantly from the norm of 1.

    • flat_bpixed.fits- 2D echellogram of the stacked Flats with bad pixels masked as nans.

    • flat_mask.fits - 2D echellogram mask showing where a significant amount of light from the flat lamp falls on the echellogram.

  • plp/calib/primary/{UTDATE}/ORDERFLAT_N{UTDATE}S{OBSNO}_{BAND}.

    • .fits

    • .json

  • plp/calib/primary/{UTDATE}/N{UTDATE}S{OBSNO}_{BAND}.

    • sky_flexcorr.fits

    • aligning_matrix.json

    • identified_lines.json

    • aligning_matrix.json

    • multi_spec.fits

    • oned_spec.json

    • orders.json

    • wvlsol_v0.json

  • plp/calib/primary/{UTDATE}/

    • distortion.db

    • flat_off.db

    • flat_on.db

    • register.db

    • wvsol.db

  • plp/calib/secondary/{UTDATE}/FLAT_N{UTDATE}S{OBSNO}_{BAND}.flat_deriv.fits