known issues - njcuk9999/apero-drs GitHub Wiki

Known Issues with the DRS

These are issues that are known (or that need looking into) and fixing. Please feel free to add into the appropriate section. If there are issues related to this please link to them (or at least write the issue number)

How to add an Issue with a link:

Issue [#999](https://github.com/njcuk9999/spirou_py3/issues/999)

Raw image

  • HC Saturated lines bleed horizontally and vertical - how does this affect products using the HCONE_HCONE files

Preprocessing

  • How do faint images deal with the corruption checks?

Dark

  • Do we need to treat DARKS in P4 and P5 differently?
  • Check and improve QC

Badpixel/background correction

  • Check and improve QC
  • Negative pixels are set to zero after background correction - the affect of this needs exploring

Localisation

  • Problem with identifying extra (erroneous) orders --> solution to median filter image?
  • Check and improve QC

Shape

  • Need QC for shape file (currently None)
  • Shape occasionally failing #540
  • slit code numpy.correlate freezes some times (i.e. similar loops take ~5s for first few and then ~ 30 mins for later ones) - why? - no answer from google searches

Flat/Blaze

  • Check and improve QC

Wave solution

  • Need to implement Etiennes new wave method
  • Need to compare wave solution methods (Etienne has a code for this? Should it be in the DRS tools?)
  • Check and improve QC
  • ic_littrow_ordeR_init has no effect? (Issue #503)
  • cal_HC triplet fit seems to ignore first and last order (Issue #502)
  • DRIFTPEAK on HC files should ignore saturated lines (Issue #474)

Extraction

  • Production of E2DS and E2DSFF do we need both? (Issue #416)
  • Check and improve QC
  • uncertainties are not calculated / propagated

BERV

  • Need a back up when BERV fails for online mode (Issue #545)
    • Error connecting to online service (i.e. astropy.utils.iers.iers.IERSRangeError)
    • Error when running in parallel using gdbm (some OS/python distributions only)
  • Need a way to make BERV parallisable (or wait) when two running at once
  • ~~ add observation hour (mid exposure time) to header ~~
  • BERV is not corrected for flux weighting (can this be done with the exposuremeter?)

Tellurics

  • Need to understand how many telluric PCA components to use (currently 5 but what happens when we use 20, 50)
  • Need to understand how-many new tellurics warrant an update of the telluric database
  • Need to write wrapper for obj_fit_tellu+obj_mk_obj_template+obj_fit_tellu (similar to obj_mk_tellu_db or part of it)
  • Need to make obj_mk_tellu_db parallisable (same with fit_tellu version)
  • Need to add Tau Water and Tau Rest calculate + to header
  • Need to fit OH/O2 emission lines
  • Problem with number of tellurics getting past QC (mk_tellu or mk_obj_template?)

RV calculation

CCF

  • RVC should not be corrected with OBJ_DARK in cal_CCF_E2DS_FP_spirou.py
  • Need to use recon files in CCF code (to add/modify mask in regions with large transmission)
  • Need to add wmeanref for AB and C fibers to the header
  • Check and improve QC
  • How should cal_CCF_E2DS_FP_spirou.py handle the wave solution (Issue #530)

Template matching

  • No code in DRS currently

DRS

General

  • Overhaul of input system (in progress) --> INTROOT2
  • Overhaul of error system (in progress) --> INTROOT2
  • Catching/handling of unexpected errors (in progress) --> INTROOT2
  • Generalization for other instruments (in progress) --> INTROOT2
  • Make sure all input files are recorded in header
  • Need to add fiber to header for all outputs that are for a specific fiber
  • Need user setup script (in progress) --> INTROOT2
  • Need dev setup script for new instrument (in progress) --> INTROOT2
  • uncertainties are not calculated / propagated

New recipes

  • Need an overall polarization recipe (i.e. extract --> telluric correct --> polar --> CCF)
  • Need an overall spectra recipe (i.e. extract --> telluric correct --> CCF)

Quality control

  • Need to look over all QC for all recipes
  • Add standard QC outputs to headers (i.e. QC1VAL, QC1LIM, QC2VAL, QC2LIM) - Issue #539

Plotting

  • Overhaul is needed of the plotting module (Issues #546, #548, #549)

Files

  • Generic read/write function (able to take 1D,2D or 3D image)
  • BinTable and Image fits types into extensions
  • main header must have info on what is in these extensions

Visualization

  • Need a proper visualization tool (command-line and GUI)

Optimization

  • Use of cython or numba or numexpr to speed up some algorithms?