LAST Data Products/pipeline I / Catalogs

The image catalog content and column descriptions.

Catalogs

Each LAST image is associated with a source catalog. There are several types of image catalogs:

1. Individual / Epoch image catalogs
2. Visit image catalogs
3. Reference image catalogs
4. Subtraction image catalog

This page is relevant to (1), (2), and (3). For the subtraction image catalog see: Each line in the catalog corresponds to a source. Sources with SN_2-SN_1<0 (e.g., hot pixels, cosmic rays) were removed from the catalog.

File names

This content is available either as a FITS table associated with the image (including headers: see ), or in the LAST products database.

Column names:

In each catalog, the following column names are available.

Column	Units	Description
XPEAK	pix	Integer-pixel X position of the source in the cropped image, measured from the local maximum in the matched-filter image.
YPEAK	pix	Integer-pixel Y position of the source in the cropped image, measured from the local maximum in the matched-filter image.
X1	pix	First-moment X position of the source.
Y1	pix	First-moment Y position of the source.
X2	pix $^2$	Weighted second central moment, $\langle X^2 \rangle$, of the source.
Y2	pix $^2$	Weighted second central moment, $\langle Y^2 \rangle$, of the source.
XY	pix $^2$	Weighted second central cross moment, $\langle XY \rangle$, of the source.
SN_1	—	Signal-to-noise ratio assuming the source is a delta function.
SN_2	—	Signal-to-noise ratio assuming the source is an unresolved PSF, as measured in the image.
SN_3	—	Signal-to-noise ratio assuming the source has a slightly extended PSF, modeled with an exponential profile.
BACK_IM	$e^{-}$	Background at the source position, measured from the background image.
VAR_IM	$(e^-)^2$	Variance at the source position, measured from the variance image.
BACK_ANNULUS	$e^{-}$	Local background around the source, estimated as the median pixel value in an annulus around the source.
STD_ANNULUS	$e^{-}$	Standard deviation of the local background, measured from the pixels in an annulus around the source.
FLUX_APER_1	$e^{-}$	Annulus-background-subtracted aperture flux of the source, measured in aperture 1, with radius 2 pix.
FLUX_APER_2	$e^{-}$	Annulus-background-subtracted aperture flux of the source, measured in aperture 2, with radius 4 pix.
FLUX_APER_3	$e^{-}$	Annulus-background-subtracted aperture flux of the source, measured in aperture 3, with radius 6 pix.
FLUXERR_APER_1	—	Relative error in the flux corresponding to FLUX_APER_1. Measured from $\sqrt{F + S^2}/F$, where $F$ is the source flux and $S$ is STD_ANNULUS.
FLUXERR_APER_2	—	Relative error in the flux corresponding to FLUX_APER_2. Measured from $\sqrt{F + S^2}/F$, where $F$ is the source flux and $S$ is STD_ANNULUS.
FLUXERR_APER_3	—	Relative error in the flux corresponding to FLUX_APER_3. Measured from $\sqrt{F + S^2}/F$, where $F$ is the source flux and $S$ is STD_ANNULUS.
MAG_APER_1	mag	Calibrated magnitude in aperture 1, with radius 2 pix.
MAG_APER_2	mag	Calibrated magnitude in aperture 2, with radius 4 pix.
MAG_APER_3	mag	Calibrated magnitude in aperture 3, with radius 6 pix. The calibration is likely derived using the transmission-fitting method and includes aperture correction.
MAGERR_APER_1	mag	Photometric uncertainty of MAG_APER_1, including source Poisson noise and background noise, but excluding calibration uncertainty.
MAGERR_APER_2	mag	Photometric uncertainty of MAG_APER_2, including source Poisson noise and background noise, but excluding calibration uncertainty.
MAGERR_APER_3	mag	Photometric uncertainty of MAG_APER_3, including source Poisson noise and background noise, but excluding calibration uncertainty.
FLUX_XYPEAK	$e^{-}$	Flux-like matched-filter or detection statistic measured at the integer-pixel peak position XPEAK,YPEAK. Exact definition should be verified from the pipeline code.
FLAGS	—	32-bit mask propagated from the mask image (all pixels within 3 pix radius from the source position). See bitmask description in: Image bit mask page.
X	pix	PSF-fit X position of the source.
Y	pix	PSF-fit Y position of the source.
XFULL	pix	PSF-fit X position of the source as measured in the full image (after overscan removal, as calculated using ORIGSEC header keyword).
YFULL	pix	PSF-fit Y position of the source as measured in the full image (after overscan removal, as calculated using ORIGSEC header keyword).
FLUX_PSF	$e^{-}$	PSF-fit flux of the source.
MAG_PSF	mag	Calibrated magnitude corresponding to the PSF-fit flux.
MAGERR_PSF	mag	Photometric uncertainty of MAG_PSF, including PSF-fit flux uncertainty but excluding calibration uncertainty unless otherwise specified.
PSF_CHI2DOF	—	Reduced $\chi^2$ of the PSF fit, $\chi^2/\mathrm{dof}$. Expected to be $\approx 1$ only for faint sources; for bright sources it grows as $\mathrm{SN}^2$ because of the PSF-model systematic error. See PSF-fit chi-square and the systematic-error floor below.
SN	—	Signal-to-noise ratio of the PSF fit.
MITER	—	Number of the PSF-fitting iteration in which the source was found.
RA	deg	J2000.0 right ascension corresponding to the PSF-fit X,Y position of the source.
Dec	deg	J2000.0 declination corresponding to the PSF-fit X,Y position of the source.
MergedCatMask	—	A bit mask indicating possible association of the source with external catalogs. Different bits correspond to different catalogs (See: MergedCat bit mask table).
DistMP	arcsec	Distance of the source from the nearest known minor planet (NaN if no minor planet within 10'').
AIRMASS	—	Hardie airmass of the source.
UPIX_PAR	—	Healpix unique-identifier of source (partition: NSide= $2^3$).
UPIX_LOW	—	Healpix unique-identifier of source (low: NSide= $2^8$).
UPIX_HIG	—	Healpix unique-identifier of source (high: NSide= $2^{16}$).
AB_ZP	mag	AB-magnitude photometric zero point used to convert instrumental fluxes into AB magnitudes.
FORCED	logical	A flag indicating if the source is a forced photometry source (true) or not (false).
primary	—	Crop-ownership flag: 1 if the source's exact PSF-fit X,Y position falls inside the crop's unique section (UNIQSEC header keyword), 0 if the source is a duplicate copy from an overlapping neighbouring crop. (See: the Overlap bit in the Image bit mask).

Comments

Forced photometry

The forced photometry is done (only on the epoch images) on a list of pre-defined positions. By default, this list contains all the known white dwarfs (GAIA), quasars, CV, and about 200,000 transients from the TNS. In order to avoid astrometric drifts due to source non-detection, the PSF fitting is limited to two positional iteartions, and can move up to 0.2 pix from the initial astrometric position. The initial astrometric position includes proper motion. Forced photometry measurments are indicated by true in the FORCED column.

Overlap de-duplication (primary)

Image crops overlap, so a source near a partition line appears in the catalogs of 2–4 crops. The Overlap FLAGS bit marks the full overlap region in all the crops covering it (issue #1180). The primary column records the ownership (primary=1 in the crop whose unique section contains its exact X,Y). Use primary=1 for de-duplication; use the Overlap bit only to know that a source lies in a multi-crop region. Catalogs produced before issue #1180 have no primary column; there, the older asymmetric Overlap semantics applies (the bit itself selects the duplicates).

PSF-fit chi-square and the systematic-error floor

PSF_CHI2DOF is close to 1 only for faint sources. For bright ones it grows roughly as $\mathrm{SN}^2$, reaching $10^2-10^4$ once MAGERR_PSF drops below about 0.01 mag (issue #1271). This is not a fit failure: it is the systematic error of the PSF model becoming visible once the photon noise falls below it.

The PSF-fit flux is the least-squares amplitude of the PSF. Write the fractional error of the PSF model as $\delta$, and split it in the noise-weighted inner product $\langle a,b\rangle=\sum_i a_i b_i/V_i$ into a part parallel to the PSF and a part orthogonal to it, $\delta=\delta_\parallel+\delta_\perp$. The fit absorbs $\delta_\parallel$ into FLUX_PSF, where it biases the flux but contributes nothing to $\chi^2$; a pure flux-scale error - scintillation, zero-point error - therefore does not inflate PSF_CHI2DOF. The post-fit residual is $F\delta_\perp$, so

$$\chi^2_{\rm sys} = F^2\langle\delta_\perp,\delta_\perp\rangle = \epsilon^2 F^2 \langle P,P\rangle = \epsilon^2\frac{F^2}{\sigma_F^2} = \left(\epsilon,\mathrm{SN}\right)^2 , \qquad \epsilon^2 \equiv \frac{\langle\delta_\perp,\delta_\perp\rangle}{\langle P,P\rangle} ,$$

where the third equality uses $\sigma_F^{-2}=\langle P,P\rangle=\sum_i P_i^2/V_i$, the PSF-fit flux variance. So $\epsilon$ is the noise-weighted rms of the PSF shape error orthogonal to the PSF, in units of the PSF's own noise-weighted norm, and the reduced $\chi^2$ expected for a correctly fitted source is

$$\left\langle \frac{\chi^2}{\mathrm{dof}} \right\rangle \simeq 1 + \frac{\left(\epsilon,\mathrm{SN}\right)^2}{\mathrm{dof}} .$$

SN is the catalog column. dof is the number of pixels inside the PSF-fit radius minus 3 (the fit solves for a flux and two positions); with the pipeline default FitRadius = 3 pix the pixel count is 25-29 depending on the sub-pixel centroid, and the measured value is $\mathrm{dof}\simeq 25$ (see below).

How to use it

Write the value of the PSF_CHI2DOF column as $(\chi^2/\mathrm{dof})_{\rm cat}$. The quantity that is $\approx 1$ for a good fit at any brightness is

$$\left(\frac{\chi^2}{\mathrm{dof}}\right){\rm corrected} = \frac{\left(\chi^2/\mathrm{dof}\right){\rm cat}}{1+\left(\epsilon,\mathrm{SN}\right)^2/\mathrm{dof}} ,$$

equivalently: reject a source as a bad PSF fit when

$$\left(\frac{\chi^2}{\mathrm{dof}}\right)_{\rm cat} > T\left[1+\frac{\left(\epsilon,\mathrm{SN}\right)^2}{\mathrm{dof}}\right]$$

rather than comparing PSF_CHI2DOF with a constant threshold $T$. With the values measured below, a fixed cut of PSF_CHI2DOF < 3 already rejects more than half of all sources above $\mathrm{SN}\approx 110$ and essentially all of them above $\mathrm{SN}\approx 200$, however good their PSF fit is.

$\epsilon$ can be re-measured on any data set as

$$\epsilon = \left[\mathrm{median}\left(\frac{\left[\left(\chi^2/\mathrm{dof}\right)_{\rm cat}-1\right]\mathrm{dof}}{\mathrm{SN}^2}\right)\right]^{1/2}$$

over unflagged sources with high SN. dof itself can be checked from the faint end, where $\chi^2/\mathrm{dof}$ has unit mean and variance $2/\mathrm{dof}$.

Measured values

On one LAST visit (LAST.01.08.03, 2023-06-16, field 346+79, 20 epochs $\times$ 20 s, 24 crops, 300,492 unflagged primary sources with $4<\mathrm{SN}<830$):

quantity	value
$\epsilon$	0.073
dof, from the faint-end scatter $2/\mathrm{var}(\chi^2/\mathrm{dof})$	24.6
$\mathrm{d}\log(\chi^2/\mathrm{dof}-1)/\mathrm{d}\log \mathrm{SN}$	2.12 (2 expected)
$\epsilon$ per crop	0.063 - 0.088 (median 0.073, scatter 10%)

With these values the model reproduces the median PSF_CHI2DOF over the whole range $4<\mathrm{SN}<830$:

Relation to the photometric-calibration rms

It is tempting to identify $\epsilon$ with the rms of the photometric calibration (PT_RMS, PT_ARMS, PH_RMS - see the header page), converted from mag to a fractional flux error by $\epsilon=\ln(10)/2.5\times\mathrm{rms}\simeq 0.921,\mathrm{rms}$. This does not work quantitatively. On the visit above the measured $\epsilon=0.073$ is 5.5 times $0.921\times$PT_RMS and 19 times $0.921\times$PT_ARMS; and across the 24 crops of the visit, where PT_RMS varies by 19% (0.0101 - 0.0225 mag), $\epsilon$ varies by only 10% and shows no correlation with it ($r=+0.04$, $p=0.85$; Spearman $\rho=-0.14$, $p=0.52$) - panel (c) above.

This is what the decomposition above predicts: the calibration rms is dominated by flux-scale errors, which the PSF fit absorbs, while $\epsilon$ measures PSF shape error. Treat $\epsilon$ as a quantity to be measured, not as a rescaled PT_RMS.

Caveats. The numbers above come from a single visit, on one field, with one PSF model; the crop-to-crop test constrains only the within-visit behaviour, and whether $\epsilon$ tracks the calibration rms between visits, nights or seeing conditions has not been tested. $\epsilon$ also rises slowly with SN (hence the fitted slope of 2.12 rather than 2), so a single $\epsilon$ reproduces the median $\chi^2/\mathrm{dof}$ to about 30% over two decades in SN. Finally, this is a relation for the median source: individual sources scatter around it, which is exactly what makes the corrected statistic useful for finding genuinely bad fits.

Artifacts

The source catalog may include artifacts of many kinds. Here are some ways to check the source quality.

Cosmic rays and bad pixels

Cosmic rays and bad pixels are removed from the source catalog. However, their positions is marked in the Mask image as CR_DeltaHT.

Bogus sources near bright sources

In some cases, bogus sources are found near bright sources. This is due to the fact that the PSF wings are not measured well and the PSF is tapered by a cosine bell. A simple way to detect such cases is by:

STD_ANNULUS ^2 / VAR_IM ^2 > 10^6