meeting 2025 06 30 n57 - JacobPilawa/TriaxSchwarzschild_wiki_6 GitHub Wiki

Following up on some of the library diagnostics from last time, trying to understand why the full library.

Takeaways:

• There are a number of takeaways in the plots below, but I think the most relevant are for reconciling all the differences we have seen so far is the influence of scan603.fits. Specifically, I was surprised to see that the set of Barth templates vs. the 12 overlapping templates in the set of 384 actually lead to quite different kinematics and different weight distributions.

• This seems to come from that fact that the template that receives the most weight from the Barth templates (scan603.fits) is absent from the set of 384 templates/(and also the 209 set of stars I created). So when I go to fit with the 384 templates, the most important star is missing and must be "made up" with some of the other templates.

• What's even more interesting is how sensitive the kinematics are to this spectra being included -- I ran two other tests where I took the 12 overlapping set and added each of the 3 missing individually, and scan603.fits being included esssentially perfectly fixes our results, whereas adding either of the two others basically changes nothing to our results.

  • I also tried a case where included the 3 missing templates with the set of the 384, but the missing templates are not actually being picked out there, suggesting that ppxf is finding a different subset of templates that are "better fit".
    • This is also consistent with the RMS being lower for these fits. While this does mean they are "better fit" numerically, it doesn't account for the added freedom that the additional templates give.

• Moving forward:

  • Maybe it's worth devising some test of the spectra's kinematics on specific templates? I am thinking back to the meeting where we briefly discusses "jacknifing" templates from the full CaT library to get a sense of the statistical/systematic uncertainties separately? Or at least get a sense of the dependence on specific templates?
  • Otherwise we might have to accept some level of differences in the kinematics coming from the template choices. Though some of the tests below seem to show a "smaller" difference in the kinematics than we had seen before (which was >= 10 km/s in sigma).
  • Maybe it's worth trying some "regularization" on the weights? Something that penalizes by the number of weights turned on, so that we can try to "maximize" ecah weight for all that it's worth? That could be quite interesting. Something like LASSO for the template weights rather than what is used now (which admittedly I need to read into...)

Note on some terminology below: just for clarity, here's what the various labels mean in the tables below:

• Barth = fiducial set of results
• Barth reprocessed = I rerean the fiducial settings to ensure that we can reproduce ourselves
• 209 Trimmed = set of 209 stars Emily passed on 05/10 ("a trimmed CAT library which is the CAT version of each of the MILES templates in our MILES_trimmed library")
• 384 Trimmed = set of 384 stars from ~last week, for which Emily had a reasonable spectral type identified.
• 12 Overlapping = 12 stars of the 384 Emily had selected are also in the Barth subset. The exact file names are near the bottom of this page.
• 3 Missing = 3 stars/15 which are in Barth but NOT in the 384 Emily had picked out. These three stars are scan603.fits, scan612.fits, scan669.fits, with scan603.fits actually being the highest weight star from Emily's library. It's a bit strange that this is not included in the set of 384 if it is given the most weight of the Barth subset alone. But curiously, scan603.fits (when combined with 384 other stars) does NOT receive that much weight.

• Note that I ran a "reprocessed" Barth which has identical settings to our fiducial case, but just wanted a sanity check that two sets of independent runs are giving the same thing. This does appear to be the case.
• Also note that the "Barth vs. 12 Overlapping" seem to show that the "Barth" are "inflated" relative to the "12 overlapping." However "Barth vs. 3 missing" actually shows Barth "overpredicting" the moments, so it does sort of make sense that the combiation of these two sets settles "in the middle" and leads to the Barth results.

• Nothing too much to be gained, but I included these for completeness, at least giving us a sense of how discrepant the different results are.

• Though take on their own they don't mean quite much, we can also see how much "better" fit the spectra with more templates are, suggesting that the differences we are seeing are a result of overfitting?

• Here's the spectra for each case above (plotted vs. pixel number) to at least give a sense of the quality of the templates. There are certainly some very strange looking templates in the "fuller" libraries that appear to be improperly calibrated?

I was very surprised to see the results comparing the 15 Barth stars to the 12 stars from the overlap between Barth and the set of 384 templates from Emily. As a quick note, here are the 12/3 relevant templates:

• 12 Overlapping: scan399.fits, scan411.fits, scan419.fits, scan437.fits, scan574.fits, scan194.fits, scan216.fits, scan219.fits, scan272.fits, scan274.fits, scan286.fits, scan304.fits
• 3 Missing (from set of 384): scan603.fits, scan612.fits, scan669.fits

I went ahead and refit the spectra with the (12 templates) + (1 of the missing templates) to see if one of the missing stars is leading to the differences:

• Note that the y-axes here say "12 missing" but they include the template in the table header

• Note that the template distribution is VERY different when using the 12 overlapping alone, but when I add scan603.fits to be available, the resulting template distributions are basically the exact same. This seems to suggest that scan603.fits is extremely important to our final fits since it gets so much of the weight.