meeting 2025 04 16 n57 - JacobPilawa/TriaxSchwarzschild_wiki_5 GitHub Wiki
Context
-
I reminimized the s=1.0 models of Grid C so we can get a sense of how similar the results from removing the outer 4 bins is to the results where they're properly masked. I've got a 1-to-1 plot below comparing their NNLS chi2s, and I also ran GPR and dynesty of the s=1.0 models for both cases so we can compare what GPR/dynesty produces.
- Note that in the plots/tables below, "Removed" = don't include the outer 4 bins in the NNLS chi2 calculation; "Masked" = properly mask the outer 4 bins by inflating the errors on the moments.
-
Takeaways:
- I'm very happy with the agreement between the "removed" and "masked" cases! A linear fit to the chi2's seems to capture much of the variation in the two sets of values, and the resulting residuals have a mean of 0 and a standard deviation of 4.06 which seems to be an exceptional level of agreement to me.
- Moreover, although the GPR and dynesty results appear quite "rough" right now, I'm not totally convinced just yet that there will end up being a large difference in our parameters. With only the s=1.0 models, GPR and dynesty has a hard time pinning down M/L and rho0 (likely due generally undersampling the space with models), and so I think this is causing much of the shift in our parameters. Because the actual chi2 values seem to follow a 1-to-1 trend exceptionally well, I'm inclined to think that GPR and dynesty will simply improve and reach a better agreement if we were to add more models.
- Alternatively, we could try to add a few of the high scalings to this grid to better cover the upper end of the M/L and rho0 parts of the space. Again this is quite cheap, around 500SU/scale.
Diagnostics
- First, here are some comparisons of the NNLS chi2's as 1d panels, plotted against one another, and plotting their residuals after subtracting away a linear fit to the data.
- I'm actually really happy with the level of agreement between the two sets of chi2s. It seems like properly masking the outer 4 bins does a bit more to the overall chi2 than simply removing them which I think makes sense (the next outermost bins are likely better fit), but there doesn't seem to be any huge shift in the landscape shape, and a linear trend seems to capture most of the trend.
- The residuals also appear to be rather Gaussian with mean 0 and standard deviation of 4 which is a really excellent agreement.
| 1d Panels | 1-to-1 | Residuals |
|---|---|---|
| [images/250416/1d_panels.png]]](/JacobPilawa/TriaxSchwarzschild_wiki_5/wiki/[[images/250416/1-to-1.png) | images/250416/residual.png |
- Here's a quick side-by-side of the two lowest chi2 radial moments:
| Unmasked | Masked |
|---|---|
| [images/250416/best_unmasked.png]]](/JacobPilawa/TriaxSchwarzschild_wiki_5/wiki/[[images/250416/best_masked.png) |
- And here's GPR + dynesty run on only the Grid C s=1.0 models with the outer 4 removed vs. properly masking the outer 4 models:
| K=40 | K=60 | K=80 | |
|---|---|---|---|
| Outer 4 Removed | [images/250416/unmasked__grid_alpha_K40_nu1.5-1.png]]](/JacobPilawa/TriaxSchwarzschild_wiki_5/wiki/[[images/250416/unmasked__grid_alpha_K60_nu1.5-1.png) | images/250416/unmasked__grid_alpha_K80_nu1.5-1.png | |
| Outer 4 Masked | [images/250416/masked__grid_alpha_K40_nu1.5-1.png]]](/JacobPilawa/TriaxSchwarzschild_wiki_5/wiki/[[images/250416/masked__grid_alpha_K60_nu1.5-1.png) | images/250416/masked__grid_alpha_K80_nu1.5-1.png | |
| All Bins Included (Fiducial s=1.0 results) | [images/250416/fiducial_test_grid_alpha_K40_nu1.5-1.png]]](/JacobPilawa/TriaxSchwarzschild_wiki_5/wiki/[[images/250416/fiducial_test_grid_alpha_K60_nu1.5-1.png) | images/250416/fiducial_test_grid_alpha_K80_nu1.5-1.png |
Notes on N57 History/Bugs
-
Going through my notes/old Github history to better understand the evolution of N57 from ~1 year ago. This will be filled in soon with triaxnnls bug details and other important bits of N57 history.
-
The triaxnnls bug was introduced in ~Sept 2024 around the time we were changing TriOS to otuput both the normal kinematics in V/sigma and the kinematics in h1/h2.
- Prior to updating TriOS, we had a set of scalings for N57 using s=(0.97,0.99,1.0,1.01,1.02,1.03) which had been run with the old set of binaries and were primarily processed using the kinem chi2s.
- The week of Sept ~10, Emily had added the h1/h2 information to be output from TriOS, and I think recompiled the code. Later that week, we added scalings for s=(0.995,1.005) to our models using the updated set of binaries. You can start to see this effect on the 9/20 page, in which I have plots showing the distribution of scale factors preferentially picking the 0.995 and 1.005 cases since these new triaxnnls binaries were preferentially smaller by ~6 (from the first line being missed).
- On 9/26 page, we reminimized the s=1.00 case to pin down by the new and old binaries gave different results. We again saw that the s=1.0 case was ~6 lower in chi2 for each model, but we didn't know why that was the case yet. To be entirely consistent, we reminimized s=0.99, 0.995, 1.00, 1.005, 1.01, 1.015, 1.02 cases all with the new binaries. You can see these panels at the bottom of the 9/26 page.
- Importantly also on the 9/26 page, we had played around with GPR and dynesty and were having trouble getting reasonable posteriors for nu=1.5, so we decided to switch to nu=0.5 for our final set of results. Switching from nu=1.5 to nu=0.5 I think is the most direct reason our BH switched from 5.5e9 in the older Overleaf to 6.3 in the new Overleaf, and this seems to be confirmed in our "final cornerplot" on the 10/01 page where we had tested the linking lenghts on our final set of results. This final posterior is identical to our overleaf, and is identical to the 9/26 page's K=60, nu=0.5 cornerplot.
-
It was around this time we paused N57 and switched over to N315, though importantly we hadn't exactly pinned down why the chi2 had differed between the old and new binaries.
- It was in continuing with N315 that I found the exact reason for the chi2 difference of ~6, and we had an email exchange over December 2/3/4 and some N315 clarifying this issue.
- After finding the reason for the difference, we had continued on with N315 and put a pause on N57 work, but hadn't gone back and reminimized the N57 models to correct this central bin issue (all scalings had been equally impacted) since we simultaneously were discovering my Barth star issue. By the time we started reminimizing the latest N57 set of models, everything had been fixed in TriOS.
-
I've put a note on the N57 Page marking the bullets here and pointing to this discussion for future reference.