meeting 2023 06 15 - JacobPilawa/TriaxSchwarzschild_wiki_5 GitHub Wiki

Context

  • This bullet contains a bit of information on sqrt(2Nkin)/the uncertainty approaches that others are using in their recent applications. I also have a few more iterations on our summary style plots below.
  • Next going to turn the table/digging below into actual text and start to place in some of the final summary plots.

A bit of literature digging...

Click to expand for table of papers + notes

  • I did a bit of digging into the history of sqrt(2Nkin)/what people are using for the more recent black hole mass measurements for their uncertainties. I started with the papers which explicitly cite Lipka+Thomas 2021, of which there are only a few which deal with mass/shape constraints.

  • The table below contains these papers and a few notes I made going through them, but the takeaways are that folks use one of the following (or something really similar) to compute their uncertainties:

    • True chi2 cutoffs in N dimensions (I think 2 is the most I have seen, though), in particular for cases where I feel they had quite a few number of models sampling the space

      • this is the closest type of uncertainty to the one which we use for a single realization

    • Cutoff criteria of something like 2sqrt(2Nkin), which, for the cases where I was able to quickly compute Nkin, are at least at large as our value of sqrt(2*654) =~ 36

    • Divide the data into two halves and compute the uncertainty on the standard deviation of these TWO measurements

      • this last category of uncertainty is most closely related to the difference across realizations

    • In almost all these cases, there are caveats in the text about how "these criteria are not meant to be statistically rigorous but practical" in recovering their values/maintaining consistency with their tests

Paper link notes
Roberts+2021 ADS Link uses chi2 cutoffs corresponding to 1sigma/2sigma/3sigma with a given # of degrees of freedom; but also has a mention (page 13) about how there has been little systematic study of these criteria and they're not statistically kosher; doesn't use meff even though it's axisymmetric because they say the galaxy is already quite round;
den Brok+2021 ADS Link axisymmetric and triaxial with upper limit on triaxial and constraint on axisymmetric results; for triaxial, they fix the shape and halo and let the black hole free, with sqrt(2Nkin) cutoff [they don't state Nkin anywhere, but it looks compare to M87?]; uses Delta Chi2 for 2 d.o.f for axisymmetric constraint and quotes 3sigma levels; quotes 99% confidence inteval (3sigma equivalent) for JAM; ALSO computed meff! but buried the results in supplementary material (which isn't even labelled correctly...). they found that their recovered shape doesn't change at all, and that the shape profiles get shallower as we would expect. it's very frustrating how little information they provide, though, and the plot that they do include. here is a link to the supplementary material; also they only computed meff for fixed black hole and fixed halo
Thater+2022 ADS Link cross checking of gas and axisymmetric stellar dynamic masses; obtains uncertainties with a Delta Chi2=11.8 cutoff which is 3sigma for 2 d.o.f.; note: no change in mass components when fixing inclination to either 89 or 45;
Santucci+2022 ADS Link doesn't explore black hole masses, and seems to use an even more generous criterion of all models less than 2*sqrt(4Nkin-Npar); they quote four different Nbins = (255,87,142,104) for one of their galaxies, which propagates to a cutoff of (~90, ~52, ~67, ~57) in chi2 for their errors.;
de Nicola+2022 ADS Link Not too relevant actually.
Thater+2022 ADS Link revisits den Brok+2021 with orbit mirroring fixed and also computes enclosed mass for ~50 galaxies before and after fix; they conclude that sqrt(2Nkin) errors lead to insignificant changes when correcting the orbit mirroring; their main conclusion is that fixing the orbit mirroring does not appreciably change the contours nor the best fit locations;
Tahmasebzadeh+2022 ADS Link uses sqrt(2Nkin) for uncertainties, but also very densely covers the space;
de Nicola+2022 ADS Link splits the data into two sets with approximately equal kinematic bins, and assigns uncertainties based on the variance between these two fits (this feels wrong...). they also have a few statements where they show "uncertainties" with an AICp cutoff of 50, but it doesn't look like they quote these ranges as formal uncertainties anywhere;
Neureiter+2023 ADS Link uses AICp for 10d search with ~3000 models, and instead of using cutoff criteria, they again split the data into two sets (x>0 and x<0) and take the standard deviation of their results for the uncertaintie; they explicitly state that using the chi2 distribution or AIC distribution is "meaningless"; they state that their work shows 5% to 10% uncertainties for Schwrazschild models are reasonable
Ding+2023 ADS Link uses criterion of < fsqrt(2Nkin) where f depends on Nbins; for Nbins<200, f=1; else f=4; they say that this is motivated not by statistics but practical use of the uncertainties
Mehrgan+2023 ADS Link Not too relevant
Santucci+2023 ADS Link again uses a cutoff of 2*sqrt(4Nkin-Npar)
Neureiter+2023 ADS Link splits the data into two halves and takes the difference between these as the statistical error in their results; their triaxial total mass is consistent with an axisymmetric measurement of the same system using their errors;
Santucci PhD Comments See CP Email Mostly addresses the different criteria raised above

Plots

  • I also made a few new plots showing our results based on the feedback you both gave in the last meeting! Here are those with a bit of a description:
  • One quick note: we had wanted to try to plot all 25 realizations on the same plot in some reasonable way, but I think there's just too much overlap between the contours and 1d histograms for me to get anything reasonable looking. Everything just looks extremely messy. I'm still working on getting something from all 25, and will hopefully have something for the meeting Wednesday.
Here's essentially the same plot that we had last time but with two additional panels showing the 1D chi2 vs. params. In this version, I kept the 2d distribution of points, and tried to color them in a way that showed a connection between the 1D and 2D panels. In later versions, I replace these scatter points with a 2D box showing the min and max of our ranges
Plot
Plotting a red box for our min/max ranges instead of the 2d scatter points to better illustrate the lack of marginalization.
Plot
Limiting the results above to a **single** case of P1 r1
Plot
I also made a few different plots for various cutoff levels to see where we would need to cut out points to get approximately the same result as our GPR and dynesty, so here are a few of those plots. Using a cut of 2.3 (which is the cut in 1sigma for a chi2 distribution with 2 d.o.f.) actually gives approximately the correct sized errorbar, which I think is pretty good evidence that we are doing things appropriately.
cutoff sqrt(2*Nkin)=36) 25 10 6.18 5 2.30 (22 models make the cut)
⚠️ **GitHub.com Fallback** ⚠️