Basic Stats - LIMO-EEG-Toolbox/limo_tools GitHub Wiki
Basic stats
The LIMO Random effect interface has a basic stats section allowing to compute summary statistics (mean, weighted mean, median, mid-decile Harell-Davis) with the associated 95% High Density Interval (equivalent to a 95% confidence interval but computed using a Bayesian Bootstrap). --> see https://github.com/LIMO-EEG-Toolbox/limo_eeg/wiki/Plotting-conditions
In addition, whenever you observed an effect, at some point you will likely want to contrast 2 time course or spectra, and best is to also show that difference (with 95% High Density Interval) --> see https://github.com/LIMO-EEG-Toolbox/limo_eeg/wiki/Plotting-differences
Ensuring there is no effect
Using a Bayesian Confidence Interval on a difference gives you more certainty on the absence of effect.
You need to remember that a (frequentist) confidence interval indicates if observed values can be rejected by a (two tailed) test with a given alpha - so a 95% CI on a difference that includes 0 tells you that the hypothesis of no effect can not be rejected with 5% chance to be wrong in the long run. This means the classic CI tells you nothing about the variation of your statistics (e.g. doesn't tell you that the mean difference varies between X and Y) nor does it tell you anything about the difference per se, only about the hypothesis that the difference is 0. Moving on using Bayesian estimates, here HDI, it actually gives you the probability coverage of the summary statistics, that is that e.g. the mean difference varies between X and Y. In turn, if for a difference, 0 is included in this interval, because one tests H1 (the hypothesis of a difference) directly and reject it, we can be more confident that there is no effect (rather than say no effect observed which is accepting the hypothesis of no effect but not testing it - see e.g. Pernet 2017)
CAVEAT The HDI computed are done independently at each frame. In frequentist terms in means that the probability coverage over the entire space is much higher than the alpha level (multiple comparisons issue). From a Bayesian viewpoint, since the HDI is not intented to provide in interval of long run probabilities but the actual variation of the summary statistic, this is not a problem.