Click for R-Script

ANOSIM tests whether distances between groups are greater than within groups. PERMANOVA tests whether distances differ between groups. Both tests are sensitive to unbalanced designs and differences in dispersion (variance) within groups (e.g. not good when your groups have different variability). Anderson and Walsh (2013) conducted a simulation-based comparison of PERMANOVA and ANOSIM and found that PERMANOVA is more robust in general for ecological data, but still sensitive to the heterogeneity of variance among groups. You should evaluate this assumption before using either test.

The F-Ratio :

The test statistic used is a pseudo-F-ratio, similar to the F-ratio in ANOVA. It compares the total sum of squared dissimilarities (or ranked dissimilarities) among objects belonging to different groups to that of objects belonging to the same group (Equation 1). Larger F-ratios indicate more pronounced group separation, however, the significance of this ratio is usually of more interest than its magnitude.

PERMANOVA uses permutation to assess the significance of the pseudo-F-statistic described above.

In a one-way test (where the interest is on whether a statistic is either less than or greater than what can be expected by chance), the P-value calculated reports the proportion of permuted pseudo-F-statistics which are greater than or equal to the observed statistic, i.e. what proportion of the permuted data sets yield a better resolution of groups relative to the actual data set following a PERMANOVA. It is generally accepted that any separation between groups is not significant if more than ~ 5% of the permuted F-statistics have values greater than that of the observed statistic (i.e. a P-value > 0.05).

It is vital that the correct permutational scheme is defined and only exchangeable units are permuted. In nested studies, this would mean restricting permutations to an appropriate subgroup of the data set. At times, exact permutation tests either cannot be done or are restricted to so few objects, that they are not useful. See Anderson (2001, 2005) for examples of permutational schemes involving complex experimental or sampling designs.

Post-analysis: a posteriori testing

As in ANOVA, a significant result indicates that there is a significant difference between the groups defined; however, there is no way of knowing which groups are significantly separated. A posteriori testing, using NPMANOVA, of each pair of groups can be performed after a significant result to determine this. As these are pairwise comparisons, the test statistic involved is the non-parametric, multivariate analog of the t-statistic, with significance determined by permutation, as above.

As this involves multiple testing, an appropriate correction should be applied.

Key assumptions

According to Anderson (2001), the only assumption of PERMANOVA is that the objects in the data set are exchangeable under the null hypothesis. That further implies:

• exchangeable objects (sites, samples, observations, etc.) are independent
• exchangeable objects have similar multivariate dispersion (i.e. each group has a similar degree of multivariate scatter. See Anderson, 2001 and 2006)

Points to Consider

• PERMANOVA takes no account of correlations between variables and any hypothesis that depends on detecting such relationships will not be addressed.
• Nested or hierarchical designs require an appropriate permutational scheme, carefully understanding which objects are truly exchangeable under the null hypothesis. Most importantly, the analyst must define "strata" within which to restrict permutations.
• This method generally assumes balanced designs, however, unbalanced designs can be handled (see McArdle and Anderson, 2001).
• Anderson (2001) warns that groups of objects with different dispersions, yet no significant differences in centres (centres are similar to means, but maybe non-Euclidean), may result in misleadingly low P-values. It is thus recommended that the dispersion be evaluated and considered when interpreting the results of PERMANOVA. See Anderson (2006) for a discussion on tests of multivariate dispersion.
• Criticisms of this and other (dis)similarity-based methods should be taken into account (e.g. Warton et al. 2012).

Source: https://mb3is.megx.net/gustame/hypothesis-tests/manova/npmanova

Click for R-Script