Reproducibility and validation of model experiments

The compilation options used for the ACCESS climate configurations mean that results will be identical in runs on different numbers of processors. However changing compiler versions or options will lead to differences due to floating point roundoff. These grow rapidly which makes comparison and validation of results difficult. Proper comparison requires long climate means or an ensemble.

The NWP configurations give different results on different numbers of processors, though runs with the same decomposition are reproducible.

The standard Met Office procedure is to compare the magnitude of differences to those from a run with a perturbed initial condition.

This graph shows an example of this effect with runs of an APS-0 city configuration. The standard configuration used 64 cores and was built with the Intel 11.1.073 compiler. The new version uses the 12.1.8.273 compiler. The RMS difference of the sea-level pressure grows rapidly and reaches 4 Pa after about 20 hours (red line). Considered in isolation there's no way to know whether this is reasonable or not. The blue line shows the effect of changing the processor decomposition from 8x8 to 8x10.

The gray lines show the effect of perturbing the potential temperature in the initial dump. The initial dump here uses fields packed to 32 bits and a perturbation of 1e-4 K was about the smallest that changed values at all. The perturbations are random values in range -1e-4 to 1e-4 K at every grid point, same change at every level.

See also APS1G_rmsdifferences for experiments with APS1 G reproducibility and ACCESS1.0_reproducibility for experiments with the ACCESS 1.0 coupled model.

Attachments

• APS0_C_rms.png