access_ACCESS1.0_reproducibility - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

Reproducibility experiments with the ACCESS 1.0 coupled model

When testing the ACCESS 1.3 coupled model on raijin, Hailin noticed that the results were different to vayu, even when using the same executables.

hailin_diffs.png

Separate experiments with the atmospheric model showed that it gave identical results when run in AMIP mode (same executable). In order to get reproducibility of results across different processor decompositions it's necessary to compile the UM with the -fp-model precise option (for details see http://software.intel.com/en-us/articles/consistency-of-floating-point-results-using-the-intel-compiler). The difference across machines is likely due to the longer vector registers on raijin. This option was not used with either the CICE or MOM builds. However if MOM is built with the -fp-model precise option the coupled model gives identical results on vayu and raijin (with the vayu executables).

To estimate the significance of these differences a set of runs with various perturbations was done with 1 year of the ACCESS 1.0 PI control.

  • Control: Run on vayu using the CMIP5 executables and processor decompositions
  • Run1: As for control, but MOM uses a 10x4 rather than 8x5 decomposition
  • Run2: MOM built with fp-model precise
  • Run3: As for control but with a random perturbation to the theta field in the atmospheric model initial condition (amplitude 1e-9 K).
  • Run4: MOM built with the 11.1 compiler rather than 11.0
  • Run5: Raijin run with the standard CMIP5 executables from vayu
  • Run6: Raijin version of run2 (was identical to run2)
  • Run7: Raijin run using executables built on raijin with default 12.1.9.293 compiler

The control run here was identical to the original CMIP5 run (data archived on dcc).

rms_coupled60.png, 60%

RMS SST difference over the first two months using daily data

rms_coupled.png, 60%

RMS SST difference over the first year using monthly mean data

sst.png, 60%

Global mean SST

These results show that the differences between vayu and raijin are of the same order as those caused by the smallest possible perturbation to the initial conditions.# Attachments