Protocol for New ChaNGa Release

The following is for ChaNGa/Charm developers. It outlines the testing to be done for a new ChaNGa release. Historically ChaNGa releases have followed Charm releases, so both software distributions are being tested in this process.

Charm builds

The latest release (or release candidate) version of Charm should be downloaded/pulled.

The build command should be executed by ./build ChaNGa COMM-SYSTEM --with-production, where COMM-SYSTEM refers to the communication layer and system being used, e.g. mpi-linux-x86_64. Tests should also be repeated with the smp option added to the build command.

ChaNGa builds

The target release or release candidate verion of ChaNGa should be downloaded/pulled.

configure should be run 1) with no arguments 2) with --enable-bigkeys 3) with --enable-cooling=cosmo, with all the tests below (except for the moving-mesh test) repeated in each configuration.

ChaNGa tests

1. Successful compilation with no errors

2. Basic timestepping/energy conservation: in the teststep subdirectory, the Makefile has a testenergy target which will run this test, but the command may need to be edited depending on the system.

3. Checkpoint/restart test: also in the teststep subdirectory, there is a second target testrestart if restarts from a checkpoint can run.

4. Cosmology test: in testcosmo is a small cosmological simulation. Note that two auxiliary programs skid and so are needed to verify the results, as well as the IDL (or GDL) package. Skid can be download from the N-bodyShop github, SO can also be downloaded from the N-BodyShop github

5. Gas collapse test: in testcollapse is collapse of a gas halo. Note that an auxiliary program tipsy is need to verify the results. Tipsy can be downloaded from the N-BodyShop github

6. Moving mesh test: in moving-mesh-ICs/test-cases/cee-test. ChaNGa must be compiled in moving-mesh mode, which uses configure --enable-moving-mesh --enable-cooling=ad. Note that we do not use --enable-bigkeys and --enable-cooling=cosmo here. Run the testing program orbit.py via python orbit.py in the same directory as the run and orbit.data is available in the same directory as well. Be sure that yt is installed. Easiest method is getting the Anaconda python distribution and using conda to install yt.

Benchmarks

Directions to run, and previous results of 5 gravity benchmarks are given on the benchmark page.

Test Results

Version 3.5 (Charm++ v7.0.0)

Machine	Date	Charm++ Build	Compilation	teststep	testcosmo	testcollapse
OLCF Summit	10/27/2021	pamilrts-linux-ppc64le-smp	O	X (testrestart hangs with 2 processes each with 2 PEs)	X (missing IDL/GDL)	X (can't bulid tipsy due to missing X11/Xaw/Scrollbar.h)
PSC Bridges-2	11/3/2021	ucx-linux-x86_64-smp	O	O	X (missing IDL/GDL)	O
SDSC Expanse	8/13/2022	`verbs-linux-x86_64 smp	O	O	O	O
TACC Stampede2	10/28/2021	mpi-linux-x86_64 smp	O	O	O	O
TACC Frontera	8/5/2022	mpi-linux-x86_64 smp	O	O	O	O
TACC Longhorn	8/18/2022	`mpi-linux-ppc64le smp cuda	O	O	O
NERSC Cori	11/10/2021	gni-crayxc	O	O	O	O
NASA Pleiades	8/12/2022	verbs-linux-x86_64 smp	O	O	O	O
NASA Pleiades GPU	8/17/2022	verbs-linux-x86_64 cuda smp	O	O	O	O