• Introduction
• Preliminaries
• Quick Start
• Compiling with MESA
• Outputs
• Analysis

Download and build Charm++ the same way for regular ChaNGa.

In the changa directory, checkout the proper branch

git checkout moving-mesh

Enable moving mesh with the configure command:

./configure --enable-moving-mesh --enable-cooling=ad

Note that cooling modules are needed to be specified. In the above example ad specifies adiabatic cooling, which is different than running with SPH. There are a number of cooling options that works with MANGA. These are:

• ad - adiabatic/ideal gas cooling; use this instead of cooling=none
• mesa - MESA EOS -- requires a modified MESA build (must specify MesaDir in parameter file)
• sroeos - Nuclear equation of states from stellarcollapse.org

ad is the usual preference and works with other modules including radiation (--enable-radiation) and GR (--enable-gr). Please note that GR is not yet publicly available.

Now make it

make -jN

where N is the number of jobs will now build a moving mesh version of ChaNGa. The first build will usually fail as there is a dependency that is not yet fixed in the Makefile. Just do another make and you will be fine.

To generate the initial conditions for common envelope requires an installation of MESA, which is a pain. Fortunately, we have two test cases in ''moving-mesh-ICs/test-cases'' in ''cee-test'' and ''tde-test''.

Create a work directory somewhere

$ cd (scratch-directory)/(test-case)

copy everything in ''cee-test'' or ''tde-test'' to this directory. Create a link to or copy ChaNGa there.

$ ln -s (changa-directory)/ChaNGa .

Now you are ready to run.

./ChaNGa +ppn (num_of_tasks) tde.param or cee.param

For those who are running cee.param, you need to edit cee.param to disable bDoublePos and bDoubleVel if you want to use yt below

#bDoublePos      = 1
#bDoubleVel      = 1

in test-moving-mesh/ic-generator, make the initial conditions using

python makeTestInitialConditions.py sedov

This will generate the initial conditions file, sedov.std.

The parameter file is called sedov.param. Put it along with the generated binary ChaNGa and the initial conditions file in the directory to run and run it using:

./ChaNGa sedov.param

This will only take a few minutes.

MANGA works well with the equations of state from the MESA code (release 10398 and 11701). It may work with later releases, but at some point the API was modified and so the latest release will probably not work.

To get it work, download MESA and compile MESA using the same compiler as you built Charm and ChaNGa. It can work with the mesasdk, but this may limit your runs to a single node.

To enable mesa in ChaNGa be sure to set MESA_DIR

export MESA_DIR=<mesa directory>

and configure with the option --enable-cooling=mesa

./configure --enable-moving-mesh --enable-cooling=mesa

and build as before.

in test-moving-mesh/ic-generator/mesa, make the initial conditions using

python makeMesaIC.py <datafile>

The datafile are profiles from MESA. An example profile is 2msunRG.data, which is a 2 solar mass red giant with a radius of 52 solar radii and generated by mesa release 10398.

makeMesaIC.py has a number of command line options. Use --help to see them, but the most relevant are

*--add_companion ADD_COMPANION -- adds dark matter companion, we use ADD_COMPANION=1 to generate a 1 solar mass secondary. *--corotation COROTATION -- rotation of primary as fraction of companions orbit. In Prust and Chang (2019) we use COROTATION=0 and COROTATION=0.95

One thing you will note is that MaNGa needs to fill the computational volume with cells. So it will fill the rest of the box with a expanding grid of atmosphere particles.

makeMesaIC.py is similar to other scripts to build initial conditions. One thing to keep in mind is that in moving-mesh codes, the space must be filled with particles or mesh generating points. Thus, xmax in makeMesaIC.py must correspond with half of the value of dPeriod in the parameter file.

Copy the ChaNGa binary, cee.param parameter file, and star.std generated by the initial conditions generator to a running directory. An example cee.param file is located in moving-mesh-ICs/test-cases/cee-test. You need to set

MesaDir = <mesa directory>

in cee.params so that the MESA EOS data can be found. This problem can also be run with --enable-cooling=ad. This is significantly faster, almost as good, and a bit more stable, but it does not use a MESA EOS, which is important in many circumstances.

Because the MESA equation of state loads the EOS data once per running instance, it is important to use threads as much as possible. So for instance, good throughput can be achieved with the following

mpirun -n <num of nodes> ChaNGa +ppn <number of cores per node> +setcpuaffinity <parameter file>

Build ChaNGa with --enable-cooling=ad. While mesa can be used, adiabatic equations of state make things a bit faster.

In test-moving-mesh/ic-generator/mesa, make the initial conditions using

python makeTDE.py <datafile>

Like the CEE problem, the datafile are profiles from MESA. A good profile for this is 1msun.data, which is a solar mass zero age main sequence star.

makeTDE.py has a number of command line options. Use --help to see them, but the most relevant are

*--beta=<number> -- this is r_p/r_t, where r_p is the pericenter, and r_t is the tidal radius default is 2. *--num_particles=<number> -- number of particles to represent the star (default is 200K)

The same rules apply for makeTDE.py as makeMESAIC.py described above.

Copy the ChaNGa binary, tde.param parameter file, and tde.std generated by the initial conditions generator to a running directory. An example tde.param file is located in moving-mesh-ICs/test-cases/tde-test.

The main output is again similar to ChaNGa: a tipsy binary file, but there are extra files (output as tipsy arrays). These include the primitive variables in the cells: density (rho), internal energy (ie), and fluid velocities (vx, vy, vz). The values in the tipsy binary files are also meaningful. Mass is the product of the density (rho) and the volume of the Voronoi cell. The particle velocities correspond to the velocities of the mesh generating points (usually similar to, but not guaranteed to be the same as the fluid velocity. The temperature is the temperature of the point based on the internal energy.

It is recommended that the primitive variables (rho, ie, vx, vy, vz) that is output as auxillary files be used in plots and analysis. However, an exception can be made for temperature as this is effectively a primitive variable.

Like ChaNGa, the key constraint for units for MANGA is G = 1. Typically the runs are set such that the length and mass are in cgs units (though it is not formally a requirement). This means that the two dimensions in the parameter file are mass in "dMSolUnit = 5.02879e-34", i.e., 1 gram, and length in dKpcUnit = 3.24076e-22, i.e., 1 cm, respectively. The time dimension is determined from these two, which in cgs units is t = G^{-1/2} = 3863.3 seconds.

YT has been the analysis framework that is useful for analyzing and visualizing simulation outputs. See the yt webpage for information on how to install it. For instance, the plot a projection

import yt

# load the dataset
dataset = yt.load(<filename>)

yt.ProjectionPlot( dataset, "z",center='m',width=4e13).save(<output.png>)

On the other hand, one can do analysis on the primitive variable as

import yt

# load the dataset
dataset = yt.load(<filename>)

# get all the data
ad = dataset.all_data()

# get coordinates of the cell center and primitive variables 
coordinates = ad[("Gas", "Coordinates")]
rho = ad[("Gas", "rho")]
vx  = ad[("Gas", "vx")]
vy  = ad[("Gas", "vy")]
vz  = ad[("Gas", "vz")]
ie  = ad[("Gas", "ie")]

# perform analysis

Examples of a plotting routine and analysis routine is in moving-mesh-ICs/test-cases/cee-test/plot.py, moving-mesh-ICs/test-cases/cee-test/orbit.py, and moving-mesh-ICs/test-cases/tde-test/plot.py