RunningInGridOrBatchMode - ccdc-opensource/dash GitHub Wiki
Running DASH in Grid Mode or Batch Mode
Overview of DASH in Batch/GRID Mode
Structure solution can now be performed by DASH using distributed processing to speed up the procedure. This can be achieved in a number of ways depending on the resources at your disposal whether this consists of a single-processor computer, a multi-core machine, or an entire network of computers. The use of distributed computing can be particularly helpful when individual SA runs are long due to a complex problem.
Setting up Input Files
During the process of setting up a series of SA structure solution runs there is the option of creating batch files (see Simulated Annealing Options): clicking on the Create Batch File button will access the Generating GRID batch files dialogue box. This section of the wizard will allow you to create the input files necessary to run DASH in a distributed fashion.
The first step is to choose the total number of SA runs that are required; these runs are then split into packages with a user-defined number of runs in each. If, for example, the user chooses 500 SA runs, split into packages of 10 SA runs each, this will produce 50 packages. Each package is represented by a .dbf file which gives DASH the instructions for that set of SA runs. A .grd file is also created which lists the filenames for each of the separate packages. For running DASH across a GRID, it is recommended to use packages containing just one SA run each, this allows greater flexibility when assigning packages to different nodes of the network.
The .dbf files that are created store all of the parameters defined for the SA runs, so if a particular option is chosen in the Simulated Annealing Options window then it will be adhered to in batch mode.
Note: The .dbf files are in simple text format and can be edited using a basic text-editor such as Notepad or Wordpad in order to, for example, modify one parameter then re-run a set of SA runs.
To generate the input files, click on the Write > button. At this point you will be prompted for the output directory and the filename for the .grd file. The.dbf files and the .dash files (when the jobs are finished) will be named using this filename as the base along with a number corresponding to the particular SA run package.
Using DASH in Batch Mode
Drag-and-Drop Running
Individual .dbf files can be dropped on to the DASH executable or the shortcut icon on your desktop. In this mode DASH will run in the background without any displayed user interface. Multiple SA packages can be run concurrently in this manner but no progress or final results will be shown. The output files will be sent to the directory where the input files were residing (see Post-Processing of Results).
Multi-Core Processing with MDASH
For computers with multi-core processors there is also a currently unsupported tool called MDASH which allows the controlled distribution of sets of SA runs across the system. The tool provides an interface displaying the progress of the SA runs and will pop-up the DASH program to show the results when completed. For further information on MDASH see the MDASH documentation within the DASH\Unsupported Extras directory.
Using DASH in GRID Mode
For users with access to a GRID-enabled network of computers, DASH can also be run in GRID-mode. To achieve this it is necessary to set up the client PCs and the GRID servers for distributed DASH. This application of DASH is not currently fully supported, but installers to set up the computers to use DASH across the GRID are available from the DASH page of the CCDC website (http://www.ccdc.cam.ac.uk/Solutions/PowderDiffraction/Pages/DASH.aspx) along with further documentation.
Post-Processing of Results
After running DASH in batch mode or in GRID mode there will usually be a large number of .dash results files output. In order to process these results it is generally easier to combine all of the files into one large .dash file. There are two ways of doing this:
-
Open a command prompt and navigate to your results directory, then type “C:\Program Files\CCDC\CSD_2021\DASH\DASH.exe MERGE D:\Results Directory\ output.dash”. This will automatically merge all the .dash files in the directory (D:\Results Directory) to produce the file output.dash.
-
Open a command prompt and navigate to your results directory, then type “C:\Program Files\CCDC\CSD_2021\DASH\DASH.exe MERGE”. A window will then appear which will allow you to navigate to your results directory and give the name for your merged dash file.