Batch processing of scenarios using ADSM Auto Scenario Runner - NAVADMC/ADSM GitHub Wiki

ADSM has a command line option that executes multiple scenarios without intervention, called ADSM Auto Scenario Runner. For example, you could run 25 different scenarios and the batch process will upgrade, verify and execute all 25 scenarios without additional user intervention. Feedback will be provided through the ADSM command window and written into a log file.

Some notes about executing ADSM batch process:

Closing the Command window with the X (upper right) will abort the process and close all work.

ADSM is designed to optimize the available processing resources on the computer where it is installed. If you have 4 processors available, ADSM will use the processors by threading an iteration to each processor. If you have 96 processors, ADSM will use all 96 processors.

ADSM is compiled for Intel processors. If your server is using another brand of processor, it is possible that errors will happen. Instructions to recompile ADSM are provided in the application documentation if you wish to customize the application to your setting. The ADSM Development Team will not provide support for this type of work.

Using a batch process can create large volumes of output. When creating Supplemental Output Files, note that an individual file is created per iteration. If you execute 10 iterations, you get 10 files for the selected output. If you execute 1,000 iterations, you get 1,000 files for the selected output. There are several ways to combine these supplement outputs. Within the ADSM application, there is a “Combine Output Files” button which will create combined files by output type, and will report metadata regarding the combined files for a single scenario. It is possible to use R or another tool to combine files. The Example R Code directory has R code to stack these files (StackedAllFiles.r).

During the batch process, outputs come in the same format as the ADSM application. Results are filled into the scenario database file (.db) and can be accessed with a SQL query tool. Any supplemental output files are created in the scenario file, in the Supplemental Output Files subdirectory. Refer to the Accessing the Results section of the ADSM wiki if additional help is needed to review these results. The exception is the Summary Map product, shown on the Results Home page. The application will not generate a complete summary map when prompted after the batch process has generated the data. At this time, invalid scenarios are attempted, give validation PASSED in the auto_output.txt and are skipped. The PASSED message is counter-intuitive, but the auto_output.txt does not report the invalid scenario as running or completing.

Preparing Scenarios

Scenarios should be parameterized using the ADSM interface. All settings on the Output Setting tab should be set as needed. For example, all Supplemental Outputs must be checked to be initiated and will generate the expected output files. The count of iterations should also be set. There is an argument that will be covered in this document that allows a limitation to the iteration count for testing purposes.

Monitoring the batch process

The Command window will provide real-time feedback as the batch process completes.

In addition, the auto_output.txt log will be created to support a batch run. This file will be created in your ADSM Application install directory, or as indicated in the --output-path argument. This log file gives a high level overview of the status of each scenario in the batch run.

Steps to run the batch process

The batch process runs from the directory where ADSM is installed. This requires navigation to the correct directory where ADSM was installed on your PC. It is expected that users be able to navigate to their specific install location using MS DOS. The command window must stay open to allow the batch process to complete.

An example of a simple batch command looks like:

ADSM.exe --run-all-scenarios

This command will run every scenario in the ADSM Workspace and put results into each scenario file in the ADSM Workspace. Additional arguments can be provided to customize the batch application. Note that each argument begins with the characters (--) or dash dash with no space before the argument. File names and path names are in double quotes. A more complex batch command could look like this:

ADSM.exe --run-all-scenarios --exclude-scenarios "Sample Scenario" "Sample Scenario with Outputs" --output-path "E:\ADSM\2020-06-10_ADSM_Output"

There are 3 groupings of arguments:

  1. Arguments to identify scenario files to run or exclude
  2. Arguments to redirect input source or process feedback output
  3. Arguments for other administrative actions

Each argument will be discussed individually.

Arguments to identify scenario files to run or exclude

--run-all-scenarios

This argument will automatically run all scenarios in your ADSM Workspace. A secondary argument can be used to exclude files.

  • All scenarios are run
  • Files are saved into regular database
  • Command line example:

ADSM.exe --run-all-scenarios

--run-scenarios

This argument will run only a selected set of scenarios. The list of included scenario names to run follow the main command. There is no need to add the file extension .db on the scenario names.

  • Only scenarios listed in the command line are run
  • Files are saved into regular database
  • Command line example:

ADSM.exe --run-scenarios “Sample Scenario”

--run-scenarios-list

This argument will run only a set of scenarios that are listed in a file. Create a text file and list the scenario names to run (one per line in the file). There is no need to add the file extension .db on the scenario names.

  • Only scenarios listed in the named list file are run
  • Files are saved into regular database
  • Command line example:

ADSM.exe --run-scenarios-list “C:/Users/MeSchoenbaum/Documents/ADSM Workspace/TestBatch.txt”

Hint: If you cut and paste from the file explorer, change the back slash to forward slash.

--exclude-scenarios

This argument accepts a list of scenario names in your ADSM Workspace that will be excluded from running. There is no need to add the file extension .db on the scenario names.

  • Scenarios listed in the command line are NOT run
  • Command line example:

ADSM.exe --exclude-scenarios “Sample Scenario”

--exclude-scenarios-list

This argument is similar to the previous --exclude-scenarios, except that it reads the excluded file names from a file instead of having to type each filename into the command line. This would be helpful if you have a long list of files to exclude. Include the file name of a text file to exclude the list from running. (one per line in the file). There is no need to add the file extension .db on the scenario names.

  • Scenarios listed in the file (one per line) are NOT executed
  • Command line example:

ADSM.exe --run-all-scenarios --exclude-scenarios-list “C:/Users/MeSchoenbaum/Documents/ADSM Workspace/TestBatch.txt”

Arguments to redirect input source or process feedback output

--workspace-path

This argument allows you to specify a path to a custom workspace folder instead of using the assigned ADSM Workspace. This can be useful if you want to drop a group of scenarios into a specific folder and use --run-all-scenarios argument. For example, you could create a specific input path for a given project, and all the output will go back into that same path.

Please note that the ADSM application will not recognize these scenarios in the ADSM Viewer. Recall that users choose the location of their managed workspace during ADSM installation. The managed workspace location is shown at the top of the Project Panel for clarity. Scenarios can easily be moved from the custom batch workspace defined by this argument back into the managed ADSM Workspace. It will be necessary to close and reopen ADSM for the application to recognize scenarios after a file move.

  • Scenarios in the specified path are run
  • Files are saved into database in the specified path
  • Scenarios will NOT be recognized by the ADSM application, as they are outside the managed workspace
  • Command line example:

ADSM.exe --run-all-scenarios --workspace-path “C:\Users\MeSchoenbaum\Documents\ADSM Batch Test Workspace”

--output-path

This argument allows you to specify a path to customize the location of the output log and iteration logs. The default location for output is in the ADSM install directory. Note these are not model results, but instead are the feedback specifically from the batch process. Combining the - -output-path argument with the --workspace-path argument would support the concept of putting all the output results and logging files into a single directory location that could be easily zipped.

  • This argument has nothing to do with scenario inputs or results datasets. It only controls system feedback in log files.
  • Command line example:

ADSM.exe --run-all-scenarios – -output-path “C:\Users\MeSchoenbaum\Documents\ADSM Batch Test Workspace\AllBatchOutput”

--store-logs

This argument specifies the storage/retention of the logs from each scenario. If not specified, the logs will not be created.

Arguments for other administrative action

--max-iterations

This argument allows you to limit the iterations any single scenario can run. For example, to test the batch process you may want to run a few iterations of each scenario without having to modify each scenario’s iteration count parameter.

  • A limited number of iterations are run
  • Command line example:

ADSM.exe --run-all-scenarios –max-iterations 3

--quiet

This argument allows you to bypass the prompt about potentially losing unsaved work. The default will prompt you once at the beginning of the batch process execution with messages:

Warning! Using the ADSM Auto Scenario Runner will cause you to lose any unsaved work in the Scenario currently open in ADSM (activesession.db). Please make sure all current changes are saved back to your Scenario in the ADSM program.

Warning! Using the ADSM Auto Scenario Runner will delete existing Results and Supplemental Output files on all scenarios that are run.

--store-logs

This argument allows you to save troubleshooting logs, which are a secondary form of outputs that are created to support troubleshooting. These troubleshooting logs are also created by the ADSM application and are not usually accessed. These files are described in the wiki Logging/Troubleshooting.

During a batch run, log files are created in a default location. An automatic output file of the raw data will be created in the ADSM install directory. Please note that this will be specific to your PC, and any examples provided are based on the ADSM Development Team test environment. Logs have the complete output dataset in an unparsed format as it is created in the ADSM C Engine. The data is written into the database in a parsed format that is much easier to understand and use. Again, logs are only for troubleshooting to allow the ADSM Development Team to see raw output.