Getting started - tlumip/CALM GitHub Wiki

The CALM model is a collection of different travel modeling components. The core components of the model system are:

• JEMnR – ODOT’s trip-based household travel demand model, implemented in R.
• Visum and Python – Zone and network data management, as well as network skimming and assignment procedures.
• University model - a tour-based model implemented in Java, using the CT-RAMP Common Modeling Framework (CMF) library of tools.
• Commercial vehicle model (CVM) – ODOT’s trip-based commercial vehicle model implemented in R.
• External model – ODOT’s external travel model based on select link analysis flows from SWIM at each model external station location, implemented in R.

The model system is run from a DOS batch program, RunModel.bat.

The modeling machine must be capable of running Visum, but this model does not otherwise require extensive machine resources.

The model is configured to run on Windows, and uses Visum, Python, Java, and R. To ease installation and setup, complete installations of Python, Java, and R are included in a software dependency package and are used when running the model.

• Python is Visum’s preferred scripting tool, Java is used for the University model, and R is used for JEMnR, the commercial vehicle model, and the external model. The software dependency package includes the required third-party libraries as well.
• Visum is a commercial transport modeling software package developed by PTV.

First, download the model program files from GitHub. It is best to clone the GitHub repository CALM to the machine that the model is going to be run, either via the GitHub Desktop software or via command line with git installed. Cloning allows for easy updates by pulling down newer changes or pushing up local changes. Alternatively, GitHub offers a simple ZIP download but this does not include git's version control features.

To clone with GitHub desktop, first sign in with a GitHub.com account with access to the repository. Click on File, then Clone a repository. Search for tlumip/CALM, click on the correct result, and select a local folder location. Click Clone to start the download.

If using a command line, follow the directions below. Otherwise, skip to Software Dependencies.

# open a dos prompt, navigate to or create the desired directory, and type:
git clone https://github.com/tlumip/CALM.git

The CALM repository is currently private and requires user authentication to be cloned. When using the command line, GitHub requires users to authenticate using a Personal Access Token over HTTPS or public/private keys via SSH. Username and Password authentication is no longer allowed.

Complete installations of specific versions of Python, Java, and R are kept in a "dependency package." This ensures that the correct version of software is always available to the model directory regardless of software installed elsewhere on the computer. CALM uses the same dependency package developed for Southern Oregon Activity-based Model (SOABM), which includes the following software packages:

• Java JDK 1.8.0_111
• Pandoc (SOABM visualizer requirement, but not used in CALM)
• Python 2.7.3
• Python 3.7.4
• R 3.4.1

Neither git clone nor the GitHub ZIP file include the contents of the dependencies folder. The dependencies are the same as those in SOABM and can be copied if available locally or obtained from the SOABM GitHub repository. A direct download of dependencies.zip is available as well.

On initial checkout/download, the user will need to:

• Expand dependencies.zip into a directory at the same level as the template folder since it will be used by multiple scenarios.
• Create and rename a copy of the template folder to create a new scenario.

7-zip is required to be installed in order to generate ZMX matrix files for the University model.

The directory structure for the CALM model consists of a software dependencies directory and a template model setup directory. The template directory can simply be copied to create a new scenario. The template directory structure is shown below. A brief description of the components of the directories follows.

The template directory should have a file structure as shown below (all paths relative to the root directory):

Following is a description of the model directory contents, including output folders created during model runs and an example scenario folder:

root/
│
├── dependencies/
│   ├── jdk1.8.0_111/ – Java installation directory
│   ├── Pandoc/ - used only in SOABM
│   ├── Python27/ – Python 2.7 installation directory
│   ├── Python37/ – Python 3.7 installation directory
│   └── R-3.4.1/ – R installation directory
│
├── template/
|   ├── RunModel.bat – Script used to run model
|   ├── modelRunner.R - Script for running JEMnR model components
|   ├── VISUM_Runner.py - Script for skimming and assignment via Visum API
|   ├── functiontable.csv - List of R scripts to source for necessary JEMnR functions
|   ├── modelReport.html - JEMnR HTML model report
|   ├── access/ - JEMnR intermediate files for accessibilities
|   ├── inputs/ - JEMnR inputs including settings and TAZ data
|   |   ├── cvm/ - production, attraction, and friction factors for the Commercial Vehicle Model
|   |   ├── externalModel/ - external model inputs and select links
|   |   ├── PopSyn/ - population synthesis inputs
|   |   ├── RData/ - data read into R from CSV or OMX is stored here in RData format
|   |   └── utilities/ - CSV files containing coefficients, factors, generation rates, or utilities for several model steps      
|   ├── modec/ - JEMnR intermediate files for mode choice
|   |   └── ... contains output folders for each trip purpose, plus a common folder
|   ├── outputs/ - skims, demand matrices, and loaded Visum networks
|   |   ├── matrices/ - OMX skim and demand matrices
|   |   └── networks/ - loaded Visum networks for daily and peak periods
|   ├── peaking/ - JEMnR intermediate files for P-A to O-D and TOD
|   |   └── ... contains output folders for each trip purpose plus cvm and trip matrices in RData format
|   ├── pregen/ - contains output household workers, children, and car ownership RData files
|   ├── rcode/ - JEMnR source code
|   |   ├── access/ - accessibility scripts
|   |   ├── comVeh/ - Commercial Vehicle Model scripts
|   |   ├── modec/ - mode choice scripts
|   |   ├── OMX/ - Open Matrix (OMX) and Zip Matrix (ZMX) APIs for R
|   |   ├── peaking/ - Time-of-Day peaking and directional scripts
|   |   ├── pregen/ - accessibility; household workers, children, vehicle ownership models
|   |   ├── tripdist/ - trip distribution scripts
|   |   ├── tripgen/ - trip generation scripts
|   |   ├── university/ - scripts to split University and non-University populations and call the University model from R (does not contain University model code or executable)
|   |   ├── inputCheck_V4.R - runs logical checks on inputs and produces warning messages
|   |   ├── inputs.R - sets up common vectors and scalars, reads in CSV files as R objects
|   |   ├── inputsSave.R - converts inputs, including skims, to RData format
|   |   ├── jemnrFunctions.R - common functions used across JEMnR scripts
|   |   ├── ModelModules.r - main script controlling model flow
|   |   ├── modelReport.R - script to generate the HTML report
|   |   └── writeToOMX.R - generates demand matrices in OMX format; replaces writeToBank.R
|   ├── resultsAnalysis/ - images and supporting files for JEMnR HTML model report
|   ├── tripdist/ - JEMnR intermediate files for trip distribution
|   ├── tripgen/ - JEMnR intermediate files for trip generation
|   ├── unimodel/ - University model inputs, scripts, Java binary
|   |   ├── ctlfiles/ - University model properties file and Utility Expression Calculators (UECs)
|   |   ├── documentation/ - University model documentation (Word documents)
|   |   ├── emmemat/ - output folder for model skims in ZMX format
|   |   ├── outputs/ - University model outputs folder
|   |   |   ├── iter0/ - first iteration output (contains output CSV files)
|   |   |   ├── iter1/ - second/final iteration output (contains output CSV files, copied one level up to outputs/)
|   |   |   ├── householdsOut.csv - University population households output file
|   |   |   ├── personsOut.csv - University population persons output file
|   |   |   ├── workersByOccupationAndTAZ.csv - Number of workers by occupation for each TAZ for University population
|   |   |   ├── parkingDemand.csv - University parking lot demand by zone, space type, and half-hour time period
|   |   |   ├── UnmetParkingDemand.csv - University parking demand
|   |   |   ├── tours.csv - disaggregate output tours list
|   |   |   └── trips.csv - disaggregate output trips list
|   |   ├── rpt/ - log files
|   |   ├── EMXtoZMX.py - Python script to convert Emme matrices to ZMX (no longer used)
|   |   ├── gpl.exe - helper executable
|   |   ├── households.csv - University population households input
|   |   ├── jar_context.txt - update notes regarding the Parking Lot Choice model
|   |   ├── lcogmc34.exe - (unknown executable)
|   |   ├── nextemod.bat - (unknown: maybe related to Emme EMX files, appears unused)
|   |   ├── Parking_Capacity.csv - University parking lot spaces by TAZ input
|   |   ├── persons.csv - University population persons input
|   |   ├── RunUniversityModel.cmd - batch file to run the University model
|   |   ├── ss11hRVMPOwithzeros.csv - households
|   |   ├── ss11pRVMPOwithzeros.csv - persons
|   |   ├── tazData.csv - University model TAZ input
|   |   ├── tpau.jar - University model Java archive
|   |   ├── UniTAZs.csv - list of University zone IDs
|   |   └── universityAccessibilities.csv - University model logsums by market segment and trip purpose from each zone to all zones
|   └── visum/ - input version file and procedures
|       ├── procedures/ - procedure sequences in XML format
|       └── CALM.ver - input version file for the Corvallis-Albany-Lebanon Model (if this file name changes, update the visumInputFile setting)
│
└── scenario1/
    ├── RunModel.bat
    └── ... Scenario folders have all the same contents as the template folder above

The access, modec, peaking, pregen, resultsAnalysis, tripdist, and tripgen folders are generated during a model run.

To create and run a new model scenario (run), do the following:

1. Ensure the required dependencies folder is in place. See Software Dependencies for details.
2. Create a scenario folder by duplicating the template folder and renaming the new folder. Every scenario is contained within its own folder, with a unique name. The dependencies folder must be at the same directory level as the template and scenario folder(s). This allows several scenario runs to make use of a single dependencies folder.
3. Edit the necessary input files for the scenario, which at a high level can be thought of as Network and Land Use changes:
   1. Network – If there are network revisions for the scenario, edit the input version file in Visum.
   2. If any edits are made in the version file, the user should ensure that the revised version file passes all relevant Network checks under the Calculate -> Network check function in Visum. All checks may not be necessary. Typically the user should be able to ensure the follow checks complete without issue:
      • Isolated nodes
      • Zones not connected for PrT
      • Check network consistency (for all auto modes)
      • Dead-end roads PrT (for all auto modes)
      • Links without succeed. link (for all auto modes)
      • Links with Capacity PrT = 0 or V₀ = 0
   3. TAZ land use data – If there are population, employment, or other zone related revisions, edit inputs/taz.csv
4. Run the Scenario
   1. Open a DOS command prompt in the scenario folder: clear the location bar, then type cmd and hit enter to launch the command prompt or via Shift + Right click then Open Command Window Here (or "Open PowerShell Window Here")
   2. Type RunModel.bat in the DOS window and hit enter to run the model (if in PowerShell type .\RunModel.bat). Prior to running the RunModel.bat file the user can consider if any changes to the bat file are needed such as the number of iterations. More information on making changes to the bat file can be found on the RunModel batch file page.