MFN Maintenance and Batchin File Generation Procedures - CMAP-REPOS/MFN_Programs-and-skims GitHub Wiki

This page describes the procedures to maintain and update CMAP's freight network (MFN). For detailed methodology documentation, see the docstring at the beginning of each individual script. All procedures to maintain the MFN can be found within the 'Network_tools' folder in this repository.

Conceptual Background

CMAP's Master Freight Network (MFN) is primarily used to develop Freight Skim inputs that are used in CMAP's Freight Forecasting Model.

Master Freight Network (MFN)

The MFN is composed of different node and link networks. For the highway and rail portion of the network, additional network detail is provided for the CMAP region.

  • Highway
  • Rail
  • Inland Waterway
  • Crude Oil
  • Coal NEC
  • Petroleum

The MFN also contains several logistics nodes which represent key terminal locations in the region. There are two notable Logistics nodes in the MFN

  • 140: Crete Rail Terminal, this rail terminal is only active in certain model run scenarios (see below)
  • 143: South Suburban Airport, this airport is only active 2035 and after

Several procedures exist to maintain and update various features of the MFN. These procedures are intended to be ran as needed and do not need to follow a specific order. These procedures are all located in the 'Network_tools' folder. If a set of procedures does not exist for the edit you are trying to make, the edit must be performed manually in ArcPro. In this instance, it is recommended that step 1 below, 'Build_full_MFN.py' is completed first. Then all manual edits can be completed on the fully assembled geodatabase.

  • process_futureLinks.R - develops the CMAP freight region highway network node and link layers based on a user specified conformity MHN for all future years
  • update_logisticNodes.R - changes the location of logistics nodes in all the necessary layers based on user specified coordinates
  • update_POE.R - changes the ID of POE nodes; this script was used when the zone system was updated from zone09 to zone17

Several procedures are then ran as tools in the MFN_tools.atbx ArcPro Toolbox to produce the batchin files used for the freight skimming procedures. These procedures can be found within 'Network_tools/src'

  1. Build_full_MFN.py - combines the highway portion of the MFN (generated every conformity cycle) with the rail, waterway, and pipeline networks (stored on the V drive) into one complete geodatabase

NOTE: if any updates are to be made to the MFN that were not able to be completed with a pre-built script, those edits must be performed now to ensure they are included in the batchin files.

  1. batch_domestic_ITINERARIES.py - produces the rail itinerary batchin files
  2. batch_domestic_PIPELINE.py - produces the batchin files for each pipeline network (crude oil, natural gas, petroleum)
  3. batch_domestic_SCENARIOS.py - produces the highway network bachin files

A final script, 'Network_tools/qc_batchinFiles.R' compares one set of the batchin files to another to quickly identify key changes.

Final batchin files produced from these procedures, for each scenario, for each year:

  • Base_ntwk.txt: highway, railway, and waterway nodes and links
  • Cos_ntwk.txt: crude oil network nodes and links
  • Nec_19_ntwk.txt: coal NEC network nodes and links
  • P1718_ntwk.txt: petroleum network nodes and links
  • Lines.in: rail itineraries
  • DomesticNetwork.csv: domestic highway network distance
  • DomesticPipelineNetwork.csv: domestic pipeline network distance *
  • unlink_lognode140.txt: transaction file to remove logistics node 140
  • unlink_lognode143.txt: transaction file to remove logistics node 143