SoundCast to BKRCast - bellevuewa/BKRCast GitHub Wiki
The folder psrc_to_bkrcast_scripts provides Python scripts that are used to convert SoundCast data to the BKRCast model's format. These scripts are not run part of a model run; instead, these scripts are used to convert data in the SoundCast model's format to the BKRCast model's zone system. if required.
A script is run by opening a command window in the scripts folder (psrc_to_bkrcast_scripts) and running the following:
[python_executable] [script_name]
where;
python_executable=Anaconda python executable path. Ex. C:\Progra~1\Anaconda2\python.exe; script_name=one of the scripts in the list below
Here is a list of the scripts available in the folder:
| Name | Description |
|---|---|
| psrc2bkr_trucks.py | Updates zones in truck inputs |
| psrc2bkr_parcel.py | Updates zones in primary parcel file |
| psrc2bkr_calibration.py | Updates zones in calibration summary input files |
| psrc2bkr_hhs.py | Updates household zone in synthetic population |
| psrc2bkr_auto.py | Updates origin and destination zones in truck model input |
| psrc2bkr_daysimtrips.py | Updates origin and destination zones in DaySim seed trips |
| psrc2bkr_intrazonal.py | Updates from/to zones for intrazonal inputs |
| psrc2bkr_skims.py | Updates zones in seed skims |
| psrc2bkr_tazdistricts.py | Updates district correspondence in the TAZ index file |
| psrc2bkr_trips.py | Updates zones in supplemental trips |
| psrc2bkr_skims_tod.py | Updates group name in skim files |
| psrc2bkr_zonedata.py | Updates zones in supplemental model zone data |
| psrc2bkr_zonegroups.py | Updates PUMA zone groups and transit zones |
| daysiminputcheck.py | Checks consistency between DaySim parcel file and network skims |
| batchRepackH5.py | Repackages HDF5 files into smaller size |
| run_bkr_sampling_tests.py | Runs sampling test runs |
psrc2bkr_trucks.py
Objective: This script updates truck inputs located under inputs/trucks: trucks.in, special_gen_light_trucks.in, special_gen_medium_trucks.in, medium_trucks_ei.in, medium_trucks_ie.in, medium_trucks_ee.in, heavy_trucks_ei.in, heavy_trucks_ie.in, heavy_trucks_ee.in, special_gen_heavy_trucks.in, agshar.in,minshar.in, prodshar.in, equipshar.in, tcushar.in, whlsshar.in, and hhemp,const.in.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:\Projects\Clients\bkr\ model\soundcast\inputs\ trucks" | Working directory containing truck input files; this should be directory with SoundCast data files |
| runSpclGen | True | True to convert truck special generator inputs in files_spclgen |
| runExt | True | True convert external truck input files in files_ext |
| runTruck | True | True to convert truck input file in files_truck |
| runShares | True | True to convert truck share files in files_manu_shares and file_wtcu_shares |
| runEmp | True | True to convert employment related input files in files_emp |
| files_truck | ["trucks.in"] | Truck file |
| files_spclgen | ["special_gen_light_trucks.in", "special_gen_medium_trucks.in", "special_gen_heavy_trucks.in"] | Special generator truck input files |
| files_ext | [ "medium_trucks_ei.in", "medium_trucks_ie.in", "medium_trucks_ee.in", "heavy_trucks_ei.in", "heavy_trucks_ie.in", "heavy_trucks_ee.in"] | External truck input files |
| files_manu_shares | ["agshar.in", "minshar.in", "prodshar.in", "equipshar.in"] | Manufacturing truck share files |
| file_wtcu_shares | ["tcushar.in", "whlsshar.in"] | Wholesale and TCU truck share files |
| files_emp | ["hhemp", "const.in"] | Employment-related truck files |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKRCast zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent. Percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data files are produced in the working directory (wd) with "_bkr" appended to input file names.
psrc2bkr_parcel.py
Objective: This script updates zones in primary parcel file: inputs/accessibility/parcels_urbansim.txt
Settings: The user needs to update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:\Projects\Clients\bkr\ model\bkrcast_tod\inputs\ accessibility" | Working directory containing the primary parcel file; this should be directory with SoundCast data file |
| parcel_file | 'parcels_urbansim_psrc.txt' | Primary parcel file name |
| parcel_bkr_taz_file | "parcel_updated_bkrtaz.csv" | Parcels to BKRCast zone system correspondence; this file should be in the same directory as the script |
Outputs: The converted data file is produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_calibration.py
Objective: This script updates zones in inputs files used in calibration summary. The input files are located under scripts/summarize/inputs/calibration: survey.h5, TAZ_TAD_County.csv, and FAZ_TAZ.xlsx
Settings: The user needs to update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| calibration_dir | r'E:\Projects\Clients\bkr\ model\soundcast\scripts\ summarize\inputs\calibration' | Calibration directory that contains files to be converted |
| zonefiles | ["TAZ_TAD_County.csv", "FAZ_TAZ.xlsx"] | Zone correspondence input files |
| survey_file | "survey_psrc.h5" | Survey file |
| inputs_dir | r'E:\Projects\Clients\bkr\ model\bkrcast_tod\inputs' | Input directory that contains BKRCast buffered parcel file |
| parcel_file | "buffered_parcels.dat" | BKRCast buffered parcel file name |
| survey_file_new | "survey.h5" | The converted survey file name when runDaySimFormat is run independently |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
| zone_file | "TAZ_TAD_County.csv" | Zone-to-district and county correspondence file |
| faz_file | "FAZ_TAZ.xlsx" | Zone-to-FAZ correspondence file |
| district_lookup_file | "district_lookup.csv" | Zone-to-district lookup file |
| zoneLatLongFileName | 'bkr_zone_lat_long.csv' | Zone latitude and longitude file in the same directory as the script |
| runSurveyFile | True | True if survey file needs to be converted |
| runDaySimFormat | True | True if the converted file needs to be converted to DaySim text format (*.dat) |
Outputs: The converted data files are produced in the inputs directory (calibration_dir) with "_bkr" appended to file names.
psrc2bkr_hhs.py
Objective: This script updates household zone in synthetic population: inputs/hh_and_person.h5.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | "E:/Projects/Clients/bkr/ model/soundcast/inputs/" | Working directory that contains PopSyn input files |
| popsynFileName | "hh_and_persons.h5" | Synthetic population file name |
| parcel_dir | "E:/Projects/Clients/bkr/ model/soundcast/inputs/" | Directory containing the BKRCast buffered parcel file |
| parcelFileName | "buffered_parcels.dat" | BKRCast buffered parcel file name |
Outputs: The converted data file is produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_auto.py
Objective: This script updates origin and destination zones in truck model input: inputs/4k/auto.h5.
Settings: The user needs to update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | "E:/Projects/Clients/bkr/ model/soundcast/inputs/4k/" | Working directory that contains input file |
| auto_file | "auto.h5" | Auto file name |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
| num_bkr_zones | 1530 | Total BKRCast zones |
Outputs: The converted data file is produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_daysimtrips.py
Objective: This script updates origin and destination zones in seed DaySim trips: inputs/daysim_outputs_seed_trips.h5.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | "E:/Projects/Clients/bkr/ model/bkrcast_tod/inputs/" | Working directory that contains seed DaySim trips file |
| seedTripsFileName | "daysim_outputs_seed_trips.h5" | Seed DaySim trips file |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data file is produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_intrazonal.py
Objective: This script updates from/to zones for intrazonal inputs located under inputs/IntraZonals: origin_tt.in, destination_tt.in.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:\Projects\Clients\bkr\ model\bkrcast_tod_new\ inputs\IntraZonals" | Working directory that contains intrazonal files |
| files_list | ["origin_tt.in", "destination_tt.in"] | Intrazonal time files |
| file_termtime | r'E:\Projects\Clients\bkr\ data\frombkr\Network\ network updates\ bkrcast_autoterminaltime.csv' | File path containing terminal time for every zone in BKRCast |
Outputs: The converted data files are produced in the working directory (wd) with "_bkr" appended to file names. The destination_tt_bkr.in file may write quotes around values in the first column. To correct this, open the file in a text editor and replace " all" with all (space should be present before the text).
psrc2bkr_skims.py
Objective: This script crops SoundCast model zones in seed skims to the length of the BKRCast model zones: inputs/[tod].h5. This script is used for processing skims for 12 time periods. (BKRCast is now four time periods; this script is no longer needed.)
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:/Projects/Clients/bkr/ model/soundcast/inputs/" | Working directory that contains skim files |
| tods | ['5to6', '6to7', '7to8', '8to9', '9to10', '10to14', '14to15', '15to16', '16to17', '17to18', '18to20', '20to5' ] | Time-of-day categories |
| num_bkr_zones | 1530 | Number of BKRCast zones |
Outputs: The converted data files are produced in the working directory (wd) with "_bkr" appended to file names.
psrc2bkr_tazdistricts.py
Objective: This script updates district correspondence in the TAZ index file: inputs/TAZIndex.txt.
Settings: The user needs to update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:/Projects/Clients/bkr/ model/soundcast/inputs" | Working directory that contains TAZ index file |
| zoneFileName | "TAZIndex.txt" | Zone index file name |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data file is produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_trips.py
Objective: This script updates zones in supplemental trips: inputs/supplemental/trips/[tod].h5. This script is used for processing skims for 12 time periods. (BKRCast is now four time periods; this script is no longer needed.)
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:/Projects/Clients/bkr/ model/soundcast/inputs/ supplemental/trips/" | Working directory that contains supplemental trip files |
| tods | ['5to6', '6to7', '7to8', '8to9', '9to10', '10to14', '14to15', '15to16', '16to17', '17to18', '18to20', '20to5' ] | Time of day categories |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percentage of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data files are produced in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_skims_tod.py
Objective: This script updates group names in skim files: inputs/[tod].h5.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:\Projects\Clients\bkr\ model\bkrcast_tod_new\inputs" | Working directory that contains skim files |
| tods | ['6to9', '9to1530', '1530to1830', '1830to6'] | Time-of-day categories |
Outputs: The converted data files are produced in the working directory (wd) with the same file names.
psrc2bkr_zonedata.py
Objective: This script updates zones in supplemental model zone data: inputs/supplemental/generation/landuse/tazdata.in.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:\Projects\Clients\bkr\ model\soundcast\inputs\ supplemental\generation\ landuse" | Working directory that contains skim files |
| files_zone | ["tazdata.in"] | Zone data file |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data file is output in the working directory (wd) with "_bkr" appended to file name.
psrc2bkr_zonegroups.py
Objective: This script updates PUMA zone groups and transit zones: inputs/fares/transit_fare_zones.grt, inputs/supplemental/generation/ensembles/puma00.ens.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| transit_dir | r'E:\Projects\Clients\bkr\model\soundcast\inputs\Fares' | Directory containing the transit fare zones (transitFileName) |
| puma_dir | r'E:\Projects\Clients\bkr\model\soundcast\inputs\supplemental\generation\ensembles' | Directory containing the PUMA zone groups file (PUMAFileName) |
| transitFileName | "transit_fare_zones.grt" | File containing transit fare by zones |
| pumaFileName | "puma00.ens" | File containing PUMA zone groups |
| tazSharesFileName | "psrc_to_bkr.txt" | SoundCast to BKR zones correspondence in the same folder as the script; format: psrc_zone_id,bkr_zone_id,percent; percent represents percent of SoundCast zone area within the corresponding BKRCast zone area |
Outputs: The converted data file is output in the working directory (wd) with "_bkr" appended to file names.
daysiminputcheck.py
Objective: This script checks consistency between DaySim parcel file and network skims.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| wd | r"E:/Projects/Clients/ bkr/model/bkrcast_tod/ inputs/" | Working directory that contains input files |
| parcelFileName | "buffered_parcels.dat" | Buffered parcel file name |
| popsynFileName | 'hh_and_persons.h5' | Synthetic population file name |
| zone_district_file | 'TAZ_District_CrossWalk.csv' | Zone-to-districts correspondence |
| tod | '9to1530' | Time of day |
| num_bkr_zones | 1530 | Number of BKRCast zones |
Outputs: Zones with inconsistencies are printed on the screen.
batchRepackH5.py
Objective: This script repacks HDF5 files into smaller sizes.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| ext | "*.H5" | Extension of the files that needs to be repacked |
| h5repack | '"C:\Program Files\ HDF_Group\HDF5\ 1.8.17\bin\ h5repack.exe"' | Repack executable file full path |
Outputs: The repacked data files are produced in the same folder as the input files with "_bkr" appended to file names.
run_bkr_sampling_tests.py
Objective: This script runs sampling test runs.
Settings: Update the following settings before running the script:
| Setting | Value | Description |
|---|---|---|
| model_directory | r'E:\Projects\ Clients\bkr\ model\bkrcast_tod_new' | BKRCast model directory |
| working_directory | r'E:\Projects\ Clients\bkr\ tasks\sensitivity_tests' | Working directory |
| taz_districts | r'E:\Projects\ Clients\bkr\ model\bkrcast_tod_new\ inputs\trucks\districts19_ga.ens' | Zone-to-districts correspondence file full path |
| config_template_path | "daysim_configuration_template.properties" | DaySim configuration template file |
| popsyn_file | 'hh_and_persons.h5' | Synthetic population file |
| shadow_prices_file | 'shadow_prices.txt' | Shadow price file |
| shadow_prices_pnr_file | 'park_and_ride_shadow_prices.txt' | Park-and-ride shadow price file |
| daysim_output | 'daysim_outputs.h5' | DaySim output file |
| zone_district_file | 'TAZ_District_CrossWalk.csv' | Zone-to-district (regions) correspondence to generate taz_sample_rate_file below |
| taz_sample_rate_file | 'taz_sample_rate.txt' | Sample rate by zone; this is an intermediate output, which is an input to popsampler script |
| columns | ['hhtaz', 'cars_per_hh', 'trips_per_hh', 'work_trips_per_hh', 'avg_commute_dist', 'trips_per_tour', 'transit_share', 'at_share', 'avg_trip_dist', 'avg_trip_time', 'am_share', 'pm_share', 'op_share'] | Columns in the summary file |
| periods | {'am': [560, 960], 'pm': [1560, 1860]} | AM and PM period duration |
| random_seeds | [1,2,3,4,5,6,7,8,9,10] | Ten random seeds |
Outputs: The outputs for each run are saved in saved_outputs under the working directory (working_directory).