Project Meeting 2025.04.17 - ActivitySim/activitysim GitHub Wiki
Agenda
Explicit Error Terms
Technical
Explicit Error Terms
Addressing ambiguity in comparable choices between base and build scenarios
Team extensively discussed two broad alternatives to the current metho which seeds RN by chooser ID and model ID
Make RN seeds less unique
Not consistent with implemented MNL and NL models or observed data
Requires implementation of different model forms
Make RN seeds more unique
Reduces comparability between base and build scenarios even when changes are directly related to changes in systematic utility
Team concluded that we should hold off on further discussion until we thoroughly test the current implementation
What we are testing
MC simulation is not adequate for economic appraisals due to simulation variance
EET simulation is adequate for economic appraisals due to reduction of simulation variance
EET simulation results in more logical choice outcomes at a disaggregate level than MC simulation with respect to changes in systematic utility between base and build alternatives
EET simulation results in more logical choice outcomes at an aggregate level than MC simulation with respect to changes in systematic utility between base and build alternatives
Scenario testing
Sensitivity Test #1: Skim Test
Develop synthetic 20 mile transit corridor; decrease transit IVT and first wait time in AM and MD periods by 50% for zones within 10 miles of buffer
Sensitivity Test #2: Land Use Test
Identify synthetic employment center; double employment within buffer
Will not iterate with feedback, but with accessibility changes
Total of six runs
Base MC, Base EET
2 Build Runs for Skim Test (MC, EET)
2 Build Runs for Land Use Test (MC, EET)
Use SANDAG's SANDAG model for tests
Calibrated working model; familiar by team members
Model Summaries
Disaggregate summaries
Cross-tabulation of base vs build choices for each model summed across decision-makers
Some models are more complex due to number of choices; instead focus more on extent of shifts
Summaries stratified by tour purpose
Aggregate summaries
Trip tables by TAZ, Mode, and Tour Purpose
Scripts will be created and run for both scenarios for each simulation technique and visualized in a Quarto notebook
Level of effort: 107 person hours; 36 hours per scenario
Addressing computational efficiency
56 hours for software improvements that can be done within budget
Propose moving forward with sensitivity testing first and then reserve budget for software improvements
Next update May 20
Roadmap draft was sent out on Tuesday and will be uploaded to Google Drive