Phase 2 Scope of Work - ActivitySim/activitysim GitHub Wiki
RSG is pleased to submit this revised scope of work and cost estimate for the year 2 ActivitySim development effort. It is our understanding that the single zone system MTC travel model one inputs and sub-models will serve as the testbed for our development efforts. We also assume that the goal of this year’s work program is to implement structural components of the MTC travel model one, but not necessarily each and every utility expression in each sub-model since that would likely exceed the project budget. As a result, and upon approval of the agency partners, we may reduce model complexity in order to deliver the scoped features within their task budgets. The key will be to implement the core functionality and integrate the required data structures, while not being overly concerned with the reasonableness of model results. We will verify features for correctness, but predicted mode shares, results by person type, etc. are not the emphasis of year 2. In conjunction with the agency partners, we may decide to revise task budgets in order to more fully implement a feature if the level-of-effort becomes unexpectedly larger than anticipated.*
Table of Contents
Task 1: Project Management
Task 2: Architecture Review and Testing
Task 3: Skim Handling Suitable for Large Set of Skims
Task 4: Accessibility Calculation Procedure
Task 5: Informed Sampling Procedure
Task 6: Fix Random Number Sequences
Task 7: Nested Logit Choice Model Engine
Task 8: Inclusion of Mode Choice Logsums in Utility Functions
Task 9: Tracing Choice Results of Households and Persons
Task 10: Additional Revisions to Phase 1 Implementation
Task 1: Project Management
The purpose of this first task is to manage the overall project, including invoicing and conference calls with the project team, and coordination with the AMPO agency partners. All deliverables, including meeting notes, software, tests, documentation, and issue tracking will be managed through GitHub. Included in this task is two one-day in-person meetings for @bstabler and @toliwaga. The first is for a project kickoff and architecture review/planning meeting, and the second is for a project update and planning meeting later in the year.
Included in task 1 is Open Solutions Group’s participation in two one-day in-person implementation design/review meetings, as well as two half-day implementation design check-ins over the phone. One of the implementation design/review meetings will include a presentation by Open Solutions Group on their experience with scientific computing applications and potential ideas for a distributed cloud-based ActivitySim implementation. Also included under this task is participation by UrbanLabs in two one-day project kickoff and architecture review/planning meetings by phone. Project team meeting are assumed to be in San Diego.
UrbanLabs and Open Solutions Group will also provide QA/QC of select project deliverables, as requested by RSG and the agency partners, under this task.
Deliverable(s): (Due 50 weeks from NTP)
- Participation in Bi-Weekly Meetings
- Pre-Meeting Progress Reports
- Meeting Notes
- Invoicing and Progress Reports
- Client Coordination
- QA/QC
###[Task 2: Architecture Review and Testing](/activitysim/activitysim/milestones/Phase 2: Architecture Review) The purpose of this task is to better understand the existing ActivitySim framework. The RSG team will thoroughly review the code, run it, and develop a few (yet defined) tests in order to better understand system strengths and limitations. The goal of this task is to provide more depth and understanding around questions such as what functionality was partially, incorrectly, or not implemented to date? What are the data I/O relationships/interfaces between components? How is Orca used and what is its strengths and weaknesses? Etc. The team will meet in person (and/or on the phone) in order to more efficiently learn the new framework. The results of this task will be written up on the project site and discussed with the AMPO team.
Deliverable(s): (Due 8 weeks from NTP) (Due April 22, 2016)
- Getting Started Tutorial
- Large Scale Performance Test
- Coveralls Testing via Continuous Integration System
###[Task 3: Skim Handling Suitable for Large Set of Skims](/activitysim/activitysim/milestones/Phase 2: Skim Handling) The goal of this task is to improve the existing skim handling capabilities in ActivitySim in order to support all the MTC travel model one network level-of-service matrices by mode and time-of-day. The improved system may need to store all the skims in the in-memory local data store and provide efficient vectorized query methods. We understand that it may be required to reduce the number of skims used by the model in order to efficiently manage runtimes and memory usage throughout the system. If this turns out to be the case, then the full set of skims would be added back into the model once this functionality has been refactored under a subsequent work order. We will start this task by converting all the MTC travel model one skims to OMX format and then attempting to read them into the local data store. Next, we will test a few of the frequent and complex queries in MTC travel model one, such as network LOS for stop location choice. Depending on the performance, we will then proceed with revisions to the functionality as agreed upon by the project team. The skim handling components will be described in the online documentation as well.
Deliverable(s): (Due 16 weeks from NTP)
- Improved Skim Handling Procedures
- Updated Specifications and Documentation
- Tests and Test Results
###[Task 4: Accessibility Calculation Procedure](/activitysim/activitysim/milestones/Phase 2: Accessibility Calculation Procedure) The goal of this task to translate the existing MTC travel model one accessibility scripts and simplified versions of destination choice logsums into ActivitySim. The procedure will specify the accessibility expressions in Python classes + pandas expression files, similar to other sub-models, and write the accessibilities into the local data store for use in downstream models. Similar to the skim matrix handling procedures, we may need to reduce complexity in order to deliver a working feature within the task budget. The accessibility components will be described in the online documentation as well.
Deliverable(s): (Due 22 weeks from NTP)
- Accessibility Calculation Procedure
- Updated Specifications and Documentation
- Tests and Test Results
###Task 5: Informed Sampling Procedure (DEFUNDED FOR NOW) The goal of this task is to revise the location choice sampling procedures so they intelligently sample destinations instead of simply randomly selecting a sample. The procedure will specify the sampling expressions in Python classes + pandas expression files, similar to other sub-models, and write the results into the local data store for use in downstream models. Transit subzones are not required but desired. Similar to the skim matrix handling procedures, we may need to reduce complexity in order to deliver a working feature within the task budget. The sampling components will be described in the online documentation as well.
Deliverable(s): (Due 26 weeks from NTP)
- Sampling Procedure
- Updated Specifications and Documentation
- Tests and Test Results
###Task 6: Fix Random Number Sequences (DEFUNDED FOR NOW) The goal of this task is to add the ability to fix random number sequences during the model run into order to replicate results. MTC travel model one used a separate random number generator object for each household and then generated random numbers in sequence from the start of the run. The model also had some additional functionality to keep track of numbers drawn before and after each sub-model in order to reset as needed, but the functions were often not called when required. For ActivitySim, we may implement an improved version with one random number generator object and each household by person by sub-model having different random seed offsets. The random number offsets will be pandas attributes and will be stored in the local data store so they will be available for downstream models. The random number sequencing procedures and attributes will be described in the online documentation as well.
Deliverable(s): (Due 30 weeks from NTP)
- Improved Random Number Seeding
- Updated Specifications and Documentation
- Tests and Test Results
###[Task 7: Nested Logit Choice Model Engine](/activitysim/activitysim/milestones/Phase 2: Nested Logit Choice Model Engine) The goal of this task is to build a nested logit choice model engine similar to the multinomial logit model already available in ActivitySim. The existing multinomial logit auto ownership model, including model specification class and CSV input table, will be converted to a nested auto ownership model in order to test and implement this feature. The new nested logit model engine will be described in the online documentation as well.
Deliverable(s): (Due 36 weeks from NTP)
- Nested Logit Model Engine
- Updated Specifications and Documentation
- Tests and Test Results
###Task 8: Inclusion of Mode Choice Logsums in Utility Functions (DEFUNDED FOR NOW) The goal of this task is to develop the functionality to calculate discrete choice model logsums and to then make them accessible to upstream model calculations. The most common application of this is calculating mode choice model logsums (i.e. multimodal accessibility) for each potential destination in a destination choice model. Since ActivitySim is solving each sub-model for all choosers at once, it also needs to solve logsums for all tours/trips/etc. at once, and then store the results in the data store for later use as pandas expressions. We will begin this task by implementing the MTC travel model one tour mode choice model and logsum calculation engine. We will likely simplify the mode choice model by including only a handful of utility expressions in order to focus on the functionality and its correctness, rather than on the actual model design. The simplified implementation will include at least one variable from each type of data object in order to ensure all the required data connections/interfaces are implemented. To test usage of the logsums, we will revise the existing ActivitySim tour destination choice model. The new procedures will be described in the online documentation as well.
Deliverable(s): (Due 44 weeks from NTP)
- Simplified Mode Choice Logsums Calculation Procedure and Interface
- Updated Specifications and Documentation
- Tests and Test Results
###[Task 9: Tracing Choice Results of Households and Persons](/activitysim/activitysim/milestones/Phase 2: Tracing Choice Results of Households and Persons) The goal of this task is to add an intelligent tracing interface to all calculation modules for a tagged person (and their household) in order to debug model results. This feature will be turned off by default in order to reduce runtime during application. The calculations, including utilities for all alternatives, will be traced to well-formed tables written out in an easy-to-parse format. The tracing interface will be added to as many of the model components as feasible given the task budget.
Deliverable(s): (Due 48 weeks from NTP)
- Tracing Procedures
- Updated Specifications and Documentation
- Tests and Test Results
###[Task 10: Additional Revisions to Phase 1 Implementation](/activitysim/activitysim/milestones/Phase 2: Additional Revisions to Phase 1 Implementation) (ADDED) The goal of this task is to successfully run all phase 1 implemented sub-models for the full set of zones, skims, and households in order to eliminate any significant issues in the phase 1 components. An example significant issue to resolve is the proper handling of unavailable alternatives. In order to get the full model run completed, Consultant will revise the code and configuration/expression files as needed. Consultant will use the new trace feature to help identify issues. Consultant will update any documentation and tests as a result of any revisions to the framework. Consultant cost on a time and materials basis cannot exceed the task budget without authorization from AMPO.
Deliverable(s): (Due 30 weeks from NTP)
- Successful Full Model Run with All Zones, Skims, and Households of the Sub-Models Implemented in Phase 1
- Updated Source Code, Configuration Files, Documentation, and Tests