There is a new formatting tool called ruff, which is wildly faster than Python Black. Ruff is able to format and check the entire codebase in a fraction of a second. Black simply formats the code, whereas ruff can check the code and run basic checks to identify things that are wrong in the code - things that are hard for a programmer to see but easy for a code formatting tool that has rules that it’s applying. This tool has had explosive adoption in the Python community. Proposal to use ruff, instead of Black, to format the code and it includes validation checks.
Does it give a user an opportunity to review the ruff changes before merging? It’s supposed to only make formatting changes that should make no difference, but there’s another mode that checks more things. Jeff would recommend automatically applying the very safe mode.
Ruff is designed to be a little more flexible than Black, which has pros and cons. Black has no configurations, so there’s no way for a user to mess it up. Ruff has configurations (i.e., we will use double quotes in all of x cases).
Can the Ruff checker help with the config files themselves? That’s where there wrong variable names are more likely to get introduced? Can ruff help with that or is it just python code? Jeff isn’t sure, and thinks it’s just python code – there may be a way to rig it to check the config files but unknown at this point.
Pre-commit ci can run a check like ruff, fix formatting and check for errors. This is a service that’s free for opensource repos that will apply fixes automatically for you.
Timing of implementing these tools is a consideration - we would want it to be a relatively clean, milestone (or pivot) point. Jeff suggested right after this round of optimization fixes, just before adding in all the BayDAG PRs.
One key challenge is if the tool updates its rules - it can make a whole bunch of changes just because of the version of the package, and the pre-commit tool would make all those changes. To prevent this, we could be strict about adhering to a particular version and then update when we want to.
Crashed Trip Mode Choice – crashed because all alternatives had very negative utilities. Sijia found a bug related to number of escortees.
WSP was able to run both legacy and sharrow modes, using latest code as of end of day Tuesday
Sharrow run finished successfully in 16 hours. Legacy is still running, for 24 hours as of the meeting.
Unintentionally, memory release issue with sharrow run seems to be fixed.
Nonmandatory tour scheduling improvements are shown.
School escorting is not showing the same memory reduction as the non-sharrow mode, which is unintuitive, especially since it’s specifically called out to not use sharrow in school escorting. Something is off there.
David is continuing working on the school escorting runtime reduction
Jeff is working on school escorting to work with sharrow
Sijia to figure out what’s not working in the sharrow mode school escorting during the memory profiling
Once all the above is done – need to run it through the implementation tests to make sure nothing is broken in the results
When do we say this is good enough for a release so folks can use the Phase 8 improvements? Joe was not prepared to answer that question. We will revisit later.