MOTIS Notes - public-transport/transitous GitHub Wiki

MOTIS Notes

Reading/Watching Material

API Observations

  • Be aware of https://github.com/motis-project/motis/issues/433 when testing against the public demo server.
  • There are detail controls for access/egress modes for journeys but not for the transport modes themselves it seems. Think e.g. of D-Ticket users wanting to exclude long distance trains and operators/providers not covered.
  • Times are all specified in UNIX timestamps, how do we translate that back to the corresponding local timezone everywhere?
    • All times are in UTC offset 0, so need to be converted to/from that accordingly. Timezones for locations are currently not known server-side, although opaquely passing on station timezone information from GTFS could be added eventually.
  • Backward search (ie. searching journey by arrival time) is really backward, ie. start and destination change their meaning. That's fine, but very different from how other APIs deal with this.
  • Only parts of the API are mentioned in the documentation, you can find more by observing the web UI and looking into the protocol .fbs files. This includes API for looking up station by geo coordinate and for querying arrivals/departures at a given station, both things KPT/PTE expect/need.
  • Location lookup by geo coordinate needs https://github.com/motis-project/motis/pull/436 for getting correct names.
  • Are there ways to control minimum layover/connection times? Some connections by default are currently extremely ambitious.
    • Not at this point, but there's other parties interested in this as well.
  • There's only the bare minimum of GTFS data passed through to clients, things not relevant for routing are omitted (e.g. line colors).
    • There's no technical limitation for this, more fields can be added and are not expected to have a relevant resource/performance impact.
  • Intermodal paths are not included in the initial routing answer but need to be queried separately for each such section.
  • There is currently no option to route without realtime data, which can be interesting for still showing base schedule information for orientation given significant realtime changes. Felix says this is easy to implement though, merely by not passing nullptr instead of the realtime table to the routing call.
  • There is no way to see cancelled journeys, and thus no way to see the explanation for the cancellation.
  • The attributes, free_texts and problems elements of the routing response mentioned in the documentation do not seem to be filled with nigiri and/or GTFS.

Operational Considerations

  • Many bundled dependencies, some haven't been updated in a while and potentially interact with hostile data (e.g. Flatbuffers). Is there a security problem?
    • Updating Flatbuffers is tricky due to extensive modifications (32+64bit parallel builds, swapped required/optional semantics, incompatibilities in newer code generators).
    • Prevent that Flatbuffer input from the outside ends up being processed anywhere, see https://github.com/public-transport/transitous/issues/8, if necessary add an additional safeguard in message.cc for that.
  • Should we expose MOTIS directly or put it behind a Apache/nginx/etc reverse proxy for SSL termination and handling the static content/UI? Would remove the HTTP and SSL parts in MOTIS from direct exposure to hostile input, and potentially give us support for more protocol details (HTTP2, compression, etc) should that be lacking in MOTIS' internal HTTP server.
    • Uses an Nginx reverse proxy now.
  • Is there support for incremental OSM data updates? The full import process for this is very expensive.
    • Not at this point, needs discussion with the OSM upstream projects (OSRM, PPR, Valhalla, etc).

Data import

  • Attempting to import the full European pbf fails even with 100G RAM. What are the limits here, are there any workarounds by doing this in smaller increments for example?
    • Felix sent us the demo server configuration which should improve memory usage for the ppr and parking import steps. The demo instance manages to complete its 6 country import within a few hours and <= 64G RAM.
    • Our current failure is the address step though, phase 2 (https://github.com/motis-project/address-typeahead/blob/b6b5e60faac2921f1c0b3813da9c11731a3ca31d/src/extractor.cc#L404), that code looks like it's loading the entire dataset into memory.
    • Felix suggested looking into doing each import step as an independent operation/run, but given we already fail on the very first step that might not help with the current issue yet.
    • Felix sees room for optimization in parking by using a lower resolution (detailed foot paths have to be queried separately anyway).
  • GTFS-RT in combination with nigiri as described in https://github.com/motis-project/motis/wiki/Real-Time-Updates fails on loading the ris module -> That is just outdated documentation, using the built-in GTFS-RT fetcher from nigiri works just fine without needing extra tooling even.
  • The code indicates support for parkendd compatible parking API, although that seems to be nowhere mentioned in the docs.
  • Importing the Swiss national GBFS feed needs https://github.com/motis-project/motis/pull/437.

Configuration

  • osrm-foot vs ppr: aren't those two providing the same functionality? gbfs depends on osrm-foot so we practically need both for now?
    • ppr can do profile-based routing and is thus needed for foot routing, osrm-foot is significantly faster and gbfs does many more foot routing requests where profiles/accessibility is of a lesser concern, so those two are practically needed for now.
    • conceptually both should offer the same API and become interchangeable though
  • what is osrm-bus needed for? is it needed at all given it's not in the demo instance config?
    • leftover from the deprecated core, not used with nigiri
  • there is also a Valhalla module, how does that relate to osrm/ppr? If that's yet another alternative, which one is the one we want/is recommended?
    • Valhalla might become an alternative in the future (it also offers profiles), but according to Felix currently doesn't expose 1:1 Dijkstra routing in its API requiring abuse of the table-based routing API.
  • other external routers are also conceptually possible, setups using BRouter as an external bike router exist for example. This could become interesting if any of those allows for efficient incremental updates.

Routing Issues