Overpass alternatives - nrenner/achavi GitHub Wiki

Potential improvements

• mmd-osm/Overpass-API: Performance optimized Overpass API fork
  • (changed!{changeset id}) filter
    • Overpass-API#322 Optimize runtime and memory footprint of augmented diff queries
  • would at least fix client-side fallback query timeouts
  • could maybe also replace S3 JSON storage with client-side live queries, as it seems to support all large bbox and time span changesets, e.g.
    • https://dev.overpass-api.de/achavi/?changeset=134140877 - 4 s (10 deg², 215 seconds, 8674 changes)
    • https://dev.overpass-api.de/achavi/?changeset=45656403 - 38 s (35026 deg², 9 hours, 42451 changes)
• for long changesets with multiple uploads (for live adiff query per changeset):
  • cluster edit timestamps into multiple time ranges (uploads) as key-value store -> multiple queries for clusters
  • or, for changesets > 1 hour (potentially auto-close after timeout): query duration minus 1h first and see if changes_count matches

Potential alternatives

Augmented diffs

• azavea/onramp: Generate OSM augmented diffs from OSM change files without Overpass
  • OSMExpress#17 augmented diff example program
• azavea/augdiff-pipeline: Generate Augmented Diffs of OpenStreetMap from minutely replication files.
  • osmesa#52 Augmented Diff Generation (comment)
  • replaced by onramp?

Databases

These are some loose links and ideas I collected over time. All would need to be evaluated if they support, or how much effort it would be to add:

• topological schema
• full history
• metadata
• minutely updates
• resolving object versions of a changeset and previous state with full history

--

• own OSM API DB schema with full history
  • cgimap?
    • changeset download + multi fetch with versions
    • way/:id/full/history - openstreetmap-website#130 + version?
  • potential optimizations
    • gather all data in DB like in Komzpa/fastmap, mmd-osm/History.sql (openstreetmap-website#130)
  • potential importers
    • osm2pgsql flex? (minutely updates!)
      • Include object metadata using imposm? - Help and support - OpenStreetMap Community
    • osmium-export - pg format (PostgreSQL COPY text)
    • osmcode/osm-postgresql-experiments
    • MaZderMind/osm-history-renderer
• Amazon Athena
  • Querying OpenStreetMap with Amazon Athena | AWS Big Data Blog
  • Tables: planet, planet_history (all versions of all elements), changesets
  • https://github.com/mojodna/osm2orc#schema
• hotosm/underpass
  • https://hotosm.github.io/underpass/pages.html
  • https://galaxy.hotosm.org/
• GIScience/oshdb: OpenStreetMap History Analysis
  • oshdb#38 [WIP] Updater
  • oshdb#380 [oshdb-filter] allow filtering by changeset id and (optionally) contributor id
• osmlab/atlas - OSM in memory
  • https://github.com/osmlab/atlas/tree/dev/src/main/java/org/openstreetmap/atlas/geography/atlas/raw/creation#README.md
• protomaps/OSMExpress: Fast database file format for OpenStreetMap
• igor-suhorukov/openstreetmap_h3

  • OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps

  • How to put the whole world into a regular laptop: PostgreSQL and OpenStreetMap / Хабр
  • «Divide and Conquer» for OpenStreetMap world inside PostgreSQL / Хабр
• osmlab/osm-wayback: Embedding OSM object histories from planet-history into GeoJSON objects for historical OSM-QA-Tiles with libosmium and RocksDB
  • jenningsanderson/Historical-QA-Tile: OSM Full History OSM-QA Tiles?
  • Blog von Jennings Anderson | Watching the Map Grow: State of the Map US Presentation | OpenStreetMap
• neo4j/neo4j: Graphs for Everyone
  • neo4j-contrib/osm: OSM Data Model for Neo4j
• ...