OSRM normalized file format - sayr777/osrm-backend GitHub Wiki

  • Author: Stephen Woodbridge
  • Date: 2013-12-08

See Issue #825 for a request to make the normalized file format more stable from 3rd party use.

This page documents the current (as of 2013-11-13 and by now probably not correct; see struct declarations in source code) normalized format of the binary files map.osrm, map.osrm.restrictions, map.osrm.names, which are generated by ./osrm-extract and consumed by .osrm-contract. This information can used to to create files OSRM files with geographical data that is not coming from Openstreetmap (OSM) files. The information on this page comes from the original page, questions asked on the list and some reverse engineering of files created by osrm-extract. This information may not be complete or even correct so be warned.

The file headers and UUID values have changed in both content and size with v 0.3.7 release. So this document is out of date. We need a better way of writing these files if it is going to be a viable way of sharing data from 3rd Party sources, ie: other than using osrm-extract. I do not plan to update this document for header changes. This means that the example hexdumps below do NOT reflect the current file formats. Header sizes and contents will differ, but ignoring these changes, I will try to keep the rest of the content definition up to date.

Writing File Headers

Until we have better support (see issue #825), you can add this to your C++ code:

#include "Util/UUID.h"

// Then as a global in your main routine (this is a singleton)
UUID uuid;

// Then to write you header
fileout_stream.write((char*)&uuid, sizeof(UUID));

You will also need to link to /path/to/Project-OSRM/build/libUUID.a. The UUID changes anytime the the underlying source code changes for the file IO routines. This is a checksum of the source files, so it can change if comments change and things that do not reflect a change to the OSRM Normalized Files, so you need to check your IO routings to make sure they match those of .../Project-OSRM/Extractor/*

Attention: all the numeric values (int and short) are little endian. Note that some bytes arose from data structure alignment.

The names file: map.osrm.names

This file contains all the street names appearing in the dataset. There is no specific order required and it is fine when a name appears only once for multiple ways with the same name.

Note: offset below means from the start of the first name in the file.

  • [(uint32) count of non null names + 2]
  • [(uint32) 0] - null of zero length name
  • [(uint32) offset to name 1] - this will be zero also
  • [(uint32) offset to name 2] - this will be [offset to name 1] + length of name 1
  • [(uint32) offset to name n] - this will be [offset to name n-1] + length of name (n-1)
  • [(uint32) offset to name cnt] - this will be [offset to name cnt-1] + length of name (cnt-1)
  • [(uint32) offset to name cnt+1] - this will be [offset to name cnt] + length of name cnt
  • [(uint32) offset to name cnt+1] - this will be [offset to name cnt] + length of name cnt (AGAIN)
  • [(char[]) buffer of names]

Here is a hex dump a small sample file with brackets areounf the fields

$ cat map.osrm.names.hd
00000000  [0d 00 00 00] [00 00 00 00]  [00 00 00 00] [0f 00 00 00]  |................|
00000010  [20 00 00 00] [2e 00 00 00]  [3f 00 00 00] [52 00 00 00]  | .......?...R...|
00000020  [5f 00 00 00] [6f 00 00 00]  [7d 00 00 00] [8d 00 00 00]  |_...o...}.......|
00000030  [9a 00 00 00] [aa 00 00 00]  [aa 00 00 00] [50 72 69 6e  |............Prin|
00000040  63 65 73 73 20 41 76 65  6e 75 65 47 72 61 6e 69  |cess AvenueGrani|
00000050  74 65 76 69 6c 6c 65 20  52 6f 61 64 52 61 64 63  |teville RoadRadc|
00000060  6c 69 66 66 65 20 52 6f  61 64 4c 61 66 61 79 65  |liffe RoadLafaye|
00000070  74 74 65 20 54 65 72 72  61 63 65 43 72 6f 6f 6b  |tte TerraceCrook|
00000080  65 64 20 53 70 72 69 6e  67 20 52 6f 61 64 4a 6f  |ed Spring RoadJo|
00000090  72 64 61 6e 20 53 74 72  65 65 74 43 61 73 74 6c  |rdan StreetCastl|
000000a0  65 77 6f 6f 64 20 44 72  69 76 65 42 65 72 6b 65  |ewood DriveBerke|
000000b0  6c 65 79 20 44 72 69 76  65 43 68 61 72 6c 65 6d  |ley DriveCharlem|
000000c0  6f 6e 74 20 43 6f 75 72  74 42 65 6c 20 41 69 72  |ont CourtBel Air|
000000d0  20 44 72 69 76 65 4d 65  61 64 6f 77 62 72 6f 6f  | DriveMeadowbroo|
000000e0  6b 20 52 6f 61 64]                                 |k Road|
000000e6

The nodes and edges file: map.osrm

This version of the file starts with a header. I'm not sure how the header is created (look at the source) by I was able to copy and paste the header to make my file work.

See the hexdump below. The header is bytes 00000000 - 000000bc and in this range bytes 000000aa - 000000bc appear to be padding and can be set to zero as far as I know. Following the header:

  • [(uint32) count of vertices]

Next for each usable node ORDERED BY ID:

  • [(int32) round(latitude*1,000,000.0)]
  • [(int32) round(longitude*1,000,000.0)]
  • [(uint32) node_id]
  • [(uint32) flags]

Where flags are a byte array with values like:

  • 00 00 00 00 - default
  • 01 xx xx xx - has bollard
  • xx 01 xx xx - has traffic light

The next part contains the edges:

  • [(uint32) count of edges]

Next for each edge ORDERED BY TARGET ID:

  • [(uint32) source_node]
  • [(uint32) target_node]
  • [(int32) edge length] - unit: m; MUST be > 0
  • [(int16) direction] - 00 00 is bidirectional; 01 00 is one way
  • [(int32) edge weight] - unit: 1/10 sec; MUST be > 0
  • [(int16) edge_type] - the edge rank number in the applied speed profile has position 0) Currently, it is only used in the instruction filtering mechanism: roughly speaking, an instruction is generated only when the rank numbers differ.
  • [(uint32) street_index] - streetname position in map.osrm.names (first street name has position 0)
  • [(uint32) flags]

Where flags are a byte array with values like:

  • 00 00 00 00 - default
  • 01 xx xx xx - is roundabout
  • xx 01 xx xx - is ignoreInGrid (This field can be used in order to ignore the edge during data preparation; probably it is driven by the excludeFromGrid-entry in the speed profile (typically used for ferry-type and internal for pier-type); probably the effect is that a route can never start or stop in the middle of a set of excludeFromGrid-segment)
  • xx xx 01 xx - is accessRestricted
  • xx xx xx 01 - is contraFlow

Here is an example hexdump of the map.osrm file:

$ cat map.osrm.hd
00000000  4f 53 52 4d 61 32 62 39  34 34 30 39 31 33 31 33  |OSRMa2b944091313|
00000010  38 66 36 66 37 39 34 63  36 31 37 35 31 30 62 38  |8f6f794c617510b8|
00000020  33 35 39 63 00 65 33 35  35 37 35 38 32 33 31 33  |359c.e3557582313|
00000030  64 32 30 65 36 39 30 36  63 30 39 63 62 66 36 39  |d20e6906c09cbf69|
00000040  32 38 33 65 36 00 33 34  36 66 34 39 39 63 36 61  |283e6.346f499c6a|
00000050  61 30 34 65 36 31 34 30  38 36 34 35 30 61 62 64  |a04e614086450abd|
00000060  30 61 38 61 33 61 00 35  33 66 37 39 30 36 30 62  |0a8a3a.53f79060b|
00000070  30 30 65 66 63 32 35 33  34 37 35 66 36 36 33 66  |00efc253475f663f|
00000080  31 61 39 31 39 65 64 00  31 61 30 38 34 36 63 39  |1a919ed.1a0846c9|
00000090  30 64 64 31 39 66 64 34  32 31 31 62 33 64 65 31  |0dd19fd4211b3de1|
000000a0  30 64 64 38 31 30 35 34  00 d3 b7 a0 6b dc 39 5b  |0dd81054....k.9[|
000000b0  ff a3 97 94 4d 8f f6 3d  1f 00 16 08 [50 01 00 00]  |....M..=....P...|  ## count of vertices
000000c0  [d0 49 8a 02] [54 ac be fb]  [01 0d e2 03] [00 00 00 00]  |.I..T...........|  ## first vertex
000000d0  f1 51 8a 02 98 8d be fb  21 0d e2 03 00 00 00 00  |.Q......!.......|
000000e0  d4 2d 8a 02 61 b0 be fb  2e 12 e2 03 00 00 00 00  |.-..a...........|
000000f0  ae 5d 8a 02 f2 a4 be fb  ad 12 e2 03 00 00 00 00  |.]..............|
...
00001590  cd 4c 8a 02 00 c2 be fb  0e af 54 5a 00 00 00 00  |.L........TZ....|
000015a0  22 4d 8a 02 07 c2 be fb  0f af 54 5a 00 00 00 00  |"M........TZ....|
000015b0  91 4e 8a 02 76 c2 be fb  20 af 54 5a 00 00 00 00  |.N..v... .TZ....|  ## cnt vertex
-- This starts the edges ...
000015c0  [51 01 00 00]  -- count of edges
-- first edge
000015c4  [1c 48 e3 03]  [01 0d e2 03] [29 00 00 00]  |Q....H......)...|
000015d0  [00 00] [3c 00 00 00] [01 00]  [07 00 00 00] [00 00 00 00]  |..<.............|
-- second edge
000015e0  [c0 16 e3 03] [21 0d e2 03]  [24 00 00 00] [00 00] [34 00  |....!...$.....4.|
000015f0  00 00] [01 00] [06 00 00 00]  [00 00 00 00]
-- third edge, ...
000015fc  da 74 e3 03  |.............t..|
00001600  2e 12 e2 03 38 00 00 00  00 00 51 00 00 00 01 00  |....8.....Q.....|
00001610  08 00 00 00 00 00 00 00  41 75 e2 03 ad 12 e2 03  |........Au......|
00001620  03 00 00 00 00 00 05 00  00 00 01 00 0b 00 00 00  |................|
00001630  00 00 00 00 93 a3 e2 03  ce 12 e2 03 01 00 00 00  |................|
...
00003a50  0e af 54 5a 0c 00 00 00  00 00 1e 00 00 00 01 00  |..TZ............|
00003a60  00 00 00 00 00 00 00 00  0e af 54 5a 0f af 54 5a  |..........TZ..TZ|
00003a70  09 00 00 00 00 00 17 00  00 00 01 00 00 00 00 00  |................|
00003a80  00 00 00 00 0f af 54 5a  20 af 54 5a 29 00 00 00  |......TZ .TZ)...|
00003a90  00 00 64 00 00 00 01 00  00 00 00 00 00 00 00 00  |..d.............|
00003aa0

The names file: map.osrm.restrictions

This version of the file starts with a header. I'm not sure how the header is created (look at the source) by I was able to copy and paste the header to make my file work.

See the hexdump below. The header is bytes 00000000 - 000000bc and in this range bytes 000000aa - 000000bc appear to be padding and can be set to zero as far as I know. Following the header:

  • [(uint32) count of restrictions]

Next for each restriction ORDERED BY THE TO-WAY(!) REF):

  • [(uint32) via-node]
  • [(uint32) from-node]
  • [(uint32) to-node]
  • [(uint32) flags]

Where flags can take the following values:

  • 00 7f 00 00 - this turn is FORBIDDEN
  • 01 7f 00 00 - ONLY this turn is allowed

Here is a simple graphic to better understand turn restrictions. If we have the following graph:

a----b-----c
     |
     |
     d

ab, bc, and db are all two way and there is no right turn from db to bc

Then the restriction for the FORBIDDEN turn is:

b, d, c, 00 7F 00 00

where:

  • b is the via-node
  • d is the from-node
  • c is the to-node

and the records should then be sorted by edge_id of EDGE[via-node, to-node].

Here is a hexdump of a map.restrictions file with NO restrictions in it:

$ cat map.osrm.restrictions.hd
00000000  4f 53 52 4d 61 32 62 39  34 34 30 39 31 33 31 33  |OSRMa2b944091313|
00000010  38 66 36 66 37 39 34 63  36 31 37 35 31 30 62 38  |8f6f794c617510b8|
00000020  33 35 39 63 00 65 33 35  35 37 35 38 32 33 31 33  |359c.e3557582313|
00000030  64 32 30 65 36 39 30 36  63 30 39 63 62 66 36 39  |d20e6906c09cbf69|
00000040  32 38 33 65 36 00 33 34  36 66 34 39 39 63 36 61  |283e6.346f499c6a|
00000050  61 30 34 65 36 31 34 30  38 36 34 35 30 61 62 64  |a04e614086450abd|
00000060  30 61 38 61 33 61 00 35  33 66 37 39 30 36 30 62  |0a8a3a.53f79060b|
00000070  30 30 65 66 63 32 35 33  34 37 35 66 36 36 33 66  |00efc253475f663f|
00000080  31 61 39 31 39 65 64 00  31 61 30 38 34 36 63 39  |1a919ed.1a0846c9|
00000090  30 64 64 31 39 66 64 34  32 31 31 62 33 64 65 31  |0dd19fd4211b3de1|
000000a0  30 64 64 38 31 30 35 34  00 d3 b7 a0 6b dc 39 5b  |0dd81054....k.9[|
000000b0  ff a3 97 94 4d 8f f6 3d  1f 00 16 08 00 00 00 00  |....M..=........|
000000c0

Next for each usable node ORDERED BY ID: