Tools - Esri/arcgis-osm-editor GitHub Wiki
This section is divided into three parts: Description of the Geoprocessing Tools, Advanced Tools (for the brave and super geeky) and Description of Editing Tools.
The interaction with the OSM server is accomplished using geoprocessing tools to download, manage, and upload data. To learn more about the role of geoprocessing within the ArcGIS framework please refer to http://desktop.arcgis.com/en/desktop/latest/main/tools/a-quick-tour-of-geoprocessing-tool-references.htm.
Geoprocessing tools are provided for the interaction with the OpenStreetMap data, and referenced in models also provided with the OpenStreetMap Toolbox.
The tools and models are described below.
At 10.2.x and above: This tool adds the ArcGIS Editor for OSM editing extension to your editing environment such that if you have an OSM dataset that you’ve translated into a feature dataset using the ArcGIS Editor for OSM tools (i.e., Download Extract and Symbolize or Load OSM File) you can edit the dataset and use the “Upload OSM Data” tool to upload changes back to OSM. This tool need only be run if you are editing OSM data in ArcGIS and want to upload it back to OSM.
At 10.1.x: If you want to use or share this an OSM feature dataset in another ArcGIS Desktop instance where the ArcGIS Editor for OSM is not installed, you would need to run the Remove OSM Editor Extension tool on the data first to remove this mapping. Conversely, if you wanted to bring data from another environment into ArcMap and upload it to OSM using the ArcGIS Editor for OSM tools, you would need to run this Add OSM Editor Extension tool first so the correct connections to the data model can be made.
This tool can be used with other data in ArcGIS, not just OSM data. This tool provides the ability to group layers, by combining any number of input layers into a single group layer. The first parameter, Input Layers, is the collection of layer files that should be combined into a single group layer. The second parameter, Output Group Layer, is the name of the new group layer. If you would like to persist the layer file on disk, the full path where to store the lyr file should be included.
If you are working with OSM data in an environment that does not have ArcGIS Editor for OSM installed, and in that environment have added or edited standalone attributes not regularly in the Esri OSM data model, this tool can be used to port those standalone attributes back to the data and then upload back to the OSM servers. In effect, this allows you to map other data into an OSM structure, and could be used for importing data to OSM.
This tool allows you to combine user-defined attribute fields into a field containing the OpenStreetMap tags. The first parameter, Input OSM Layer, is the feature class containing the attribute fields to be combined. The second parameter, Select OSM Attributes, are the attribute fields that can be translated into tag key/value pairs for OpenStreetMap attribution.
The converse of this tool is the OSM Attribute Selector.
This tool provides the ability to create a network dataset from a given OSM dataset. This capability requires a configuration file to define the barriers, junctions, turns, and roads. This configuration file can be generated from a script also provided upon installation. For detailed instructions for using this tool in a workflow, see Create a Network Dataset from OSM Data. The first parameter, ‘Input OSM Feature Dataset’, is the path to an OSM-enabled feature dataset. You can obtain an OSM-enabled feature dataset by running the Download, Extract, and Symbolize OSM Data or the Load OSM File tools. The second parameter, ‘Network Configuration File’ is the path to a configuration file that defines the barriers, junctions, turns, roads, and other information for generating the network dataset. Sample files you can use come with the ArcGIS Editor for OpenStreetMap installation. The final parameter, ‘Output Network Dataset’, is the name of the network dataset that will be created. This field is auto-populated after filling out the first parameter, but is editable if desired. The result of running this tool will be that your OSM feature dataset is transformed into a Network dataset. In addition to the new network feature class that is created, it will also include a feature class for barriers, turns, junctions, and roads. After generating the network dataset, you can use the ArcGIS Network Analyst extension tools to work with the dataset.
This tool allows you to download vector data from the OpenStreetMap server and store it locally inside a geodatabase. The first parameter is the URL for the OpenStreetMap server hosting the data. The second parameter is the geographic extent for which you would like to download data. Since this a request against the live OpenStreetMap server, the area of interest needs to be smaller than 0.25 square degrees or have less than 50,000 nodes in the server response (per the OSM API - http://wiki.openstreetmap.org/wiki/API_v0.6]). The next parameter is optional and allows you to download all references that are mentioned in the initial download request. Due to size and time considerations the initial OpenStreetMap download optimizes the data download. This optimization can result in including only references to data as opposed to the actual data itself. For the majority of editing downloads this behavior is desired but in cases where you would like to request the all-inclusive dataset, an option is provided for the tool to resolve all the referenced data. Caution: checking this option could result in a great number of requests and a longer download time. The fourth parameter is the target feature dataset where you would like to store the downloaded data from the OpenStreetMap server. File and enterprise geodatabases are supported as the download containers.
The result of the download are three feature classes containing the point, line, and polygon features translated from the node and ways from the OSM notation. Relations are stored as a standalone table in the geodatabase and are currently not exposed to you.
Download, Extract and Symbolize OSM Data is one of the most often used models in the toolbox. Using the 'Download OSM Data' tool and the 'Symbolize OSM Data' models, it is used to make a connection to OSM, download the OSM XML describing the features, and then parse and symbolize those features into a feature dataset you can use in your map document. To know more about the resulting feature dataset, see Details on the OSM Feature Dataset.
There is a toolset specific for when you want to Download, Extract, and Symbolize OSM data to a feature dataset stored in SDE for PostgreSQL. This different set of models is necessary because of the PostreSQL reserved word for 'natural', which is also a value used in the definition queries for symbolization. If you are downloading into a PostgreSQL SDE repository and do not use the PostgreSQL-specific model, you will see errors when the data is symbolized in the ArcMap interface.
The Download OSM Data (XAPI) tool will download data from an OSM xapi endpoint and converts it into an .osm file. It also supports querying the OSM data that is downloaded such that only certain features are downloaded. The features to download are defined using a predicate; for example amenity=hospital would retrieve only features where the amenity name tag is hospital. After the script finishes running, use the #Load OSM File tool to load the newly created .osm file into the map document as a feature dataset.
NOTES:
- This tool can only be used when the map document's Data Frame uses the Geographic WGS84 Coordinate System.
- Check that the xapi server in the 'Download URL' parameter is responding - see a list of xapi servers at http://wiki.openstreetmap.org/wiki/Xapi#jXAPI. If the server is not responding, you can choose another server from the list. The trailing forward slash for this parameter is important (e.g., http://open.mapquestapi.com/xapi/api/0.6/ rather than http://open.mapquestapi.com/xapi/api/0.6 - note no forward slash).
This tool is used to synchronize your downloaded OSM data with the planet diff files available from the OSM server. Two possible use cases are:
- You've loaded and symbolized an OSM file into your ArcMap interface, and want to get the latest updates from the OSM server without downloading all of the features again.
- You want to synchronize your data with the latest updates programmatically.
- OpenStreetMap planet diff files base download URL: the URL to the planet diff file location. The parameter specifies the base URL for the hour-replicate and minute-replicate directories, but this tool uses both to determine what changes need to be retrieved.
- Input OSM Feature Dataset: the target feature dataset you've been working with that you want to be synchronized with the OpenStreetMap server data.
- Start Time for Diff Files (optional but recommended): If you want to specify a start time for the synchronization, indicate it here. Time must be provided in UTC and expressed according to ISO 8601 (e.g., 2015-03-04T14:00:00, means March 4th, 2015 at 2 p.m.). If no argument is provided the synchronization will be made for the last hour.
- Load updates only inside current AOI: Indicates if only new or updated data within the current area of interest for the current dataset with be recorded. Recommended that this be checked.
- Verbose geoprocessing logging: The parameter specifies the amount of logging information provided by the tool.
This tool allows you to load standalone OpenStreetMap XML files (also known as planet files). The OSM database extracts do not have a size and feature limitation as the direct download tool and the extract can encompass producer-defined geographic areas. For more information about planet files and custom OSM database extracts please refer to the OpenStreetMap Planet file documentation (http://wiki.openstreetmap.org/wiki/Planet). The result is a feature dataset you can use in ArcMap - to know more about the resultant feature dataset, see Details on the OSM Feature Dataset.
Important: The OSM world dataset is so large now that the file geodatabase model is insufficient to store the planet dataset. Smaller geographies - continents, countries - are OK. Therefore, it is recommended that you load the smallest .osm file that meets your needs.
The first parameter is the location of the OSM file on disk. Download a planet file of choice from one of the known download locations. The second parameter is the target feature dataset where you would like to store the data from the stand alone OpenStreetMap file. File and enterprise geodatabases are supported as the download containers.
The Adjust input schema is an optional parameter that allows you to specify additional fields in your OSM feature layers that map to other OSM tags not added by default. If you specify additional fields here, the field is created in your schema but not populated until you run the OSM Attribute Selector tool. This tool also allows you to specify additional fields, but by specifying them when the schema is created with the Load OSM File tool, you save time when you run the OSM Attribute Selector tool.
The result of the loading process are three feature classes containing the point, line, and polygon features translated from the node and ways from the OSM notation. Relations are stored as a standalone table in the geodatabase and are currently not exposed to you.
Due to the flexible nature of the OpenStreetMap data with its key/value pairs as attribute information, it can be a challenge to present the information in the normalized way of a relational database. The downloaded OSM data is optimized for editing as well as keeping the number of feature attributes manageable. The geodatabase schema for points, lines and polygons has the following default attributes outlined in the table below.
| Attribute Name | Attribute Type |
|---|---|
| OBJECTID | ESRI unique ID field |
| SHAPE | ESRI Geometry field |
| highway | OSM Theme type (user defined) |
| barrier | OSM Theme type (user defined) |
| waterway | OSM Theme type (user defined) |
| railway | OSM Theme type (user defined) |
| aeroway | OSM Theme type (user defined) |
| aerialway | OSM Theme type (user defined) |
| power | OSM Theme type (user defined) |
| man_made | OSM Theme type (user defined) |
| building | OSM Theme type (user defined) |
| leisure | OSM Theme type (user defined) |
| amenity | OSM Theme type (user defined) |
| shop | OSM Theme type (user defined) |
| tourism | OSM Theme type (user defined) |
| historic | OSM Theme type (user defined) |
| landuse | OSM Theme type (user defined) |
| military | OSM Theme type (user defined) |
| natural | OSM Theme type (user defined) |
| geological | OSM Theme type (user defined) |
| route | OSM Theme type (user defined) |
| boundary | OSM Theme type (user defined) |
| place | OSM Theme type (user defined) |
| OSMID | Unique OSM ID |
| osmTags | OSM key/value pair storage container |
| osmuser | OSM user (metadata) |
| osmuid | OSM user unique ID (metadata) |
| osmvisible | OSM visibility (metadata) |
| osmversion | OSM version (metadata) |
| osmuser | OSM user (metadata) |
| osmchangeset | OSM changeset (metadata) |
| osmtimestamp | OSM timestamp (metadata) |
| osmMemberOf | OSM is a member of what relations |
| osmSupportingNode | Has attributes (key/value pairs) (points only) |
With this schema some of the core cartographic and analytical functionalities like labeling or computing feature statistics are inaccessible.
An example is if you wanted to create labeling in your map using an attribute such as speed in the Highway Lines feature. Because extracting such information for all features would result in an unwieldy and sparsely populated table, you must run the OSM Attribute Selector tool to get this additional attribute information. This tool allows you to extract keys from the OSM key/value collection field and add them to the selected keys as separate attributes in the feature class.
The first parameter, Input OpenStreetMap Feature Class, should be the feature class from which the OSM tags are to be extracted - e.g., Highways (lines). The second parameter, Select OSM Tag Keys, is the tag keys to extract from the tag collection and store as attributes in the feature class table – e.g., maxspeed. After running the tool, the additional feature information is added to the end of the attribute table for your selected feature class.
The converse of this tool is the Combine OSM Attributes tool.
The Remove OSM Editor Extension tool removes the ArcGIS Editor for OSM custom feature class extension from the selected feature class or feature layer. As of 10.2.x, you don’t need to run this tool unless you used the Add OSM Editor Extension tool in an earlier workflow, and you want to share your OSM dataset with a user who doesn’t have the ArcGIS Editor for OSM installed.
The converse of this tool is the Add OSM Editor Extension tool.
This geoprocessing tool allows you to upload local changes stored inside a geodatabase back to the OpenStreetMap server. Note, as of 10.2.x you will need to run the Add OSM Editor Extension tool before you start editing OSM data to contribute back via this Upload tool.
- The first parameter specifies the URL for uploading the local edit changes. The URL needs to specify a valid OpenStreetMap server instance. The default is to publish to http://www.openstreetmap.org. However, you can also consider publishing to the OSM development endpoint, especially when you are first getting used to publishing to OSM. The development endpoint is http://api06.dev.openstreetmap.org/.
- The second parameter is the local revision table storing the changeset information that is submitted to the OSM server. This table will be created in your geodatabase when you download the data from OSM or run the Load OSM File tool, and autopopulated as you edit the OSM dataset in ArcMap. Do not edit this table manually - all changes will be done automatically by the software. This table also stores the upload status, i.e. if the data upload was successful or if the upload resulted in conflicts. The conflict editor will use the same table to guide you through the conflict resolution. The table name has the schema of name_osm_revision where name is the name of the feature dataset where the OSM data resides.
- The third parameter should be a brief yet descriptive statement about the nature of the local changes. This statement is stored on the server and will be used in the review process of all changes.
- The fourth parameter 'Bundle Data Upload(s)' is an option that allows you to upload all the changes as one change request to the OSM server. Generally, this reduces the overhead on your client and the server because all changes are uploaded at one time. If you leave the 'Bundle Data Upload(s)' option unchecked, the changes will be uploaded one at a time. The advantage of uploading them one at a time is that if one change has issues, you can then resolve the issue with the Conflict Editor, whereas when all are uploaded in a bundle, if there is an issue with one change then the whole upload will fail. Recommended practice is to attempt to upload as a bundle, and if something in the bundle fails, attempt again with this option unchecked.
- The fifth parameter is the user name and the password of your editor credentials. To be able to submit changes to the OSM server you need to have a valid editor account. The credentials are part of the upload, and authenticate the submissions against the server instance. An editor account can be easily created by completing the editor sign-up page at https://www.openstreetmap.org/user/new.
To enable Advanced tools, open ArcMap (in Windows 7, you'll need to open it as an Administrator) and click on the Toolbox icon to show all toolboxes. At the top level toolbox, right-click and select "Add Toolbox...". Browse to your OpenStreetMap toolbox file and add it.
Once added to the tree, right-click on the OpenStreetMap toolbox and select "Add", and then "Tools...". Browse again to the OpenStreetMap toolbox and click the box next to it. Now you see two sub-toolboxes, and within them the OpenStreetMap tools, including some hidden advanced tools. The advanced tools are described below.
This tool allows the user to compare two feature classes to determine the similarity between the two feature representations.
- Input (source) features: The source feature class assumed to contain the data to be compared to the OpenStreetMap data.
- Field to store number of intersections: An attribute field in the source feature class to store the number of intersections between the source and the matching feature. The field needs to be of type integer. A number of 0 indicates that the source does not intersect any other feature, suggesting that the source has no matching entity in the match feature class, making it ok to import. The higher the number of intersections, the more features are geometrically similar and special considerations for imports are required.
- Field to store intersected feature OIDs: An attribute field in the source feature class to store the enumerations of object IDs of intersected features from the match feature class. The fields needs to be of type string. The list of object IDs is comma separated.
- Match feature class: A feature class to which to compare the source. In the context of the ArcGIS Editor for OSM the match feature class is one of created feature classes containing the OSM data.
Export OpenStreetMap feature classes into the OSM XML format.
If you have installed the prototype ArcGIS Editor for OpenStreetMap Feature Service component, you will see an additional toolbox installed in your ArcMap environment, called OSM_on_AGS. The scripts in this toolbox are for running the Feature Service components, and are not for use in ArcMap. This toolbox will not appear if you have not installed the Feature Service component.
The ArcGIS Editor for OpenStreetMap is an extension to the existing editor environment for the desktop. Please be sure to read the introduction to editing data in ArcGIS at http://desktop.arcgis.com/en/desktop/latest/tools/editing-toolbox/an-overview-of-the-editing-toolbox.htm. Here we will discuss the Attribute Window and the Conflict Editor.
As you start editing, the Create Features window lists all of the available feature templates. The choice of templates can be filtered down by geometry type, by layer, and by frequency.
Using one of the available templates creates a feature with the specified “default” OSM attributes. To set additional attributes (key/value pairs), open the attribute window (
).
The attribute window for OpenStreetMap is a little different from the default ArcGIS attribute dialog and it aims to guide you in collecting the appropriate tags for each feature. The attribute window has three columns containing the name of the tag in the left most column, the value of the tag in the middle column and potentially a link to the OSM wiki page for further information about the type and the collection rules for each attribute. If the cell value has a combo box it means that a list of suggested values and tag names does exists. The list of values is merely a list of suggestions and user defined entries can be added at any time. Tip: You can modify the suggested list such that a suggested data model - like the Humanitarian Data Model described earlier - could be used for an ad-hoc/emergency data automation effort. Different configuration files can be set in the Editor Options using the OpenStreetMap tab. For more information on how to represent a different data model please refer to Change the Data Model.
At the time of the data automation process, the extension checks if the feature is a member of a relation. Deleting a point, line, or polygon can have an effect of changing the relation in which the feature participates. A conflict message will appear telling the user about the modifications and which features are affected by the changes.
A number of things can happen between the time you download data from OpenStreetMap, make the edits locally and then submit the changes back to the OSM server. Other OSM users could be working in the same geographic area and submit the changes at different times. As a result, you might encounter a data state which is different from the time when you initially downloaded the data. In that case, the server might reject the submission and indicate a conflict between the server and the client state. To mitigate the conflict the ArcGIS Editor for OpenStreetMap offers a Conflict Editor - shown below - to help you resolve this situation.
In the top left of the Conflict Editor window is a list of datasets and their noted conflict. In this example there is one downloaded dataset called seattle_osm_revision in the current map. This one dataset has one type of conflict with respect to a line feature. When you hover over the node, a tooltip with the conflict description will be shown.
The top right section of the conflict editor shows the attributes of the local feature (edits that were made on the local machine) and the attributes of the feature as the OSM Server currently sees it. In our example we have a race track in downtown Seattle and the local version has the name OpenStreetMap First Edition. However due to the popularity of the race track someone had already changed the name to OpenStreetMap Second Edition and submitted the changes back to the server. This change now causes a conflict because the server assigned the already submitted changes the version number 2 whereas the local attributes still refer to version 1.
In addition to the attribute change then was also a change in the geometry of the feature shown in the bottom area of the Conflict Editor. The geometry of the local feature is shown on left side and the geometry on the OSM server is shown in red on the right side. The colors (green for local and red for server) are fixed but the symbology derives from the default drawing symbology settings in ArcMap.
You now have to choose which information is accurate and should be kept: the local attributes (“OpenStreetMap First Edition”) and the server geometry (red line) or the server attributes (“OpenStreetMap Second Edition”) and the local geometry (green line). When you do a mouse right click on the listed error and choose on the suggested solutions for this error. In the screen capture below the user decided to keep the server attributes (highlighted in green for the attribute table) and the local geometry (green check mark for accepted in the left map and red cross-out for rejection in the right map for the server geometry).
Not all conflicts have a resolution associated with them, and there are always two entries to revise the state of the conflict itself. The first option, Reset the current revision status, clears the conflict status but leaves the error description in the database. It is then up to you to determine a conflict resolution based on the description before the data can be submitted back to the server. The second option, Remove the revision incident removes the conflict altogether so that the local state of the feature becomes disassociated from the server feature they are no longer influencing each other.
Back To: Documentation