***Update*** 

I should probably point out at this point that the tool currently linked from here does not work with the  current xml format of openstreetmap.  I will post a new script soon that does, although you can probably tell from how long ago the format changed that I have had a few othe real-life things get in the way of blogging recently.

***End Update*****

A while back I started getting interested in the OpenStreetMap Project, initially just filling some detail around the area that I happen to live in.  Access to free detailed data has always been a bit of an issue in the UK and this looks like a great project that seems to be getting a lot of traction.  One thing to note is that the project has been approached very differently from a typical GIS data project. 

Ease of use of data capture has been the main goal and rightly so, as capturing the data in the first place is obviously the biggest challenge. However this does mean that if you approach the data expecting something similar to say navtech or O.S. Data you are in for a bit of a shock.  There is a documented XML structure, but it lets you put pretty much whatever content you want into the data.  This leads to a wonderfully rich set of attribution but also poses some challenges to using the data from a software point of view.  There are some guidelines to attribute feature coding here and there has been lots of heated debates on the discussion list about whether a new data model should be adopted , or whether a topological model is better, or if a spatial index is needed on ways and segments.  If you are used to commercial data the main thing you will notice is that there is very little consistent metadata at the feature level.  That said I’m sure the data has plenty of uses above and beyond cartographic display.

Most of the current tools are aimed at data capture tasks, or at processing the data into nicely rendered map tiles for use in the Google style slippymap.  There is now a pretty large set of data to work with, and it would be great for people to start putting the data to other interesting uses, so to help with that process I’ve just released a copy of a python script I wrote last year (As an exercise in learning python)  for loading the data into a file Geodatabase for use in ArcGIS.  I was planning on re-writing it, but never seem to have the time or enthusiasm, so I’ve released it as is.

The OpenStreetMap data can be downloaded as a Planet dump file containing the current set of data.  The data consists of nodes (both points and vertices) segments (a linear segment between two vertices) and ways (a linear or possible area feature made up of an ordered list of segments and attributes) .  The loader constructs the Ways from the nodes and segments and loads them into polyline or polygon layers. It also loads those nodes that have attributes, as a point set of data.  Attributes in OpenStreetMap are based on key value pairs  such as k=”highway” v=”motorway”, you can think of k as the attribute name and v as the attribute value.  The loader creates attribute fields for all the “standard” key values and loads them into a separate table that can be joined easily to the nodes and ways based on id.  The non-standard key values are loaded into a separate table. with a node_id,name and value columns.  You can use a relate in ArcGIS to join these to the ways or nodes.

The tool runs as a geoprocessing tool from the toolbox, and was written so as not to use any more memory as the data volumes increase, so it should carry on working even when all the tiger data is imported.  the OpenStreetMap data is usually downloaded as a compressed XML file.  This tool does not use an XML parser, as in the past the data has not always had the correct utf8 encoding, also you don’t need to decompress the data before hand which should save a bit of space.  If you are interested in different tags then you can just edit the script to have a different list of standard or ignored tags.

If you’re interested in exploring the data give the loader a go and see what you can do with  OpenStreetMap.  You can download the loader from ArcScripts.

About these ads