October 2006


Well with the UK mashup event coming up I was having a look around to try and find some interesting mashups to explore, but it seems there is a lot of similarity. Yes there are some cool applications with nice interfaces and some interesting datasets, but 99% of them are just data overlay applications. The mashup phenomenon has been going on for getting on for 2 years now and the initial excitement and innovation seems to be wearing off a bit. Most of them consist of one of two recipes.

  1. Find interesting list of something on the web, process list format, geocode and display points using GYM API
  2. Take homegrown list of something optionally geocode then display points using GYM API.

There are some exceptions to this for example combining several different services such as calendaring, events and mapping, however most spatial mashups are still just pushpin maps of locations. These are fine if you are really interested in the data, but many mashups seem to be there just for the sake of doing a mashup. There are two things that would really make mashups that much more interesting and useful and hopefully we will start to see these appearing more often.

The first has already started to happen which is to open up the data to to other users. When you think about mashups you would expect that people using for example googlemaps and some point data would now have made that point data available for all to see, but often this is not really the case. The point data rendered on the map is often no more useable or accessible to other applications than it was before. Yes you can visualise it against a map background, but you cant easily access the raw data; for example to use in a spatial analyst process to find the density surface of those locations. What would be great would be if when creating mashup sites the authors were making the data available as a service, not just an end user application. This can be achieved in several ways, the simplest and most common is to make the same data available in a “standard” format, the most popular of these are; GeoRSS and KML followed by GML and the other OGC formats. This is now pretty common and there are many sites that reformat data into these forms as part of a mashup. The other approach is to create a service or API to access the data you are publishing or accessing as part of the mashup. This is much rarer as its a pretty complex process involving server based software or processes. However there are a few tools that can help you do this. A great example is Dapper, this allows you to create a web service out of any third party website, it will map that site to an XML schema which can in turn be transformed into any common format, such as a googlemap URL or icalendar format. You can also map it to a custom format such as KML or GML. This is an amazingly powerful if slightly contentious tool that allows you to create some interesting applications. I can see people using it to generate KML services from pretty much any website containing locations.

Once services and Data are available the next step is to do more analysis of that data. Currently mashups combine data so the user can for example look for patterns in the geographic data that they will never have noticed before. What would be even better would be for the systems to look for patterns and relationships rather than leaving it up to the users. Spatial analysis is something we are all comfortable with on the desktop, but mashups are still stuck in visualisation mode. This is understandable as many of the tools to do this analysis have not been available in the web environment. However if data is provided as GML, KML or WFS service then users who have the tools could use the data in much more innovative ways. With standards support built into ArcGIS users could utilize online data sources with many of the ArcGIS extensions such as spatial or geostatistical analyst. At 9.2 with ArcGIS server geoprocessing support, organisations can now begin to make analysis services available as well as just raw data. For example there are many sites displaying houses for sale along with information about nearby schools and amenities, what would be great would be for the user to input their preferences about the importance of certain features, such as schools, transport links house prices, distance to shops, pollution and then for the site to calculate the ideal location based on a specific users preferences. This type of problem is a classic spatial analyst problem of overlaying different surfaces to come up with a ideal locations. Another example of this is this story about using Google Earth to search for potential archeological sites, why not combine this with an automated feature extraction model to search for specific data within the imagery, an example of this is documented in this paper that searched for road junction in online raster maps. Hopefully soon with the release of ArcGIS Server 9.2 we will start to see more of this type of analysis capability move to the web and become incorporated into some new and innovative mashups.

Advertisements

mashed.jpg

It seems like the mapping mashup phenomenon is really beginning to take off in the enterprise, the recent story about a legal software provider integrating Google Maps into their software is one example. Now this is nothing particularly new, software vendors have been integrating with 3rd party services for some time, for example the Searchflow conveyancing web site was integrated, maps and all, into a number of 3rd party legal software products. The difference now appears to be the simplicity with which this can be done without having to build a relationship between the different service suppliers or the end users. This ease of use has done a great deal to promote the use of geography and spatial visualisation of data, but it also seems to be causing some confusion in the commercial sector.

I have seen several ITT’s recently for enterprise GIS systems that request the use of “Google satellite imagery” in the solution. These requirements have been driven either by the fact that they already have a web application that displays Google imagery, or someone has seen Google maps and wants the data in their applications. There are 2 main issues that organisations looking to use so called “Google data” in enterprise applications need to look at, both of these seem to be legal rather than technical.

The first issue is the use of the Google maps API in a non public site. It seems like the Google terms and conditions require any service to be publicly available. If its not then you can use the Google enterprise maps edition. This is not free and although they have just announced availability in Europe this doesn’t include the UK yet. I’m not sure whether this means you can’t do commercial mashups at all or whether they will allow the non commercial API to be used until its available in the UK. The reasoning behind having a commercial service is pretty obvious. As our mapping agencies are fond of mentioning collecting and maintaining high quality spatial data is an expensive business, and why would users go to say DigitalGlobe or even the O.S. when it seems they can get essentially the same data for free from Google. The availability of free applications and data has meant that the perceived value of the data is essentially free, whereas the reality is very different. Someone is paying for the data, presumably Google, but increasingly it is going to be “enterprise” users.

The second issue with integration, is the requirement that you use the API, rather than access the data directly, this is obviously driven by the same issues of data licensing, however it does put a pretty big hurdle in the way of organisations who wish to use the data in a non-web environment. Sometimes we may be guilty of assuming that everyone just uses web applications these days, but there are many many users using desktop applications for data maintenance, analysis and publishing purposes and these users are not going to be able to use the Google services.

Google however are not the only service provider that can deliver mapping data, there are free services and API’s from Yahoo, Microsoft and of course ESRI, but again these all have “commercial” versions or restrictions. For example the Yahoo maps API does not really distinguish between commercial or non commercial, but they do have some technical restrictions such as not using it with GPS derived data! (see 1.f(vi) in the terms) they also have an Image API that allows you to directly access the map images without having to use a web browser client which makes integration a bit simpler.

Microsoft have a dedicated commercial API, but don’t limit what you can do with the public API, as long as you use the API.

The ArcWeb public services are designed just for that, i.e. non commercial use. There are also terms available for organisations who wish to use services in commercial applications.

I’m no lawyer so I wouldn’t take any notice of my interpretations of the T’s &C’s. I guess the thing to remember is that while there are many services available to end users delivering high quality data and imagery free of charge, the same is not necessarily true if you wish to make use of these services within a commercial setting. You should take great care in examining the terms of use before using these services for enterprise mashups.

cassette.jpg

OK So I’m officially useless at this blogging lark, its been a while and I’ve been a little busy but still no excuse. There have been a bunch of events that I failed to get to, I managed to miss both the AGI and the oracle spatial user group by being in Redlands catching up with the PLTS team, and getting a final sneak preview of 9.2.

Talking of which, if you haven’t been on the beta program you now have a chance to get a look at 9.2 in all its glory. ESRI(UK) are running a round of Tech update sessions across the country where you will get to see all the cool new stuff. You can register on the website here the dates and locations are:

  • Oct 31st Edinburgh
  • Nov 9th Manchester
  • Nov 14th Birmingham
  • Nov 16th London

The agenda is still to be confirmed, but as its all about 9.2 you can probably expect to see lots about ArcExplorer and ArcGIS server. I would really recommend you try to get to one of these events, as 9.2 really is more than a “dot” release. It introduces some fundamentally new ways of sharing GIS data and functionality around organisations so if you are involved in planning, architecting, managing or building GIS systems its worth seeing what’s new.

A couple of other events to note, If your in Birmingham for the 14th, you could also stay an extra day and catch the second day of the oracle spatial stream at the Oracle UK User group.

Also the UK Geospatial mashup is happening on October the 20th along with an evening of musical mashups the night before where you can pit you DJ skills against our very own DJ CharlesK.