More than just PushPins

I’m currently going through the preliminary stages of looking to move house, this means that I seem to be coming across an awful lot of pushpin maps and Google mashups on a variety of property web sites. Most of these sites are very good at aggregating data into a simple map view displaying a number of data sources.  Typically you choose an area to search, a bunch of parameters, and all properties are displayed on a map. Some sites will go a little further and add in demographic data, showing what the area you are looking at is like.

London house prices aside, none of these sites seem to do what i need by letting you search for areas in general.  What I’d like is to ask;

 find me all the areas within an hours drive  (at rush hour) from my office and 30 min’s public transport from my wife’s work, with a demographic of people I’d like to live near (I’m not so bothered if they want to live near me!). 

 This is actually a pretty difficult problem to answer, and certainly not one that can be quickly solved by a combination of Pushpin maps and a tiled streetmap.  To answer this question you need both data (multimodal transport network demographics, historic traffic density) and some geoprocessing capability to generate service areas and combine these with demographics. 

 ArcGIS Server would be the perfect answer to this when combined with the right data.  Geoprocessing functionality based either on road networks or travel time grids can be used to generate areas that meet the criteria.

You can’t pre-generate data as you can in the tiled map model, as everyone’s query is different, perhaps I’d rather calculate my travel area based on CO2 emissions or to different locations.

Solutions to parts of this problem are available on the web but not combined together.  Transport direct does a great job of giving driving and public transport travel times based on single journeys rather than a service area (which is harder to calculate).  Chris Lightfoot did some fantastic work turning this data into travel time maps. 

There don’t seem to be many sites that actually do on-demand processing of data to generate custom service areas.  One example that does is the Business Analyst online service.  Which among other things, will create drive times from a business location, and also generate a demographic or business report.  Its a good example of what can be done with geoprocessing capability in the background.  The ArcGIS server development blog also has a nice demo of building geoprocessing into a web application.

None of the above sites is any closer to finding me the perfect location to live in though, I definately need more than just pushpins.

 

On Location mashup*

I happened to be working in Canary Wharf on Tuesday, so on the way home I made a last minute decision to detour to the mashup* event which was themed ‘On Location’.  I’d made a mental note of it a while ago, but then completely forgotten, anyway I’m glad I  went as it was pretty interesting.  There was a good talk from TeleAtlas, about what they are up to, particularly in the Internet mapping and wireless mapping space.  I was most interested in the information they presented about the commercial penetration of the different map sites around the world.  Google was the top one, but the interesting bit was that it seemed that in most places Google was first and a local provider (such as multimap in the UK) was second.  I’m not sure why this would be, maybe a language thing, or historical or perhaps its a map presentation cartography issue, I know my wife always uses multimap when looking for addresses, as she says they look like proper maps!  They also mentioned that Nokia is planning on launching a new global (tile based) mapping platform and API for use with their devices.

The talk was followed by a panel discussion, which focused mostly on the privacy and security issues surrounding things like google streetview and also locational information, kept for example by your phone service provider.  Do you care that your phone company, or your employer or your spouse can locate you anytime? or do the benefits of you and trusted entities knowing your location outweigh the issues?

One thing the talks and the interesting applications that many of the presenters and demonstrators were discussing, made me think about is to what extent does the ubiquity of high quality reference data available in web mapping sites and API’s lead to a downward spiral in the perceived value of data.  I guess this is a question that TeleAtlas and other data providers probably worry about too. 

Most of the demonstrators were showing property search based web sites, which seems to be where the money\talk is at the moment, but being a geek at heart I was more interested in Widr and their location API, Its a similar concept to Navizon, location by wireless networks.  Its pretty cool, and the model of revenue sharing with people who contribute to collect the location data is nice.  I guess the key is how  much coverage they get, this will partly depend on how much data is collected by contributors, and the problem I found originally with Navizon when it was just a mobile application, is that to collect data you need a GPS, but if you have a GPS you (mostly) don’t need Navizon, Having a web site API opens it up a bit as while you may have a GPS but your desk or laptop bound website users may not, but how important location is to laptop users vs mobile users I’m not so sure, its still  a cool idea though.

 

OpenStreetMap Data in ArcGIS

***Update*** 

I should probably point out at this point that the tool currently linked from here does not work with the  current xml format of openstreetmap.  I will post a new script soon that does, although you can probably tell from how long ago the format changed that I have had a few othe real-life things get in the way of blogging recently.

***End Update*****

A while back I started getting interested in the OpenStreetMap Project, initially just filling some detail around the area that I happen to live in.  Access to free detailed data has always been a bit of an issue in the UK and this looks like a great project that seems to be getting a lot of traction.  One thing to note is that the project has been approached very differently from a typical GIS data project. 

Ease of use of data capture has been the main goal and rightly so, as capturing the data in the first place is obviously the biggest challenge. However this does mean that if you approach the data expecting something similar to say navtech or O.S. Data you are in for a bit of a shock.  There is a documented XML structure, but it lets you put pretty much whatever content you want into the data.  This leads to a wonderfully rich set of attribution but also poses some challenges to using the data from a software point of view.  There are some guidelines to attribute feature coding here and there has been lots of heated debates on the discussion list about whether a new data model should be adopted , or whether a topological model is better, or if a spatial index is needed on ways and segments.  If you are used to commercial data the main thing you will notice is that there is very little consistent metadata at the feature level.  That said I’m sure the data has plenty of uses above and beyond cartographic display.

Most of the current tools are aimed at data capture tasks, or at processing the data into nicely rendered map tiles for use in the Google style slippymap.  There is now a pretty large set of data to work with, and it would be great for people to start putting the data to other interesting uses, so to help with that process I’ve just released a copy of a python script I wrote last year (As an exercise in learning python)  for loading the data into a file Geodatabase for use in ArcGIS.  I was planning on re-writing it, but never seem to have the time or enthusiasm, so I’ve released it as is.

The OpenStreetMap data can be downloaded as a Planet dump file containing the current set of data.  The data consists of nodes (both points and vertices) segments (a linear segment between two vertices) and ways (a linear or possible area feature made up of an ordered list of segments and attributes) .  The loader constructs the Ways from the nodes and segments and loads them into polyline or polygon layers. It also loads those nodes that have attributes, as a point set of data.  Attributes in OpenStreetMap are based on key value pairs  such as k=”highway” v=”motorway”, you can think of k as the attribute name and v as the attribute value.  The loader creates attribute fields for all the “standard” key values and loads them into a separate table that can be joined easily to the nodes and ways based on id.  The non-standard key values are loaded into a separate table. with a node_id,name and value columns.  You can use a relate in ArcGIS to join these to the ways or nodes.

The tool runs as a geoprocessing tool from the toolbox, and was written so as not to use any more memory as the data volumes increase, so it should carry on working even when all the tiger data is imported.  the OpenStreetMap data is usually downloaded as a compressed XML file.  This tool does not use an XML parser, as in the past the data has not always had the correct utf8 encoding, also you don’t need to decompress the data before hand which should save a bit of space.  If you are interested in different tags then you can just edit the script to have a different list of standard or ignored tags.

If you’re interested in exploring the data give the loader a go and see what you can do with  OpenStreetMap.  You can download the loader from ArcScripts.

 

Build your own touchtable

I was lucky enough recently to see a great demo of the giant TouchTable, and I had forgotten since seeing it 3 years ago at the user conference, what a very very cool piece of kit it is.  With microsoft recently announcing their forthcoming Surface platform, I got to wondering what would be involved if i wanted to build one myself.  The thing that makes these devices so compelling to use is the multitouch and collaborative aspect of the interfaces.  Pretty much all the devices that are available to consumers such as tablet PC’s, or businesses such as the smart boards and kiosk PC’s are single touch only.  The devices just emulate normal mouse interaction with a single cursor.  The touchtable, surface pc and perseptive pixels devices all allow you to interact with multiple touches at the same time, either both hands or several people at once.  This bit is the most challenging, both from a hardware sensing, and a software processing perspective.  I’m not too sure how the touchtable and perseptive pixels stuff works, but the Surface PC works using IR sensing, as decribed here. 

Anyway, after a bit of searching I found this site dedicated to building your own touch table using the same principles, there is a diy guide to putting together the hardware, as well as links to an opensource effort to build the software to interact with the hardware.

All I need to do now is think of a good reason for making one!

 

GIS Tech 2007 Agenda Online.

The Agenda for GIS Tech is up on the website The green tracks are the more technically oriented sessions, mostly being given by people from the consultancy group.  There’s some good AGS talks, particularly on the second day, I’ve had a preview of some of the demo’s that Matt and Dan are doing in their Ajax\Web 2.0 talk and that should be worth seeing. On the first day I’ll probably stick with the green sessions as well as they all look pretty interesting. 

 

GIS Tech 2007 registration now open

Registration is now open for GIS Tech 2007.  This year we are going to be in Nottingham, although the format is going to be pretty similar.  Starting with a plenary session followed by various streams with 1 hour technical sessions covering pretty much all the technology so  you will be able to get to see all the 9.2 stuff.  The details of the event and how to register can be found here. 

Its a great way to get more detail about products and technology that you may be using or planning to use as there is a wide range of presentations, I think I’ve been roped in to doing one.  Its also a good opportunity to get to meet many of the technical staff at ESRI(UK). 

 

ArcGIS OSTN02 Transformation Download

You can now download the free OSTN02 transformation tools for ArcPad 7 and ArcGIS 9.1.  To download it you need to log in to http://www.myesriuk.com/esriuk/members/downloads.asp.  Its listed as OSTN02. 

obviously you need to be registered with myesriuk.com to do this.  The download adds a new transformation to ArcMap that you can use when using OSGB36 based data It  also has an installation to support OSTN02 in ArcPad 7. 

 

Five Things to Geocode

Well it seems like Ed has tagged me, in this scheme which I’m wondering whether is a very subtle Google ads pyramid scheme started off by someone at the top of the chain, but anyway I’ll oblige and tell you five things about me that you don’t (need to ) know, as this is a GI blog I’ve kept that theme and given you a little geocoding puzzle to solve.

1)   I was Born somewhat prematurely in 1973 and so far I seem to be the only Dominic Stubbins in the world (well on Google anyway), I was born  here:

 osgb1000005733056

2)  I don’t have fully opposable thumbs, but despite that I grew up, and developed my geekiness tendencies  here:

 -50784,5804480

3)   I first came across digital mapping systems when doing work experience at a sweatshop doing digitisation of the OS Landline maps for British Gas. I didn’t have an auspicious start as my main job was to refill the industrial size pen plotters used for test plots, but I didn’t realise it was important that the correct coloured pens were in the right caddy.  That was here:

tqsrsrsrrrtsqrstrqqt

4)  I first used GIS in anger at University, mainly Unix based ArcInfo back before there was a distinction between workstation and desktop  here:

13/4059/2591

5) I’m getting married in a few weeks and after that you might be able to locate me somewhere in

GB 100105

Lat/longs on a postcard please, most of them should be fairly obvious especially if you cheat and use a search engine, although I guess it shows the importance of metadata.  The first one is interesting as TOIDS are supposed to be the new way of information sharing, but I wonder how much it costs to geocode the single toid I’ve given to a lat,long without purchasing a full mastermap coverage.

I believe the correct etiquette is to pass these tags along, so in no particular order.

Matt, Art and Andrea

 

ArcGIS Server Cache and Optimal Hard Disk Block Size

We tend to take storage for granted these days, as you can pick up a 1/2 TB disk for about £100. which considering the first computer I owned was a ZX81 with no storage other than a cassete drive and an 8K ROM is pretty amazing. However there are still plenty of ways to use storage up, one way that works quite well is to format your drive with the wrong block size and then store an ArcGIS server Map Cache on it.

My laptop is currently running Windows vista, but for ArcGIS server stuff I tend to use a windows 2003 Virtual server. I was doing some experimenting with performance and using different image cache tile sizes and swapping between fused and Layer caches. One interesting thing that this highlighted was the fact that the disk block size can have a huge effect on the amount of storage taken up by the map cache.

The Virtual server I was using had a default virtual hard disk that has a block size of 4K this is fairly standard, although you can vary it when formatting the disk, for example my media server at home has 64K blocks as this gives better performance when accessing large files.

The Default size in ArcGIS server when creating a map cache is 512×512 pixels, for the vector data and rendering I was using these came out at about 5-10Kb in size. To compare performance I created the same cache with 128×128px tiles. I was expecting that in general the total size of the cache would be slightly bigger, because there would be more images (16 times as many), but they would all be 16 times smaller in pixel size and maybe 10 -15 times smaller in file size. There is going to be some overhead in image headers and maybe the png compression would be slightly less efficient, however for the same data at the same cache levels, the information content and total file size should be much the same, regardless of tile size.

In reality what I found was that the 128 pixel tiles took up almost 10 times the total storage space than the 512 pixel tiles. The reason for this was the disk block size was too large. The average size of each 128px tile was about 400 bytes. however with the block size at 4096 bytes, each file occupies a whole block on disk which means most of the space in each block is wasted. The data I was using was an extreme case, The data was very sparse so a lot of the small tiles were completely empty but were still occupying 4K on disk. This is compounded if you build a separate cache for each layer with small tile sizes, as each tile is much less likely to have any data in it but will still occupy a whole block. If on the otherhand your data contains a lot of detailed imagery, smaller tile dimensions will not be as inefficient as most tiles will contain information regardless of how small they are. They will still take up plenty of space but at least you will be using it efficiently. The key is getting the block size large enough to contain the small tiles without wasting too much space.

So the moral of this story, is that in order to make best use of the storage that you have you should format the disk with a suitable block size for the data that you will store on it. If you expect to have a lot of small and possibly blank or near blank cache tiles you should use a small block size such as 512 bytes. If your data is dense, and your tiles larger then you can get away with a larger block size. Normally for data you don’t really have to worry about the inefficiencies of disk block size as the numbers are so small, but if you are building industrial size map caches, the number of tile images can be in the 100’s of millions. Use the right block sizes and the cache can provide huge performance and usability gain, use the wrong block size and the gains will be at the cost of a lot of wasted storage.

 

ArcGIS 9.2 Sizing and Capacity Planning.

One of the tasks I often get involved in is system sizing and capacity planning for GIS systems. The key reference for this is the excellent System Sizing Document produced twice a year by Dave and his Group at ESRI. The Paper is pretty long and can be kind of daunting when you first see it, however understanding it all is vital if you are going to be undertaking any detailed sizing tasks.

I had been planning on putting together a Post about which were the key bits of the document to look at if you wanted to do a rough sizing effort. So i just had a look back at the document and noticed that it has been updated with an October 2006 edition. There have been some pretty fundamental changes, to bring it in line with some of the new tools that are being developed to help the process and make it more up to date.

The Key chapters 7 and 8 have been completely re-written and updated. With the Release of 9.2 the new document has been updated with details of the sizing and performance as well as licensing mechanisms and recommended configurations. A new hardware baseline has been included and it also takes account of multicore hardware. The ArcGIS Server sections are much more comprehensive than in the last document, and i suspect there is still a bit more to come to help with some of the more complex aspects of 9.2 sizing issues. All I have to do now is find the time to digest the whole document, maybe on the flight back from Barcelona tomorrow.

 

Microsoft Tech-Ed Europe

This is obviously the week for conferences, what with the ESRI European User Conference happening in Athens, its also the first week of the Microsoft Tech-ed conference.  This year is a little different, with the conference smaller but spread over 2 weeks and split into 2 streams.  This week is the developers conference and next week is the IT Forum.  I think I preferred it as it used to be with developers and system admin mixed together as I tend to be involved in both areas, however 2 weeks is much too long to justify being at a conference, so I chose the developer half this year. 

 Barcelona is a great city and so far the weather has been pretty good.  Microsoft certainly know how to look after delegates, with an endless supply of free doughnuts and sugared beverages!

The key things that are being talked about here are .Net 3.0  and related technologies such as LINQ, WPF and WCF, also there is the launch of office 2007 and Windows Vista. LINQ in particular looks like a really cool and very useful framework. 

Unlike previous Tech Ed events there are no Mappoint or spatial related sessions, there is a virtual earth stand where they are showing off the new Virtual Earth 3D ActiveX component, It does have a certain wow factor seeing it embedded in a browser, even if it is IE only.  There was also an interesting talk about Windows live mashups, though it focused mainly on things other than maps such as contacts.  Nothing very new although a hint that some kind of online storage service may be coming as well as a possible link up between Virtual Earth 3D and the new MS Flight Simulator.

Its certainly a great place to be to find out whats coming that will impact windows projects over the next couple of years.

 

Spatial Mashups Re-Processed

Well with the UK mashup event coming up I was having a look around to try and find some interesting mashups to explore, but it seems there is a lot of similarity. Yes there are some cool applications with nice interfaces and some interesting datasets, but 99% of them are just data overlay applications. The mashup phenomenon has been going on for getting on for 2 years now and the initial excitement and innovation seems to be wearing off a bit. Most of them consist of one of two recipes.

1. Find interesting list of something on the web, process list format, geocode and display points using GYM API
2. Take homegrown list of something optionally geocode then display points using GYM API.

There are some exceptions to this for example combining several different services such as calendaring, events and mapping, however most spatial mashups are still just pushpin maps of locations. These are fine if you are really interested in the data, but many mashups seem to be there just for the sake of doing a mashup. There are two things that would really make mashups that much more interesting and useful and hopefully we will start to see these appearing more often.

The first has already started to happen which is to open up the data to to other users. When you think about mashups you would expect that people using for example googlemaps and some point data would now have made that point data available for all to see, but often this is not really the case. The point data rendered on the map is often no more useable or accessible to other applications than it was before. Yes you can visualise it against a map background, but you cant easily access the raw data; for example to use in a spatial analyst process to find the density surface of those locations. What would be great would be if when creating mashup sites the authors were making the data available as a service, not just an end user application. This can be achieved in several ways, the simplest and most common is to make the same data available in a “standard” format, the most popular of these are; GeoRSS and KML followed by GML and the other OGC formats. This is now pretty common and there are many sites that reformat data into these forms as part of a mashup. The other approach is to create a service or API to access the data you are publishing or accessing as part of the mashup. This is much rarer as its a pretty complex process involving server based software or processes. However there are a few tools that can help you do this. A great example is Dapper, this allows you to create a web service out of any third party website, it will map that site to an XML schema which can in turn be transformed into any common format, such as a googlemap URL or icalendar format. You can also map it to a custom format such as KML or GML. This is an amazingly powerful if slightly contentious tool that allows you to create some interesting applications. I can see people using it to generate KML services from pretty much any website containing locations.

Once services and Data are available the next step is to do more analysis of that data. Currently mashups combine data so the user can for example look for patterns in the geographic data that they will never have noticed before. What would be even better would be for the systems to look for patterns and relationships rather than leaving it up to the users. Spatial analysis is something we are all comfortable with on the desktop, but mashups are still stuck in visualisation mode. This is understandable as many of the tools to do this analysis have not been available in the web environment. However if data is provided as GML, KML or WFS service then users who have the tools could use the data in much more innovative ways. With standards support built into ArcGIS users could utilize online data sources with many of the ArcGIS extensions such as spatial or geostatistical analyst. At 9.2 with ArcGIS server geoprocessing support, organisations can now begin to make analysis services available as well as just raw data. For example there are many sites displaying houses for sale along with information about nearby schools and amenities, what would be great would be for the user to input their preferences about the importance of certain features, such as schools, transport links house prices, distance to shops, pollution and then for the site to calculate the ideal location based on a specific users preferences. This type of problem is a classic spatial analyst problem of overlaying different surfaces to come up with a ideal locations. Another example of this is this story about using Google Earth to search for potential archeological sites, why not combine this with an automated feature extraction model to search for specific data within the imagery, an example of this is documented in this paper that searched for road junction in online raster maps. Hopefully soon with the release of ArcGIS Server 9.2 we will start to see more of this type of analysis capability move to the web and become incorporated into some new and innovative mashups.

 

Enterprise Mashups in the UK.

It seems like the mapping mashup phenomenon is really beginning to take off in the enterprise, the recent story about a legal software provider integrating Google Maps into their software is one example. Now this is nothing particularly new, software vendors have been integrating with 3rd party services for some time, for example the Searchflow conveyancing web site was integrated, maps and all, into a number of 3rd party legal software products. The difference now appears to be the simplicity with which this can be done without having to build a relationship between the different service suppliers or the end users. This ease of use has done a great deal to promote the use of geography and spatial visualisation of data, but it also seems to be causing some confusion in the commercial sector.

I have seen several ITT’s recently for enterprise GIS systems that request the use of “Google satellite imagery” in the solution. These requirements have been driven either by the fact that they already have a web application that displays Google imagery, or someone has seen Google maps and wants the data in their applications. There are 2 main issues that organisations looking to use so called “Google data” in enterprise applications need to look at, both of these seem to be legal rather than technical.

The first issue is the use of the Google maps API in a non public site. It seems like the Google terms and conditions require any service to be publicly available. If its not then you can use the Google enterprise maps edition. This is not free and although they have just announced availability in Europe this doesn’t include the UK yet. I’m not sure whether this means you can’t do commercial mashups at all or whether they will allow the non commercial API to be used until its available in the UK. The reasoning behind having a commercial service is pretty obvious. As our mapping agencies are fond of mentioning collecting and maintaining high quality spatial data is an expensive business, and why would users go to say DigitalGlobe or even the O.S. when it seems they can get essentially the same data for free from Google. The availability of free applications and data has meant that the perceived value of the data is essentially free, whereas the reality is very different. Someone is paying for the data, presumably Google, but increasingly it is going to be “enterprise” users.

The second issue with integration, is the requirement that you use the API, rather than access the data directly, this is obviously driven by the same issues of data licensing, however it does put a pretty big hurdle in the way of organisations who wish to use the data in a non-web environment. Sometimes we may be guilty of assuming that everyone just uses web applications these days, but there are many many users using desktop applications for data maintenance, analysis and publishing purposes and these users are not going to be able to use the Google services.

Google however are not the only service provider that can deliver mapping data, there are free services and API’s from Yahoo, Microsoft and of course ESRI, but again these all have “commercial” versions or restrictions. For example the Yahoo maps API does not really distinguish between commercial or non commercial, but they do have some technical restrictions such as not using it with GPS derived data! (see 1.f(vi) in the terms) they also have an Image API that allows you to directly access the map images without having to use a web browser client which makes integration a bit simpler.

Microsoft have a dedicated commercial API, but don’t limit what you can do with the public API, as long as you use the API.

The ArcWeb public services are designed just for that, i.e. non commercial use. There are also terms available for organisations who wish to use services in commercial applications.

I’m no lawyer so I wouldn’t take any notice of my interpretations of the T’s &C’s. I guess the thing to remember is that while there are many services available to end users delivering high quality data and imagery free of charge, the same is not necessarily true if you wish to make use of these services within a commercial setting. You should take great care in examining the terms of use before using these services for enterprise mashups.

 

ArcGIS 9.2 UK Tour.

OK So I’m officially useless at this blogging lark, its been a while and I’ve been a little busy but still no excuse. There have been a bunch of events that I failed to get to, I managed to miss both the AGI and the oracle spatial user group by being in Redlands catching up with the PLTS team, and getting a final sneak preview of 9.2.

Talking of which, if you haven’t been on the beta program you now have a chance to get a look at 9.2 in all its glory. ESRI(UK) are running a round of Tech update sessions across the country where you will get to see all the cool new stuff. You can register on the website here the dates and locations are:

• Oct 31st Edinburgh
• Nov 9th Manchester
• Nov 14th Birmingham
• Nov 16th London

The agenda is still to be confirmed, but as its all about 9.2 you can probably expect to see lots about ArcExplorer and ArcGIS server. I would really recommend you try to get to one of these events, as 9.2 really is more than a “dot” release. It introduces some fundamentally new ways of sharing GIS data and functionality around organisations so if you are involved in planning, architecting, managing or building GIS systems its worth seeing what’s new.

A couple of other events to note, If your in Birmingham for the 14th, you could also stay an extra day and catch the second day of the oracle spatial stream at the Oracle UK User group.

Also the UK Geospatial mashup is happening on October the 20th along with an evening of musical mashups the night before where you can pit you DJ skills against our very own DJ CharlesK.

 

3D Video camera demonstrated

James posted that he hadn’t seen any new “must see” hardware this year such as the touch tables and 3D printers that have wowed people in the past. Well I think the 3D video camera demonstrated in this video of a presentation given to Google might be a contender for next year!

It uses a laser array to capture a 3D image up to 5km away like lidar but a whole frame at a time and combines this with a standard 2D visible or IR image to create a 3D model of each frame, it can capture up to 30 frames a second, which means it can capture moving objects, or can be mounted to moving platforms such as an aircraft or car. It can also see through windows to capture the inside of buildings in 3D, or through particulates such as smoke or fog. There must be so many applications for this camera, from performing detailed 3D and visual surveys of streets and towns, for mapping hazardous areas, or just quickly collecting data that would make great content in a 3D earth browser application!

Whenever I saw a movie where the intelligence services took a cctv image then enhanced it and started flying around it in 3D it always made me laugh at the unfeasibility of this, well with this camera you can actually do this, and they demonstrate doing exactly that around the Google campus.

I’m putting one of these on my Christmas shopping list, though I’m still waiting for my 3D touch screen “coffee table”.

 

UK Geoblogging

Well they always say that if you write a blog, it doesn’t matter what subject you theoretically focus on you will invariably just end up blogging about other blogs. So in the best traditions of blogging this blog is all about blogs.

Reading peoples reports of the geoblogger meetup at the UC this year and at where 2.0 got me wondering what would happen if you tried to do the same in the UK. The basic conclusion I came to was that there probably wouldnt be very many people turning up. Not because there are no blogs at all, but the few that there are cover an amazing diversity of subjects with not much overlap, and it would be hard to think of a UK event that would have more than a couple of people at. I guess the biggest traditional GI event would be the AGI conference in september but even then i’m not sure there would be many bloggers, there may be more at some of the opengeodata events, which seems to have a pretty vibrant community building up around it.

Its perfectly possible that I’m just not paying attention and there are hundreds of UK based geoblogs out there that i haven’t yet found. It may also be that blogs are not so popular in the UK, perhaps we prefer keeping ourselves to ourselves here. However there are a number of popular mail lists here that have quite vocal people on but not so many blogs.

Does it matter where bloggers are based? It seems like the majority of things that people talk about are technology based and have broad international appeal, so some of the blogs I’ve listed below you’d be hard pressed to know they were UK based, and I’m sure there are others that are UK based but no-one knows. I did find http://geoblogger.eu/ which has a good list of european blogs in many languages.

I’m sure there are plenty of subjects that are of interest to the UK only, both technology such as working with UK projections or UK specific Data and organisational or even just events information, but not many people post about these. Maybe there are not many people in the UK who read blogs for local GI news, I have to say that most of the comments I have had left or emailed on my blog have not been from the UK.

Anyway below is a short list of blogs that are uk based (i think) that could be called geoblogs. I’m sure people will point me at hundreds more, but this is a start.

http://mapperz.blogspot.com/

http://bullsworld.blogspot.com/

http://www.edparsons.com/

http://kosmosofgis.wordpress.com/about/

http://www.ogleearth.com/

http://www.opengeodata.org/

http://thinkwhere.wordpress.com/

http://blacksworld.net/blog/

http://blog.dixo.net/

 

ESRI to Support Open Source DBMS

Well I haven’t blogged anything for ages, as I’ve been a little busy finishing up my current project, and pretty much onsite most of July, but hopefully things will calm down a bit now. I’ve just been trawling through the UC Q&A pages seeing what all the lucky people who are off to the user conference will get to see. (Not that I’m jealous or anything!). Everyone seems to be focusing on the new licensing/tiering of ArcGIS Server. I think the change is a good thing, though it will take a while to get used to thinking about 1 product “ArcGIS Server” with multiple levels rather than all the different individual products (ArcSDE, ArcGIS, ArcIMS etc) as we do at the moment. Some people have complained that its now even more complex, but i think once you get used to it its actually simpler, and its going to be a lot easier and more consistent when trying to explain to potential customers.

Anyway to the point: I discovered this little gem in the Q and A section that seems to have been overlooked in all the Server discussion. It seems that ESRI will be supporting an open source DBMS in the form of PostGreSQL, this support is currently in testing and due for release sometime after the initial 9.2 release. You can read about it at the Q&A section under the Geodatabase and ArcSDE section (Q11).

I’d heard rumours of this, and noticed a few bugs logged in the 9.2 system that mentioned Postgres, which pointed to something happening, but its nice to see something public about this, It will be interesting to see whether people actually choose this platform to run an enterprise Spatial database with ESRI technology, or whether organisations that adopt an open source database will be less inclined to use it with commercial software.

Technorati : ESRI, GIS Open Source, postgres

 

Google’s new GI related Patents Published.

I came across this article, which looks at some of the new patent applications recently published from google, a number of them relate to spatial searches and google local. The two that caught my eye are:

Transportation routing
Inventors: Henry Rowley, and Shumeet Baluja
US Patent Application 20060149461
Published July 6, 2006
Filed on December 31, 2004

Abstract

A computer-implemented method of providing personalized route information involves gathering a plurality of past location indicators over time for a wireless client device, determining a future driving objective using the plurality of previously-gathered location indicators, obtaining real-time traffic data for an area proximate to the determined driving objective, and generating a suggested route for the driving objective using the near real-time traffic data.

And

Indexing documents according to geographical relevance
Inventors: Daniel Egnor
US Patent Application 20060149774
Published July 6, 2006
Filed on December 30, 2004

Abstract

A local search engine efficiently indexes documents relevant to a geographical area by indexing, for each document, multiple location identifiers that collectively define an aggregate geographic region. When creating the index, the search engine may determine a set of geographical areas surrounding a geographical area relevant to a document and associate references to the set of geographical areas with the document index.

When I first read the routing patent abstract, it seemed like something that is being done already by many in-car sat nav systems that avoid areas of heavy traffic and dynamically re-route you, however on closer inspection of the full application, it seems that what google have in mind, is keeping track of peoples historical patterns of location, and using these as a basis for generating a route. This would in a sense be a learning algorithm, that would take account of your preferred routes at certain times of day, or any short-cuts that you have on the way home from work at specific times. It would learn you preferences based on day or time and traffic flow and then use these to provide routes. The only flaw i can see in this, is that in areas that you drive a lot and have collected preferences, you probably don’t need driving directions. The Patent doesn’t mention sharing this information, but i can see you might want to extend this functionality, so that if you are driving in an area that is new to you you could take advantage of other peoples shortcuts or time based preferences with routes generated to use that information even though it is someone else’s preferences, I can see that being a much more powerful solution, utilising a community of preferences for everyones benefit.

Technorati : GIS, google

 

Data Interoperability Extension for ArcGIS, my favourite extension

I see that Rob Elkins gave a good link to the Data Interoperability Extension on his blog, which reminded me that i keep meaning to write something about this.

Data Interoperability Extension or DIE as we are not allowed to call it, is by far and away my favourite ArcGIS extension. I have used it extensively in the last two large projects I have worked on. Not only does it have great support for a wide range of data formats, but it can also go a lot further than just reading data natively. You get a full support for defining custom transformations, which allow you to convert data not just between formats, but use a whole bunch of “transformers” on the way. The workbench application gives you drag and drop support for these transformers, allowing you to do things like field manipulation, string transformations and geometry calculations. Once you have built these custom transformations, you can embed them in geoprocessing models so that they are really well integrated into ArcGIS. If I was only able to use a single ArcGIS extension the DIE would be the one. The only confusing thing I find about FME is the number of different license levels that you can have. If you get the DIE, then this is an extension that you buy from ESRI, however if you need additional formats or functionality, you can extend the extension by purchasing additional licenses from safe software. This page outlines some of the options you have with DIE and Fme extensions. Basically you have three options, You can use the standard Data Interoperability Extension, on top of which you can add the snappily named FME For ArcGIS Format pack for ArcGIS Data Interoperability, which builds on the ArcGIS Extension with additional format support. Finally you can go the whole hog and use the full grown FME for ESRI, which gives similar integration with ArcGIS, but also the standalone workbench application. When you combine this with the support for geoprocessing, especially in 9.2 with server and engine support, there is almost no type of data that you cannot support with ArcGIS.

Technorati : ArcGIS, ESRI

 

Whats in ArcGIS Explorer

Since James Posted all his screen shots of ArcGIS Explorer Beta 2 it seems like everyone has been getting all excited about whats going to be in ArcGIS Explorer. At the moment its still in private beta so there have been more questions than answers. However there is a reasonable amount of information posted on the ESRI website about ArcGIS Explorer. You can find it funnily enough in the ArcGIS explorer section. There is an FAQ which should answer a lot of peoples questions, things could still change before release, but it gives a good idea of some of the cool stuff you could do with this. Theres also a couple of videos and a bunch of screen-shots in addition to those shown below.
Reading the FAQ, theres lots of interesting things about ArcGIS Explorer such as support for local data and WMS services, but the thing that i think is most exciting is the extensibility that is available. This applies to both the data and the application.

One of the cool things is being able to publish your own globe and 2D services either commercially or within an organisation. So if the standard imagery is not good enough for you, you could always subscribe to a commercial globe service offering more detailed data, or use a commercial Mastermap service and stream this into the 3D viewer. If as an organisation you are interested in global data, but not necessarily the land based imagery published as default, perhaps you have climate or meteorological data, or Nautical charts or maybe environmental data, you can publish your own 3d globe data across the organisation without relying on the standard published data.

The second area of extensibility is the ability to publish custom tasks, and use the SDK to create custom interfaces to these tasks inside ArcGIS Explorer. The ability to make advanced server based Geo-processing available to the end users is a really powerful way to use ArcGIS Explorer not just for visualisation and pushpin type apps, but to allow users to do real analysis, but still using a simple interface. Things like custom network tracing for utilities networks, routing over non standard data, such a cycle or foot paths, environmental modelling, emergency management and modelling scenarios. All of these as server based tasks initiated by users inputing simple parameters through a task based user interface. I’ve had a quick play with the .Net SDK and there are all sorts of possibilities opened up by this, especially when combined with the support for Geo-processing and models within ArcGIS Sever.

It should be interesting to see what people build with these tools and whether this helps to advance the wider usage of geographic data and services beyond the simple mapping and visualisation that google earth has done such a great job of promoting.

Technorati : GIS, arcgisexplorer, esri, googleearth