Friday 14 September 2012

Releasing public data really works!

[03 Nov 13 update: "relooped the loop" by testing the corrected Ordnance Survey data
Feb 2013 update: Ordnance Survey cartography stylesheets made available for QGIS
19 Jul 2019 update: reposted it on ArcGIS Online]

UK Ordnance Survey released Open Data to the public two and a half years ago. English Parish boundaries have been more or less constant since the Domesday survey in 1087. That allowed me to post University of Cambridge Don HC Darby and Yale University student Julie Bowring socio-economic data, by simply adding attribute data to the Ordnance Survey shape files. That onerous, if one-time, task was entirely manual: when 1SpatialCloud launched Online Validation  it seemed only natural to try it out; they actually wrote some simple rules and we thus co-branded as quality assured by 1SpatialCloud Online Validation Service wherever I posted the data. Here are the resulting error shape files:

Note: I had QGIS software but not University of Cambridge Division of Archaeology - this was before ArcGIS for Home Use and ArcGIS Online - so below on as a live map the top two features are the validation points:

In the right hand margin of this blog, you can find a flare for my original private cloud.

The resulting 25 spikes and kickbacks out of 1900 Parish polygons for East Anglia are, at 1.3%, a ratio I might have envied in my digital data capture days (1982 - 92 here). Whilst they didn't affect the integrity of my project - Parishes are used as anchors for attributes by which wealth is measured, rather than as areas or perimeters per se - it seemed only natural to inform the Ordnance Survey creators of said deviations.

While I as a user can find and point to the errors, only the authors can fix it - indeed the Ordnance Survey's copyright notice only allows to copy the data under fair dealing - but returning to the download page I found no feedback mechanism other than a Customer Service email... But wait! They promote their data not only online but also in open events publicised by @plangfordsmith such as  this one. I asked there how to feed back this information and a few emails later they check Socium's error polygons and reply that these will be incorporated in the next update... rimshot

So that loops the loop:
  1. from @pterhaar unofficial announcement at Where2.0 early 2010 (thanks @stevenfeldman for this) to the actual release that April
  2. through augmenting a portion in East Anglia in my project, that lead to QCing late last year upon Socium's launch and uncovering errors
  3. to returning the error files this month to the Ordnance Survey, and finally the update of said data at the next data release cycle
In other words in a small way here, releasing public data really worked, and thanks everyone for their help - Drs Oosthuizen and Willmoth at Cambridge University, @giscloud and @mapsnrocks for helping with and post this data,, and for posting it, @sociumcloud for online validation and @ordnancesurvey for reaching out.

No comments:

Post a Comment

Please send me a copy of your prospectus to