Sunday, 25 September 2016

"They call me the cleaner"

[Update: a mirror article on LinkedIn Pulse expands on historic and business background]

I refer to the character Victor played by Jen Reno in Luc Besson's 1990 film "Nikita".

from YouTube

In the process of helping a 2017 French electoral campaign, I received an electoral list that was complete but with four address fields that were filled in pell-mell by voter registrants. A Safe 2015 FME World Tour presentation showed an unusual use of FME Workbench: to scrape, cull and list music title lists. I did the same here to normalise address fields for upload to a voting campaign website.

Now I'm a spatial not a regex kinda guy, so Safe's tech support engineer Richard Mosley gave me a hand, kudos for his support.  The result is a clean list that can be uploaded to

Here are a few more technical details if you're interested:
  • CSV Feature Writer allows to manipulate non-spatial data
  • StringSearcher transformer on each address field parses them
  • AttributeManager formats them to write them out consistently
  • Country is the constant last field and others are written leftward
  • thus addresses are consistent ADR1, ADR2, ZIP, CITY, CTRY
  • needless to say there's a lot of cleaun-up still to do after that...

And when updates arrive, run this only on the new data or deltas:
  • Another CSV Feature Writer is used with FeatureMerger transformer 
  • use Requestor for the original and Supplier for the new file
  • Merged outputs the common fields and NotMerged the different ones
  • only run the first process against the hopefully shorter difference file
  • this can also be used to find the rejected fields for later QC & reload

Lessons learned were:
  • metadata is king - no verbose PDF? no clean up of CSV lists!
  • a bit of logic and reasoning went a long way toward crafting an FME clean-up script  
  • but there are no shortcuts to QC and final clean-up, "you get out what you put into it"
And thanks again for Safe Software's help.

Tuesday, 6 September 2016

Massive online activity - all is not as it appears - More

[Update: another thorough review and insider's view of same on LinkedIn Pulse
Update2: and a follow-on Pulse post also, prompted by a comment on yet another

This is one more follow-on to the Pokémon Go series, which suggested we cool our jets. In my previous post, I compared Pokémon Go to sandbox video game MineCraft, online game Foldit that helped crowd-source the resolution of complex protein structure and Sim City the granddad of virtual worlds. Here are juxtaposed in Google Trends their search hits as a measure of interest in these.

Friday, 12 August 2016

A question of business models in webmap offerings

Eighteen months ago, Google quietly deprecated its Maps API, and ESRI offered and alternative with ArcGIS Earth, then Mapbox and Carto in quick succession: I blogged then Esri, Google and if the shoe fits... Part 1 and Part 2, mirrored on LinkedIn here and here, respectively. Safe Software, LINQ Ltd. and I basically saw it as the next phase in the battle for The Internet of Things (IoT), which has been gaining traction of late.

Sunday, 7 August 2016

Massive online activity - all is not as it appears - Fin

[Update: interesting tweet exchange w @0mgould at bottom]

I wrote over the past weeks why Pokémon Go is not what it seems:
  • from being hailed as the next big thing in geospatial (neither open nor intended to be)
  • to hiding the access / marketing in&outs its hosts indulge in (internet privacy anyone?)

Monday, 1 August 2016

Massive online activity - all is not as it appears - Cont.

[Update: Caveat emptor, lawsuits already started on Pokémon Go trespassing - BBC
Update 2: follow-on posts here & here on geo-ramifications to virtual reality mapping]

I posted last week re: spatial ownership issues Pokémon Go raised, from the personal (visiting family the weekend it was released in France)

Friday, 15 July 2016

Twenty years... and five generations on the web!

Hard to believe I started my first website in hand-written HTML code in Dallas 20 yrs ago! The original impetus was that at Landmark I had an intranet page using a hand-crafted index bar along the top, not unlike this glossary or my life and work pages thru 2000 and 2004 respectively.

Saturday, 4 June 2016

When is a map not a map, Part II

I just posted on LinkedIn Pulse Opinions are free, but Facts are sacred, taking off from Simon Rogers ex of Guardian Data now at Google Data by way of Twitter. This was spurred by the EU Referendum, and setting aside debates raging around it, this is my contribution in my field of petroleum geology in general and mapping / GIS in particular. UN Comtrade has a fabulous collection of statistics, which are so easy to search & discover, that I simply copy&pasted screenshots into this video.

Monday, 30 May 2016

When is a map not a map

Friends of mine lived in London, then LA and now in Wellington, NZ. I wondered where their peripatetic moves might take them, in a purely geographic destination sense?

Friday, 20 May 2016

Andrew's GIS Platforms reloaded

A GIS group discussion prompted me to update this list of selected desktop & web platforms by delivery and cost - note that it excludes commercially serviced FOSS, as well as web & mobile apps - and the usual caveats apply, see details on last page.

Saturday, 14 May 2016

CoDE Conference 2016: Creative Communication

Just attended a conference crossing over art and science at Anglia Ruskin University, Cambridge Culture of Digital Economies CoDE. It ranged from the impact of video games, through a master class in drone cinematography and video exhibit The Crossing, to interactive demos and lightning talks. Two items stood out for me: Sonic Pi freeware to help introduce school children to digital music, and Lichen Beacons a Raspberry Pi and beacon interactive display of imagery and poetry: