Andrew Map selection

My web presence

Saturday, 27 September 2014

Web maps on steroids

The last 6 blog posts over the last 3 months chronicled the use of dynamic maps using time attribute - years for historic ship tracks and wind data from same 150-350 yrs ago - not to animate maps but to filter them by decade and manage data fetches on ArcGIS Online. A parallel series of posts showed mega data sets on Amazon Web Services, as Mapcentia assured me postGIS handled giga datasets...

click image to enlarge or go to map directly

Well this turned out to be quite true, as detailed on map catalog blog: almost half a million points  totalling almost a gigabyte of data post at a blistering speed... faster than the desktop in fact! This is  due to tile caching in Leaflet on the Mapcentia GeoCloud stack reading AmazonWeb Services data on postGIS.

Why post so may points, one may ask? Because CLIWOC chronicles over 180 weather and shipping attributes almost daily along sailings over 100 years (plus a sample 100 years older) from British, Dutch, French, and Spanish maritime agencies! This resulted in over 230,000 points that doubled to 470,000 when multilingual lookup tables were joined to make wind force and direction computable attributes. I queried this on my blog too, in case I lost my way somewhere... In fact my friend Hussein Nassr, who wrote two books on ArcGIS Server and Geodatabases, pointed out that:
It is not recommended to use relational model for such huge data. This is the era of Big data, in technical words, De-normalize, minimize the number of tables, eliminate unnecessary look ups and domains, it is ok to have duplicated data, we don't have a memory or space problem anymore.
So I did just that, and posted  a redux version of same that maintained the features but cleaned the attributes, and resulted in a 181Mb file geodatabase (33Mb compressed). These can be found on ArcGIS Online and ShareGeo Open as shapes.
And lest we forget, this is just the geo part - climatologists and historians who know this data are in the process of collecting historic weather data, to study climate change history in multidisciplinary and multinational efforts such as ACRE...

Saturday, 6 September 2014

On joining and merging historic multi-lingual geodata

Earlier posts chronicled the history even the beauty of historic shipping and climate data from CLIWOC. British, Dutch, French and Spanish maritime agencies transferred paper logs to digital records. In doing so look-up tables allowed to convert multi-lingual records into quantifiable attributes. Something odd (to me) happened in the process of mapping these: over 1/4M records doubled to almost 1/2M when look-ups were joined and then wind and direction tables merged to create maps symbolised by wind force and orientation.

Wednesday, 27 August 2014

The happenstance art of maps

I showed recently how CLIWOC weather data from ships captains logs dating 1662 to 1885 totalled almost 1/2M points. It started with a 1/4M ships tracks, and combining look-up tables from four maritime agencies they yield numeric wind force and direction...

Friday, 15 August 2014

Dynamic maps final (for now)

[Update: longhand version posted on my catalog blog]

This is the end installment of progressing from static to dynamic maps online. A few lessons learned along the way on posting a quarter million point dataset, which ballooned to almost half a million after links & joins...

Saturday, 9 August 2014

Lessons from 'static to dynamic maps'

Last month related the stumbling blocks in posting too much data on - time animation pushed the limits of stock web service even when limits are set above 250K points - and other services don't offer animation as yet.

Sunday, 27 July 2014

From static to dynamic maps, continued

Last month I reported on posting on some time-stamped, time-slider or time-aware maps - I showed my WhereIsAndrew map - I also mentioned how time-sliders are a great way to roll-up diverse datasets that are time dependent. CLIWOC Captain's ships logs was the other example cited, and I proceeded to post it as a service from ArcMap on

Saturday, 14 June 2014

From static to dynamic maps, my travel so far

I tell people "I know just enough java to be dangerous", and it has served my well in my prior attempts logged in my old web page. These were all Google Maps API v.2 I built about 5 years ago. This blog as well as my new map catalog showed how I built maps in QGIS then ArcGIS, and then posted them on and AWS via Mapcentia GeoCloud2. I recently posted maps on on desktop and smartphone, static results of 'traveling salesman' geoprocessing on the desktop or online.

Saturday, 10 May 2014

Handy maps

[Update 3: see the traveling salesman problem executed for just a portion of it.
Update 2: see the traveling salesman problem executed for the post code area.
Update1: see the social map created from this on a custom account.]

A 'handy' is what Germans call a mobile or cell phone. I uploaded free ArcGIS for Android on my  smartphone, which now has a decent screen to read maps on - love my Navigon Europe on it, and Google Maps too - but here is a quick&easy application of for the rest of us. 

Sunday, 30 March 2014

... now HOW open is open?

A lot of (virtual) ink has flowed around opening up data, as in this blog, GISuser crowdsourcing open data (below left) etc. etc. And everyone is getting into the act, from White House (below center) and Whitehall (UK Cabinet Office) to the number of open data hits (below right).

Sunday, 16 March 2014