Sunday, 21 April 2013

Vectors are your friend, Part II (updated)


[Update: ESRI blog post here for clear explanation and treatment of same, thanks Eileen Buckley! See also my ArcGIS Online / Amazon Web Services update at bottom...]

Following on my previous post about giscloud.com posting vector maps directly on-line in HTML5, I loaded NOAA's GSHHG - A Global Self-consistent, Hierarchical, High-resolution Geography Database in its entirety. You must be crazy, you say, to load a 425 Mb dataset on line! But here is the workflow:


  1. go to the website and read the description - cool! world vector data sets for free in a self-consistent format and at multiple scales - but wait! do NOT download 158Mb zipped file
  2. created an account and a project for free on giscloud.com - note they have a freemium model, that is free for personal use, then charges monthly for commercial use
  3. create a map and go to File Manager > Web to Cloud upload - do NOT download then upload even zipped shape files! - GSHHG 425Mb unzipped transferred in 3 min that's 2.5Mb/s
  4. if you want to further increase performance or perform GIS functions, then load them into PostGIS on giscloud.com an access 253,635 (that's 247K) vectors lightning fast
  5. in fact it's so fast you no longer need the hierarchy - 'f' the most detailed level at 278Mb will show in full detail the Aland Islands, S Baltic Sea famous for its numerousness


I  wrote in 2004 that "accuracy and speed no longer need to be trade-offs against one another according to available computer performance", and in 1992 that "worldwide databases are available and the right database engine will show interactive coastline features down to 30 meter resolution". 
Well guess what? This data set plus on-line vehicle just got us there!

There are of course many worldwide datasets, such as the ESRI basemaos, the NACIS Natural Earth free world maps at varying scales too, as well as the arcgis.com datasets such as on this blog's banner map and many other on RT Wilson's site. But there are many innovations here:
  1. online vector maps merge speed and accuracy
  2. freemium business model to enable online access
  3. web-to-web data transfers that save on ISP bills
The map below is an ArcGIS Online update, accessing the same on my AWS stack:

View larger map

In other words as Shakira sang famously:

"You can have it all
... Nothing too big or small
... What you get is exactly what you give
... What you give is exactly what you receive
... You can have it all"