Global harmonization of climate & temperature data since 1850

Update: a post in SciTech Daily shows that ocean  atmospheric science - shown in this and previous post - is alive and may help with assessing climate change: it's about the doldrums driven by downdrafts not updrafts; at a localized scale, downdrafts  were dramatically reported in passenger flight incidents here or the sinking off Sicily of a superyacht here, and updrafts in increasing mid-Atlantic hurricane generation here. 

CLIWOC tall ships sailings from captains logs

Update: global harmonization of climate data since 1850 follows next.

"CLImatological database of World OCeans" was my first outing using Esri tools as personal project. I found this fabulous dataset with ¼ M points! Detailed here, it came from a 5 yr. EU project to scan captains ships logs for climate data in the 17-19th c. before weather data were recorded. Tall ships were mainly British, Dutch & French, Portuguese notably absent due to Lisbon archive destruction in the 1755 "Great Earthquake" (Wikipedia). 

Dynamic maps - finis

[Update 2:  this Story Map chronicles the 15 years of this on-again-off-again  work
Update 1: new Spillhaus projection is too cool for ocean data, see this on YouTube]

My last post on dynamic maps and its preceding project recap five years ago outlined how I used ¼M point free-to-use dataset on global historic shipping and climate data. This is the original video of ships'  locations produced a decade ago on Esri ArcMap:

A fond farewell to two old friends

[Update: read here my  new occupation inspired by these six months later]

Over seven years after starting to post on arcgis.com and almost five years after posting mega-datasets on GeoCloud2 via AWS, I have to seriously reconsider my investment in web data. I already mentioned my new direction two posts ago, and now stood down my AWS instance - thanks @mhoegh for his help on Mapcentia - and I will let my arcgis.com account lapse next May, five years after it was created (I already rationalised my Esri accounts, hence the two year gap with opener). Do not despair however...

Historic climate data revisited - 5 - live polar maps

Here are the maps promised at the end of the last post c/w description in that post:

Historic climate data revisited - 4 - polar is POpuLAR

[Update: Part 5 will be the last installment as mentioned at the bottom of this blog-post]

Having explored polar maps here, here and here, was it ever a delight to find one of the earliest maps in that same projection! In This Is the World's Largest and Oldest Map, Culture Trip report how David Rumsey recreated a digital copy of a 1587 map from Milan in no less than 60 pieces:

Historic climate data revisited - 3 - circum-Arctic update

[Update: Part 4 provides a further update on platforms available to map polar data]

In my previous blog, how many layers can be combined in an arresting polar Arctic view. They show the almost zero overlap of historic and current weather and climatic data. They herald the importance of oceanic climate data going back before 1880, when any weather data get scarce.

Historic climate data revisited - 2 - circum-Arctic

[Update: more data for the Arctic were found here and here when writing a paper

Note: Part 3 adds World Port Index data compared to historic Gazetteer
Follow-on class materials the last two blogposts generated are posted here.
Go also to section 5: Arctic / Antarctica GIS application, of 1000 GIS applications ]

Following on the Antarctic blogpost, I took my lessons-learned to the antipodes for these reasons:

Historic climate data revisited - 1 - circum-Antarctic

[Update 2: this series has full CC BY-SA 3.0 class notes here
Update 1: please see a mirror project for the Arctic in Part 2.]

With ongoing debates whether Antarctic ice in increasing or not, and its effect on climate change, we must avail ourselves of as much data as we can. If historic climate data is at hand, not only do they get scarcer going farther back, but 1880 also marks a time prior to which their reliability falls off.

So having mapped climate data off tall ships captains logs from 1750 to 1850, I wondered how far south they sailed, and how much they augmented historic climate data around the Antarctic?

Vectors are your friend, Part III

Esri just updated its World Vector Basemaps (v.2). I updated below the notoriously complex and details Aalan Archipelago offshore Turku, Finland. I contrasted in this blog 4½ years ago posting the GSHHG world vector dataset in giscloud.com, a web vector mapping system, and ArcGIS Online, richer in data but then in raster. See an OpenStreetMap backdrop earlier in my companion blog.

Web maps on steroids

The last 6 blog posts over the last 3 months chronicled the use of dynamic maps using time attribute - years for historic ship tracks and wind data from same 150-350 yrs ago - not to animate maps but to filter them by decade and manage data fetches on ArcGIS Online. A parallel series of posts showed mega data sets on Amazon Web Services, as Mapcentia assured me postGIS handled giga datasets...

On joining and merging historic multi-lingual geodata

Earlier posts chronicled the history even the beauty of historic shipping and climate data from CLIWOC. British, Dutch, French and Spanish maritime agencies transferred paper logs to digital records. In doing so look-up tables allowed to convert multi-lingual records into quantifiable attributes. Something odd (to me) happened in the process of mapping these: over 1/4M records doubled to almost 1/2M when look-ups were joined and then wind and direction tables merged to create maps symbolised by wind force and orientation.

The happenstance art of maps

I showed recently how CLIWOC weather data from ships captains logs dating 1662 to 1885 totalled almost 1/2M points. It started with a 1/4M ships tracks, and combining look-up tables from four maritime agencies they yield numeric wind force and direction...

From static to dynamic maps, continued

Last month I reported on posting on arcgis.com some time-stamped, time-slider or time-aware maps - I showed my WhereIsAndrew map - I also mentioned how time-sliders are a great way to roll-up diverse datasets that are time dependent. CLIWOC Captain's ships logs was the other example cited, and I proceeded to post it as a service from ArcMap on arcgis.com.

From static to dynamic maps, my travel so far

I tell people "I know just enough java to be dangerous", and it has served my well in my prior attempts logged in my old web page. These were all Google Maps API v.2 I built about 5 years ago. This blog as well as my new map catalog showed how I built maps in QGIS then ArcGIS, and then posted them on giscloud.com and AWS via Mapcentia GeoCloud2. I recently posted maps on arcgis.com on desktop and smartphone, static results of 'traveling salesman' geoprocessing on the desktop or online.

Global Sailings (1662 - 1856, English, Spanish, Dutch, French) revisited

[Update: climate data (wind speed & direction) have now been added in short & long posts]

I originally extracted CLIWOC (CLImatological database for the World's OCeans) ship captains' logs ships locations over a decade ago, to demonstrate the processing of 250K+ points in ArcGIS desktop using then new File Geodatabase. Five years later I posted this on my old website with instructions how to use it in  old ArcGIS Explorer and KML, and then I put a layer package on arcgis.com - both related historic details like de laPerouse's demise below, the importance of data standards and metadata, and the interst it generated elsewhere - more recently I posted a time-based variation of same, where using a time slider helps clarifiy complex data on desktop GIS.

Around the world in 30 maps

[Update 2: this map gallery is now the first pane of my personal portfolio]
[Update 1: Apple iOS doesn't support Flash, please try this old link instead]

I revamped my blog to make its header more engaging. I replaced the static image I changed from time to time, with a map gallery. It comes from a Flickr slide show of maps I have collected over time, posted in chronological order. Although each screenshot is captioned and tagged in Flickr, here is the text in the same order:

Roundup of web projects

Is it spring in the air or LinkedIn's new (to me) facility to post projects? Here is a round-up of various projects in the past five years as recently posted on my LinkedIn page:

More on time-based GIS

Time-lapse GIS helps clear the clutter of a quarter million points of ship sailings from captain's logs from 1662 to 1855 re-posted last week. As an at-home project I previously split the data into arbitrary half-century time slices to better visualise it all. But that interfered with seeing trends across the span of data.

The Joy of (con)Text

It's a common geo-rant (thanks AGI'09) that metadata cause alternatively boredom or angst among geo-geeks - why? because we know our data, our professional audience does too, but our wider audience does not. In other words, if we don't write metadata, no-one else will understand the context later on. I found a clear example, when I mapped Captain Cook's ships logs a while ago, and posted on ArcGIS Online beta: