Global harmonization of climate & temperature data since 1850

A recent post, DCENT: Dynamically Consistent ENsemble of Temperature at the earth surface in Harvard Dataverse V1, is a complete dataset to accompany a paper in Nature. This came in perfectly, as a significant extension of the CLIWOC dataset originally posted 20 yrs. ago and reposted here just last week! Rationalizing disparate climate datasets spanning the last 1¾ c. make my efforts look puerile - indeed mine were before breakfast on the then-new internet per earlier blogpost - yet the message and the need are never greater than in the current Climate Emergency. Re: ongoing changes in climate modelling & temperature increases, my comment on an "Media Tell the Truth" WhatsApp group underscores the stark message:

Aaaand it appears someone rationalized ancient & modern climate data... only to find a lower baseline, ergo a higher rate of warming w.r.t. pre-industrial levels! Will it ever stop?

A first blog on CLIWOC reloaded  went from time-intervals to time-series on ship captains' logs, as early climate data collection pre-1880: roughly the start of systematic meteo data gathering.  Then time series graduated to time-enabled maps that compacted vast and complex datasets - ½ M points over 1½ c. with 20+ attributes each from ½ doz. sailing nations - into simple maps. Here is Generation 3 with multi-dimensional datasets in netCDF formats that combine time series into composite files.

Here are three videos at 1, 5 & 10 yr. intervals of the ensemble mean monthly resolution of land and sea surface temperatures in 5×5° squares worldwide since 1850.  

YouTube decadal above (1 min.), quinquennial (2 min.) and annual (8 min.)

The eagle-eyed will note early (19th c.) records are mainly maritime upscaling - 5×5° areas, compared to points in original video reposted below - of ships tracks, and that continental data take ascendance in later (20th. c.) records:

CLIWOC wind force direction (YouTube)

CLIWOC tall ships nationality (YouTube)

Without going into details, the Jupyter Notebook tools (advanced, not used here) display the raw global temperature trends seen across publications on climate warming:

Annual global mean difference w.r.t. 2088

It also shows a bell distribution of temperature data, remarkable considering the harmonization of a 200-member ensemble of climate data.

Another set on monthly climatological data details the above. It is meant to adjust and calibrate the temporal data shown above. Here are the monthly global stats for 1982-2014 stepping from months 1 thru 12. It shows what I call the annual breathing of the globe thru the seasons. 

Last but not least, polar effects are the most severe for climate change: that's where the most ice reside on the globe that are at risk of melt and sea level rise, and that's where the wind patterns converge toward and multiply climate change effects; so here are the Norh and South polar Arctic and Antarctic views of the same, projected onto previous maps, say, here... Don't the annual breathing of the poles thru the seasons show well?

North polar view of monthly climatological data (YouTube)

South polar view of monthly climatological data (YouTube)

CLIWOC tall ships sailings from captains logs

Update: global harmonization of climate data since 1850 follows next.

"CLImatological database of World OCeans" was my first outing using Esri tools as personal project. I found this fabulous dataset with ¼ M points! Detailed here, it came from a 5 yr. EU project to scan captains ships logs for climate data in the 17-19th c. before weather data were recorded. Tall ships were mainly British, Dutch & French, Portuguese notably absent due to Lisbon archive destruction in the 1755 "Great Earthquake" (Wikipedia). 

A return to my roots

Updates: mapping climate data from historic ships & global harmonization follow respectively here & here.

 "You can get Andrew outa maps, but you can't get maps outa Andrew" quipped a GIS map friend when I left Kuwait a dozen years ago... Well after quitting socials, Esri(UK) graciously helped me recover my desktop app. While I lost my story maps and web maps content, I maintained a free dev account - story maps and maps&data - this was chiefly to preserve my Living Atlas content inspired by John Nelson. 

"AI for the rest of us", Part V

While I described opportunities AI gave in prompt engineering and text processing (ending at previous post here), I tried for fun to create a billboard from an advert I saw in TX or CA 20 or 30 yrs ago: it was Home Depot's "got wood?", after the wildly famous "got milk?" campaign from the Milk Board. 

Showing folks  sporting white 'mustaches' from drinking milk too eagerly, it was the dawn of the photoshop era: milk was traced into mustaches or smiles... "and no milk was spilled in the production of the advert".

Web maps for the rest of us

 The previous post on Community Engagement updated the rebranding of cambridgeshire.ai, with accompanying use of "AI for the rest of us" (work backward from here) this blogpost title came from. One of the mentioned changes were working with Wikimedia, OpenStreetMap and Climate Central - my Esri Developper or Non-Profit stacks are free and frozen, respectively - I had a whole lot of work put on ice, the same time I relinquished my original website www.zolnai.ca. This blog is OK however.

"AI for the rest of us", Part IV

Update: follow on Part V.

 The previous post (here) recapped our purpose to use current lessons-learned in new tech to help our community engagement. Here is another way to use AI, to summarise and to decant - summarise in a structured manner - information from an article my colleague asked me to try using Google Gemini (formerly known as Bard, their AI tool).

"AI for the rest of us", Part III

Update: follow on Part IV.

 Part II (here) showed a small but significant use of AI in preparing our Prospectus (link in that post). Let's look at how we're taking this further now.

The Spectator did a great state-of-the-AI here, including basically what we did above on steroids. They also highlighted Google Gemini, Bard's successor we signed up for. We also joined Wikimedia UK in open data space.

"AI for the rest of us", Part II

Update: follow on Part III.

 My previous post (here) showed a useful if trivial example of Bing AI (here), not only replying in the addressed language but also quickly & easily teasing a relationship among related items.  Now let's look at a non-trivial common task, to summarise a 110 page document.

"AI for the rest of us"

 Update: follow on Part II.

In my other blog (here) Language and mores, Part VII (here) described what I found out about my family's status as immigrants in our complex story of a pregnant mother's escape from Hungary and my siblings ½ generation later. When I later discussed how I thought my marriage ended, my family saw a Jungian influence in my thinking. Jung's disciple Adler apparently inspired Kishimi & Koga's recent best seller "The Courage to be disliked" (Google books). Ensued a three-language dicussion to understand what this was all about... Mum prefers our mother tongue Hungarian, my brother lives in Montreal and my sister in Paris.

Community engagement and social prescribing

 This follows an update on this blog here.

We recently rebranded Cottenham.info to Cambridgeshire.ai: the domain name is acquired but page not done yet finished; we have a prospectus as an evergreen document, meaning ever evolving.

Our community involvement over 5 years in March is one object of this blog starting  here (follow the links) and listed here. We engaged with various community interest  parties at the impact of climate change then the pandemic on social isolation in East Anglia.