Fun with puzzle maps, part II

 Having moved from Cambridge to my family home in SW FR, I found the historic map puzzle i described here on a previous visit. 

A brief history of mine

Update 2: see my journey on the web here.

Update 1: a duo of posts on my Medium professional channel here relates my early computing.

As I go through a 'hard reset' in my life and plan to exit social media, this may be a good time to pause and reflect on my IT journey.

Calgary 1986 see here

Although I started on mainframes at Univ. of Calgary 1979 (here), I transitioned to desktop at Queen's Univ. (here). That post's intro lists articles illustrating the progression from geology to computer mapping and GIS. The reason I was able to do that, is that my eight years of high-school Latin allowed me to read, fix and re-run Unix scripts running petrodata management software in the oil industry, where I spent the bulk of my career.  

That then prepared me for HTML coding in the early pre-Internet days... This was 1986, 3 yrs. before Tim Berners-Lee ushered in the World Wide Web! The City of Calgary in W CAN was optically wired by the then government telephone co. AGT (now Telus) for free, betting on the fact they'd recoup the money - and they did so handsomely - via the traffic of all the oil companies jammed downtown. This was against a backdrop of the Province of Alberta having forward-looking Land Information System based in the capital Edmonton a couple hours drive north (here), but that was the heyday of early data transmission protocols (rpc)  and open binary data (EBCDIC) that opened the data flow cross province on said optical network.

Anecdote 1: as an undergrad at University of Calgary, I learned of Ted Nelson's Xanadu project (here), to catalog the world's information via what later became Hypertext Markup Language (HTML) - few people realise Tim Berners-Lee didn't invent anything, he built on exisiting systems... as we all do - but Nelson was a propeller-head who couldn't  market his idea (here). John Walker, co-founder of Autodesk, reportedly bought the technolgy but it never got anywhere... I heard of this as I was cooling my heels awaiting to meet him, with Carol Bartz his then PA - the same who eventually lead Autodesk then Yahoo, a true pioneering woman in IT - when a surveyor friend and I tried to sell them the idea of a mapping front-end to AutoCAD ... in 1986! Remember MS Windows was launched just the year before (see Side Story here and illustration here incl. photo above). Did you know I received a copy of  Windows v.1.O as part of a hardware upgrade to boost my computer's memory from 640Kb to 1Mb?! Sidebar: Was Bill Gates not famously quoted that no personal computer would ever need over 640Kb RAM  (here)?

CD-ROMs

Note that was the early to mid eighties and the web had yet ot be invented! In fact I helped launch a CD-ROM publishing firm, that put all of the Alberta well data on one CD and petroleum production info on another... using a Texas school board hashing algorithm. CDs were so rare in fact, that we leased the CD readers together with the discs themselves. Aside from the hashing algorithm, CD-ROM publishing conferences at the USGS with Gerry McFaul encouraged us to publish CD data. Another effect of pioneering CDs was that we shipped data to Dublin OH to have the CDs pressed... and US customs did not know what Exabyte data tapes were, so we shipped data as videos!

Anecdote 2: Microsoft was largely credited with popularizing CD for software distribution. While that is true, the real initiative was the US Dept. of Defense - the same people whose DARPA developed the precursor to the internet linking US military to government contractors, chiefly universities in the beginning - the reason? In the Cold War era, it was feared the Soviets might detonate an atom bomb at 5-10 mi. altitude that would fry the entire electronic infrastructure of mainland US, thus disabling the military - oh! and the rest of the country is collateral - I learned via the USGS mentioned above, that CDs were immune to resulting electromagnetic pulses.

Anecdote 3: as I had been on contract to the Geological Survey of Canada Calgary office across from the university, we visited the geological library and offered to scan the index cards to electronically catalog them on CD-ROM - after all our software came from a school library application - do you know what the librarian said then? "If we made the catalog that easily accessible, can you imagine the amount of work that might create?" We obviously had work to do, convincing a traditional business of the advantages of electronic access, taken for granted now!

 

Pre-WWW

Notwithstanding work on CD-ROMs, we benefited form the optical network to start building HTTP networks as we called them, as internet never mind intranet or the WWW hadn't been invented yet, not outside DARPA. My blog posts the first screenshot of an early 'website' (term as yet n/a) under 1986 at top left of the banner menu:

Progressively larger firms in the petroleum service sector took me to Texas, where we did lots of things on the nascent internet. That's when I built my first website, playing with HTML formats on my life and work stories by year, patterned after a glossary I co-authored with another counsellor. You'll recognise the website structure inherited or inspired by my first site a decade earlier:

Standards & Metadata

Parallel to this, I got involved in standards and metadata via CNC/CODATA to try and rationalize semantics of geological terms, which are quite complex. At the time the mining geological community and the USGS made some inroads, and the North American Geological Data Model emerged from that. But that was a long and arduous process that didn't come to fruition until after I switched to GIS in California in 2000. That is in turn written up in the manuscript splitting data into concepts (data taxonomy that evolve over time) and occurrence (geographic location that are fixed). In fact I took one of the geological map sheets and formatted it accordingly as an example: 

There followed a series of standards&metadata papers given at Esri and followed up on my blog and Medium channel (here, here, here, here, here, here and here) as I moved from US to UK. That put me in touch with AGI and eventually the Geospatial Commission, who have big plans supporting digital twins to enhance and enable UK infrastructure.

Volunteered Geographic Information

After I left "da awl bidness" (the oil business Texas-style) in the economic downturn, I became full time Volunteered Geographic Information as this blog is titled. I started a Community Interest Company to address East Anglia rural isolation affected by climate change and the pandemic via local community engagement. I also bird-dogged COVID polling NHS and ONS data. This is on the back of using Ordnance Survey parish data released in public domain with HC Derby Medieval Fendlands and Drainage of the Fens data to build up  a series of agro-history maps here... since Domesday in 1067! Sharing that was followed by a series on interacting with data agencies and improving their data ending here. My highlight was running a YouTube livestream on dsiplaying & tracking COVID data here.

Building from that I drew a series of sea level rise maps and training materials branching into environmental issues for East Anglia re: climate change ending here. I also engaged in the new medium of story maps key along the top level menu, a professional portfolio through the Community Interest Company as well as some of personal interest. In order to help others build the same, I shared here a series of "build your own" resources. This topographic map has sea level rise calculated  from NOAA:

I also posted atop the blog my professional Medium channel on various geodata topics here. Feel free also to peruse my YouTube channel and other resources along the top level menu bar of this blog: It is only visible on the desktop version, so on mobile make sure you scroll to the bottom and select "View web version".

Unlocking Open Data from a legacy site

 In the process of looking at land cover per previous post, I found Natural England's Natural Capital Atlas - their story map encouraged me to enter EsriUK's latest competition - their Esri dataset is accompanied by ample documentation under data.gov. To save us reverting to ArcMap, let's use the provided layer files and a file geodatabase in three simple steps to map our own region on ArcGIS Pro.

Challenger Expedition DIY Web Map

[Update: Spilhaus video here c/w DIY add-your-own image on eve of 150th anniversary]

The last two blogposts showed how to create correct polar maps in ArcGIS and QGIS here from publicly posted class notes blogged.

Let's look at now creating complete maps in AGOL from publicly available data, and analyzing it over time to see their historic significance. This results in this web map of the HMS Challenger 1873-1876 Expedition:

"They call me the cleaner"

[Update: a mirror article on LinkedIn Pulse expands on historic and business background]

I refer to the character Victor played by Jen Reno in Luc Besson's 1990 film "Nikita".

When is a map not a map, Part II

I just posted on LinkedIn Pulse Opinions are free, but Facts are sacred, taking off from Simon Rogers ex of Guardian Data now at Google Data by way of Twitter. This was spurred by the EU Referendum, and setting aside debates raging around it, this is my contribution in my field of petroleum geology in general and mapping / GIS in particular. UN Comtrade has a fabulous collection of statistics, which are so easy to search & discover, that I simply copy&pasted screenshots into this video.

A day in the life of a petro-data manager - intro

[Update: Talend made the early version complex, so simpler one was posted later]

Have you ever been given plain text geodata and wondered how to database and map it? And has this happened to you lately with tens of thousands of lines of data? Well help is at hand! Here is an ETL  workflow (extract, transform, load) useful to any data manager in or out of petroleum using free tools:

Releasing data really works, Part VI

Three months into posting data on my web service - Part V - I created a short list of free data for Great Britain on my Map Catalog called GB Freebie.

Global Sailings (1662 - 1856, English, Spanish, Dutch, French) revisited

[Update: climate data (wind speed & direction) have now been added in short & long posts]

I originally extracted CLIWOC (CLImatological database for the World's OCeans) ship captains' logs ships locations over a decade ago, to demonstrate the processing of 250K+ points in ArcGIS desktop using then new File Geodatabase. Five years later I posted this on my old website with instructions how to use it in  old ArcGIS Explorer and KML, and then I put a layer package on arcgis.com - both related historic details like de laPerouse's demise below, the importance of data standards and metadata, and the interst it generated elsewhere - more recently I posted a time-based variation of same, where using a time slider helps clarifiy complex data on desktop GIS.

Standards & Metadata, Part VIII

My previous post on this topic stated how careful documentation and appropriate metadata high-grades any information that is shared online by giving origin, context and other information. It helps build bridges and I quipped a well-known tear down this wall that also closed my second last post on free data and apps.

Releasing data really works, Part V

It took five days (after hours) to stand up, learn, tweak and display my East Anglia Fenlands project on Mapcentia's web service. It started with a GISuser group post on LinkedIn on Monday, I used my Amazon Web Service free EC2 trial and GeoCloud2 under beta, and by Friday I had it working and styled. No small thanks to Martin Hogh's original work and help, the result is a simple yet modern and pleasing web map.

To Geo or not to Geo, that is the question...

... at least in oil&gas. I always wondered why petroleum was only 5% Esri's market (unofficial from my tenure as petroleum manager there, they publish no figures as a private company). A current rationalization project at an oil major hinted why - I've 'pushed Geo' for 25 yrs. so I saw that in my previous tenure at Halliburton, but that only crystallized later - whilst a large percentage of data has a spatial component in oil&gas, only a small part of it is stored in spatial databases. GIS are generally for surface infrastructure like geology, plants and pipelines, rather than for subsurface exploration and production. Surface data can actually be seen and measured directly on or near the ground, whereas subsurface are interpolated data from drilling and seismic deep in the subsurface. Indeed the challenges in oil exploration in the news of late revolve around this frontier.

On-line spatial data validation, part II

[2013 update: Socium is now 1SpatialCloud]

Three weeks ago I introduced Socium's Online Validation Service (OVS). I showed the Ordnance Survey vector data QC, on the UK Parishes used as a geographic unit for the Medieval Fenlands project. Two weeks ago Socium kindly created a new rule, to post unexpected variations in adjacent feature attributes. In an essentially agricultural era, economic wealth of the Domesday period was quantified by Darby via the number of plough teams per parish. That is the attribute the rule was written for (it's hard-wired right now and Socium plans to add metadata pick lists in the future).

Clouds gathering over the horizon, part V

Will spectacular outages of AWS and now BT cloud services casts a pall over the excitement that has reached even politicians? And now comes this geopolitical issue only hinted at by MSFT's tribulations in Europe over Internet Explorer. This posted in Germany (Google Chrome will translate this page for you) by Ruth Lang - 'SVG queen' to Dino Ravnic 'web vector king'.

Simple Feature or Full Feature Specification for OGC?

The issue of how to write-to and read-from geographic databases has been around for quite some time. Esri shapefiles were a runaway success partly because of their open specification. As we moved onto spatial databases, the Open Geospatial Consortium (OGC) offered the simple feature specification (SFS) that all the players could read to or write from. This came in especially handy for consuming web mapping services (those and many other specifications have grown since). But it gets trickier when it comes to reading from and writing to spatial databases generically. By that I mean not from the native application but from others', like with shape files.

More temporal web maps

Here is last year's Fenlands mashup on 2nd gen. beta GisCloud.com (this data is also posted on the UK academic ShareGeo site and the data.gov.uk apps site):

Trending oilelefant.com, part VI

Last week I posted to relevant Linkedin Groups, a two-page extract on Slideshare of my article in the June Digital Energy Journal - the full source of Better Metadata for GIS can be found in my previous blogpost. I also updated slide 10 of my presentation on same.

"Better metadata for GIS"

Just posted my two-page extract from the June issue of Digital Energy Journal:

We are going to see much more improved "metadata" system for geo-graphical data - which will help integrate it much more closely with bigger information management systems, writes Andrew Zolnai, sales and marketing director, Interactive Net Mapping Ltd.

"The proof of the pudding is in the making"

The FOSS4G conference early this month in Barcelona raised a host of issues as usual. One picked up by James Fee and Jo Cook's blogs among others, is the role of SpatialLite in particular and exchange file formats in general? My main takeway is Jim's point, that while file exchange formats are important, efforts should be focused on internet exchange formats. We all agree that it's usage eventually that will dictate future formats, rather than vendors or standards bodies...

SAGE: Syndication, AGgregation & Entitlement

For-free vs. for-fee is an issue that won't go away any time soon. I discussed this here before, and it came up recently with the adoption of OpenStreetMap on Bing Maps and arcgis.com. In a macro sense it's about data, systems and traffic control, as evidenced by the lock-down of Google search and Blackberry access in China and parts of the Middle East, respectively. At a micro scale it came up in a discussion group on LinkedIn: who owns map symbology derived from public sources?