Lat × Long Homepage

Following last week’s post explaining the tile-addressing schema in PMTiles version 3, Brandon Liu now discusses its compression approach reducing the disk space required to store a global tile dataset to only 91.6 MB.

Update: Brandon clarified on Twitter, the 91.6 MB mentioned above only holds the data required to map Z,X,Y coordinates to the corresponding Tile ID. The actual global map data set is 80GB. (15 August 2022)

Kyle Barron demonstrates how to use deck.gl to render data in GeoArrow format, starting with a GeoJSON dataset of one million building footprints in Utah.

We’ve been able to make web maps with GeoJSON data for some time now, and converting GeoJSON to GeoArrow and preparing the data for deck.gl requires extra development work, so why would you want to use GeoArrow? The short answer: It’s incredibly fast.

GeoArrow overlaps almost exactly with the format that deck.gl expects! So deck.gl can render from GeoArrow’s physical representation very efficiently. For point and linestring geometry types, the underlying coordinates array can essentially be copied directly to the GPU with no CPU processing required. For polygon geometries, only polygon tessellation still needs to happen on the CPU.

We’re looking at the not-so-distant future of web mapping here, when we can render millions of features onto a web map without a noticeable impact on performance.

MapLibre 2.2.0 introduces 3D Terrain Maps

A new version of MapLibre was released just yesterday. Release 2.2.0 is a minor version in semantic-versioning speak, but it adds a major feature: 3D terrain maps, enabling developers to visualise the topography of land surfaces in interactive maps on the Web.

Mapbox introduced a similar feature to Mapbox GL 2.0.0, the first of the library’s releases after moving away from an open-source license. MapLibre is a fork of Mapbox GL that was created to preserve and continue the work under an open-source license. Thanks to the efforts of MapLibre maintainers, developers can now build interactive 3D maps for the Web using open-source technology.

I’ve played around with a small example based on an example by Oliver Wipfli, and the results look really slick. Some minor issues still exist, for example, the hill shade disappearing when you tilt the map close to a horizontal view.

MapLibre's 3D terrain in action; also demonstrating smaller issues when tilting the map close to a horizontal view.

It’s great to see the project under such active development. Mapbox has paved the way for modern web mapping with vector tiles, but it’s good to have more competition in the space — even better when the competition is open-source.

PMTiles version 3 introduces a new tile-addressing schema. Instead of using Z,X,Y tile coordinates, the new schema uses tile IDs based on the tile’s position within a series of Hilbert curves:

The TileId 36052 corresponds to the Z,X,Y position of 8,40,87. The calculation of ID uses a pyramid of Hilbert curves starting at TileId=0 for zoom level 0. The next zoom level, a 2x2 square, occupies the next four IDs in the ID space TileId=(1,2,3,4), the next level being the next 16 IDs, and so on.

And to not duplicate tiles that contain virtually no information (for example, tiles just showing water) the RunLength indicates how many times a tile will be repeated within the Hilbert curve, so vast areas of the ocean can be represented with just one tile.

Ocean tiles are not only repetitive, but sparse and often contiguous in Hilbert space. This entry:

TileID=2578427,RunLength=107977,Offset=3650651795,Length=42

means that the 44 byte vector tile with a single square in the layer ocean is repeated over 100,000 times, starting at Z,X,Y=11,285,1311 and ending at 11,19,1304.

Neat.

Update: A new post outlines the disk layout and compression approach of PMTiles version 3. (15 August 2022)

Outreachy Opens Applications for Tech Internships Starting in December 2022

Outreachy has opened applications for the winter cohort of interns beginning in December 2022.

Outreachy is an internship program designed for young people to make their first mark in open-source software development, specifically people from underrepresented groups in the tech industry. (So, if you’re a European white dude, you don’t need to apply.) The internships come with a $7,000 stipend for three months and are fully remote.

The mentoring projects for the next round will only be announced at the end of September. Still, it usually includes projects with a data-collection and data-management focus, some with a geospatial element. The most recently completed round included interns at Ushahidi and ODK-X.

The program is an excellent opportunity to get into open source and add a fantastic project to your portfolio. I’ve mentored Outreachy interns before, and some went to build successful careers at big names such as RedHat and Google. (Obviously, because my former mentees are smart and driven software engineers and not because I’m a particularly great mentor.)

Ohsome Adds New Filters Feature for Detailed Analysis of OpenStreetMap's History

Ohsome Dashboard, part of a suite of products for exploring OpenStreetMap’s history built at Heidelberg Institute for Geoinformation Technology (HeiGIT), added a handy new feature last week:

We have a new advanced filter feature available in ohsome dashboard. Now you can globally analyze arbitrary combinations of tags and geometry types over the history of OSM.

The Ohsome Dashboard lets you explore the history of OpenStreetMap by looking at arbitrary combinations of tags, OSM object types, periods of time, and areas of interest. Using a more practical description, you see how the length of all ways tagged with highway=primary has developed over the last five years and compare those numbers between Germany, France, and the UK.

Screenshot of a Ohsome graph depicting the development of length of primary highways on OpenStreetMap in Germany, France, and the UK.
Ohsome showing changes of length of primary highways on OpenStreetMap in Germany, France, and the UK.

Amazingly, these results can be produced on the fly. Sure it takes a minute or two to compute, but we’re dealing with vast amounts of data here. The data-exploration products from HeiGIT and the GIScience group at Heidelberg University have come a long way in the last ten years. OSMatrix, which we first released in 2012, was nice to look at, but it wasn’t nearly as helpful in exploring OpenStreetMap’s vast dataset. All of OSMatrix’s data was precomputed into hexagonal bins, and comparisons were only possible for a tiny area.

Django 4.1: New Features for Handling Geometries

Django 4.1 brings two new features for handling geographic geometries; allowing developers to create valid and canonical geometries more easily:

  • GEOSGeometry.make_valid() returns a valid representation of a geometry. Unless the geometry is already valid, then the geometry is returned without changes. The function is very similar to ST_MakeValid in PostGIS.
  • GEOSGeometry.normalize() returns the canonical, or standardised, representation of a geometry. The method now accepts the optional clone keyword indicating that the normalisation must not change the geometry and instead return a new geometry object.

Additionally, support for GDAL 2.1 and PostGIS 2.4 has been removed, and the Django Admin’s OpenLayersWidget is now deprecated.

Check the release notes for all other changes in Django 4.1.

James Fee, on the Geomob Podcast, Remembers Blogging Before Social Media

I really enjoyed this episode of the Geomob podcast with James Fee.

There’s not a lot of geo content in this episode. Instead, it’s a trip down Memory Lane with James and Ed reminiscing about the good old days of blogging. That time during the mid-noughties when people were writing blogs, linking and discussing each other’s work — and before everyone went to Twitter to complain. I’ve been feeling a bit nostalgic recently about that time, so this episode hit a spot.

Good content is still out there, except everything is now hidden away on Substack or Medium, and even indy sites rarely publicise RSS feeds. Flashy images, memes and hot takes create clicks, so well-written 500-word stories aren’t widely shared. It’s one of the main reasons I started this blog. I want it to be a place to share content, cool projects, and long-form writing beyond three 280-character blocks of words topped off with a meme.

Kyle Onda with a concise overview of file-based and web-API vector data formats. The post looks specifically at the applicability of sharing water data, but the conclusions can easily be transferred to other domains.

Humanitarian OpenStreetMap Team has teamed up with local organising partners to host community events in several locations around the world.

Instead of hosting a single event, this year we are investing our time and resources in supporting global, regional, and local conferences and community events around the world to bring the spirit of the Summit to thousands of new people.

I love this idea. Instead of flying-in people to one big event hosted in Europe or Northern America, surely excluding many people from attending because of travel costs and visa requirements, HOT brings the event closer to the community and the people who benefit from their work.

Twelve events are currently planned until the end of 2022:

  • 19 August: State of the Map – Florence, Italy
  • 20 August: Ensemble pour la cartographie participative – Butembo, DRC
  • 22 August: FOSS4G – Florence Italy
  • 19 September : National Meeting of Geography Students (ENEG) – Guanajuato, Mexico
  • 24 October: GeONG – Chambery, France
  • 30 October: Popular & Collaborative Disaster Risk Management and OpenStreetMap – Lima, Peru
  • 31 October: Teto Brasil annual gathering + Cidade em Foco (City in Focus) – São Paulo, Brazil
  • 11 November: Conference Internationale sur la digitalisation des Territoires (CIDT) – Bamako, Mali
  • 14 November: Pista ng Mapa – Cebu City, Philippines
  • 24 November: CAFDO3 – Tunis, Tunisia
  • 28 November: Pacific Geospatial Conference – Suva, Fiji
  • Date tbc: Kombi Kartografi / SotM Haiti – Port-au-Prince, Haiti

Editing OpenStreetMap with Mapillary and RapiD

Detailed editing in OpenStreetMap, adding buildings, turn restrictions, or street crossings can be laborious and time-consuming. But more data and newer tools are available to assist armchair mapping from the comfort of your home:

  • Mapillary provides street-level imagery and point data extracted from the images, which you can use to guide editing in popular editors like iD or JSOM.
  • RapiD, an extended version of OpenStreetMap’s default iD editor, provides additional datasets from Microsoft, Esri, FacebookMeta and functionality to integrate the data into OpenStreetMap.

Open Mapping Hubs and Meta recently hosted an online workshop introducing how to use Mapillary and RapiD to edit OpenStreetMap, and the recording is available on YouTube.

Both helpers come with caveats. During my very unscientific review (I checked a few neighbourhoods around the world that I’m familiar with), I noticed that Mapillary images can be pretty outdated – most images I saw were from 2019 or earlier, some even from 2014. And for RapiD, the OpenStreetMap Wiki includes a big banner saying that every edit must be reviewed individually, otherwise the modifications are considered an import.

Christopher Beddow takes an in-depth look at Visual Positioning Systems (VPS), the solutions companies like Google, Niantic, or Snap have built, and what possibilities the technology opens.

VPS is naturally associated with Augmented Reality (AR), because of the way it enables AR services. It serves as one of several bridges between the more legacy geospatial topics like maps, data, location, and the world building that demands more than legacy systems typically offer.

Advancements in alternative positioning technologies seem to rekindle the hype around augmented reality. So far VPS is mainly used with video games and in product demonstrations of navigation technology, but I haven’t seen any applications of augmented reality beyond that.

But then there is, of course, the metaverse.

Whether you’re just beginning with Leaflet or you’ve been around when Leaflet 0.7 was the current release, this collection of more than sixty Leaflet examples is a valuable resource for anyone. There are basic examples like setting up a Leaflet map or adding a marker, solutions to more complex problems like fitting bounds with padding, and advanced concepts such as overlaying images or searching across layers.

Planet outlines its updated strategy, aiming to become a company that doesn’t just operate earth-observation satellites and provides remote-sensing data. Planet wants to be a company that also runs an earth-data platform allowing users to gather insights from Planet’s data.

The most interesting part of the marketing material is that it’s one of the rare cases that (sort of, in a sugar-coded way) admits that their product isn’t just used to save the environment or ensure every human can eat. Geospatial products are often used to achieve questionable goals, including fighting wars:

There are also security threats, very present as we write this during the war in Ukraine, for which the transparency created by daily broad coverage imagery can help illuminate events in a factual, unbiased and democratized way, reducing likelihood of miscalculation and escalation, and providing a common operating picture for society.

Working in geospatial, we all want to use our skills to create tools or to produce data that ultimately contribute to a better life on earth. But the companies we work for still have to make money, and the clients with the deepest pockets usually aren’t the ones that primarily care about world peace and ensuring every human on earth is well off — a conundrum Tom MacWright captured previously in Ethics in Geo.

A new release of the web-mapping library OpenLayers is out. v6.15 includes, among other things, performance improvements, enhanced styling options for symbols, and bug fixes.

SatSummit is back this year after a four-year break, bringing together experts from the satellite industry, global development, environmental protection, and governments to discuss how satellite data can contribute to addressing the planet’s most pressing issues. It’s scheduled for 28 and 29 September 2022 at Convene in Washington, D.C.

The conference schedule is still in the making, but tickets are on sale now.

Now here’s a very cool project: Allmaps, a browser application made by Bert Spaan and Jules Schoonman, lets you geo-reference images from public sources such as libraries and public archives. You provide the URL of an image and set the reference points via the Allmaps interface. Ideal for laying digitised images of old maps over modern-day data and understanding how the geography has changed.

A digitised map from Antwerp overlayed of modern-day data.
A map from 1860s Antwerp from the Boston Public Library layed over a modern-day digital map. Screenshot from Allmaps.org

What is unique about Allmaps are the technical underpinnings of the application; relying heavily on open standards to access images and store annotations:

  • Image resources are accessed via International Image Interoperability Framework’s (IIIF) Image API, which doesn’t just return an image; it allows developers to specify region, size, rotation, and format of the returned image — ideal to crop, resize and rotate an image into place on a map.
  • The map’s control points are stored using a format based on W3C’s Web Annotation Data Model. The custom format uses GeoJSON to represent geographic coordinates of control points while placing the corresponding pixel coordinates inside the properties object.

Allmaps combines existing standards in a novel way, designing for interoperability from the start and enabling sharing of geo-referencing data with other applications.

mbtiles-s3-server is a new Python library, developed by Michal Charemza, which reads vector geo-data from an MBTiles file stored in S3 and serves it as vector tiles. The library leverages range requests to query an SQLite database in S3.

Very much in line with advancements in file-based, cloud-optimised data storage we’ve been seeing in the last couple of years.

Bill Dollins reflects on the value of industry standards after working with proprietary product APIs:

In the geospatial field, the work of OGC gives us a bit more shared understanding. Because of the Simple Features Specification, we have GeoJSON, GML, GeoPackage, and various similar implementations across multiple open-source and proprietary database systems and data warehouses. Each of those implementations has benefits and shortcomings, but their common root shortens the time to productivity with each. The same can be said of interfaces, such as WxS. I have often been critical of WxS, but, for all the inefficiencies across the various specs, they do provide a level of predictability across implementing technologies which frees a developer to focus on higher-level issues.

OGC’s W*S specifications (e.g., WMS, WFS, or WCS) share similar features. Each provides a getCapabilities operation advertising the service’s — well — capabilities and operations to access the service’s items (getMap, getFeature, or getCoverage). The precise parameters required to execute the requests do vary, and so do server responses, but a good understanding of one specification can be transferred to other similar specifications.

The same flexibility and predictability in built into newer standards today, like OGC API - Features, and community specifications like STAC — both share the same foundation. OGC’s processes may be slow, and the specifications may not make for an entertaining read but its diligent process leads to predictable API design, enabling service and client developers to implement applications consistently and predictably.

You appreciate that more once you had the pleasure to build a service against the Salesforce API.

Will Cadell, CEO of Sparkgeo, starts a newsletter:

Starting up a substack to talk about the super-niche topic of strategic geospatial thinking and tools. It’s going to be like 6 of us, but you’re invited. The bonus is that we get to see into the future and, if we are really clever, we get to sculpt it too.

Very niche indeed, but this will be interesting if Will’s recent thread discussing the future geospatial market is any indication. I wish it was a blog and not on Substack. Thankfully there’s also a secret RSS feed if you don’t want to hand out your email to follow along.