The GeoParquet community is pleased to announce the release of GeoParquet 1.0.0. This is a huge milestone, indicating that the format has been implemented and tested by lots of different users and systems and the core team is confident that it is a stable foundation that won’t change. There are more than 20 different libraries and tools that support the format, and hundreds of gigabytes of public data is available in GeoParquet, from a number of different data providers.
GeoParquet is one of the most promising geo-data formats introduced in the last few years. It adds ability to encode geographic geometries in Apache Parquet.
GeoParquet’s adoption is growing, some big names are already offering data in GeoParquet:
We’re also starting to see data providers like Microsoft, Maxar, Planet, Ordnance Survey and others put new data in GeoParquet. And the community is also converting a number of interesting large scale datasets like the Google Open Buildings and Overture Maps data to GeoParquet on Source Cooperative.
The GeoParquet community will now take the specification through the OGC’s standardisation process, which won’t fundamentally change the current specification but it will add a formal stamp of approval.
Chris Holmes announced the imminent finalisation of the GeoParquet 1.0 specification with the release of the 1.0 release candidate.
Our objective is to launch GeoParquet 1.0.0 at the beginning of September, unless we hear any critical feedback. So this is a great time to try it out and make sure it works with your tools. Please do let us know if you add support to a new tool or make a new GeoParquet dataset, so we can add you to the 1.0.0 release announcement.
On April 25, with the help of 47 contributors and 2,790 commits, the STAC API specification reached its 1.0.0 version release. With this release, the STAC API specification is fully aligned with OGC API - Features Version 1.0 standard and the project aims to maintain alignment with OGC standards as they mature.
Following this milestone for STAC, the community is now working to align STAC extensions with the API spec so each of the extensions reaches 1.0 at some point.
A comprehensive overview of the current statuses of these extensions can be accessed at stac-api-extensions.github.io. At the time of writing this blog post, none of the extensions have reached the 1.0.0 milestone yet. However, no significant changes are expected for the Fields, Sort, Transaction, Filter, and Query extensions, and they are anticipated to attain the 1.0.0 status in the near future.
One thought about the OGC Tiles API lives rent-free in my head. We’ve had working de facto standards for years. Google Maps introduced the idea of map tiles in 2005, and the Tile Map Service specification followed a year later. We’re nearing twenty years of well-established conventions for tile services, so why now? Why do we need a bloated document to describe what mostly fits in a blog post?
Of course, I didn’t read the OGC Tile API specification or any of its siblings. I read the entirety of the WMS and Styled Layer Descriptor specification for university, and reading OGC documents isn’t something I would recommend for fun.
Tim Schaub’s post doesn’t answer the Why, but it sheds light on what this new set of specs add. The OGC Tiles API allows more detailed descriptions of the services behind an API. It formalises tiling for raster, vector and rendered map tiles and it allows to advertise projections, geographic extend and limitations on tile sets on each zoom level.
You could get all of the information by reading the specs but Tim’s summary does a much better job at lowering the barrier to start developing compliant APIs. We need more readable summaries for OGC standards like this.
I’m surprised there isn’t more noise about this: The STAC API Specification has reached its 1.0 milestone.
While core STAC specification defines interfaces to publish and organise data in catalogues and collections, STAC API adds dynamic interfaces to enable machines to search and crawl datasets. For example, it adds STAC search, allowing clients to find STAC items across collections, based on filter criteria like a bounding box or date ranges and more complex queries against specific data properties, like cloud cover.
STAC API 1.0 is a continuation of the 0.9 specification, its previous release. The updated specification introduces a long list of changes; few are fundamental, but some are breaking changes—hence the major-version increase. For a detailed list of changes, check the changelog.
The FlatGeobuf community has started the process to formally adopt the specification as an OGC Community Standard. (A document linked by Bert Temme on Twitter, providing reasoning for the spec to become a standard, is not publicly available anymore.)
FlatGeobuf provides binary encoding for geospatial vector data. It is lossless, streamable, enables random feature access, and is supported by a wide range of geospatial tools and libraries, including QGIS, GDAL, Fiona, and PostGIS.
OGC Community Standards are developed outside the more formal OGC standardisation process, usually by a group of individuals who also implement reference solutions (instead of panels of representatives from large organisations).
Tim Schaub built and ran a crawler on public STAC catalogs to better understand the current adoption of the standard. A couple of figures stand out:
About a third of the catalogs are static, but API-based catalogs provide access to disproportionally more items and assets.
The vast majority of catalogs already implement STAC 1.0.0, which was only finalised in May.
Almost half of all items across all catalogs are not satellite images or other spatial data; they are additional meta represented in various formats such as JSON, XML or HTML.
PMTiles version 3 introduces a new tile-addressing schema. Instead of using Z,X,Y tile coordinates, the new schema uses tile IDs based on the tile’s position within a series of Hilbert curves:
The TileId 36052 corresponds to the Z,X,Y position of 8,40,87. The calculation of ID uses a pyramid of Hilbert curves starting at TileId=0 for zoom level 0. The next zoom level, a 2x2 square, occupies the next four IDs in the ID space TileId=(1,2,3,4), the next level being the next 16 IDs, and so on.
And to not duplicate tiles that contain virtually no information (for example, tiles just showing water) the RunLength indicates how many times a tile will be repeated within the Hilbert curve, so vast areas of the ocean can be represented with just one tile.
Ocean tiles are not only repetitive, but sparse and often contiguous in Hilbert space. This entry:
means that the 44 byte vector tile with a single square in the layer ocean is repeated over 100,000 times, starting at Z,X,Y=11,285,1311 and ending at 11,19,1304.
Neat.
Update: A new post outlines the disk layout and compression approach of PMTiles version 3. (15 August 2022)
Zarr originated in genomics research but has since been adopted by the geospatial community because of its ability to quickly access multi-dimensional data in chunks. Zarr allows accessing a window of its data without having to first download the entire data set and dissect it locally. Think of a thirty-year time series of a grid of ocean-surface temperature data, where you can just retrieve the area around the Canary Islands for the last three years.
Zarr data can be stored in a wide range of storage systems, including object stores, such as AWS S3 or Google Cloud Storage, or in storage accessible via HTTP APIs. This makes Zarr the ideal candidate for cloud-native storage and processing of large, multi-dimensional datasets.
Community standards are a way for the OGC to formally adopt specifications developed outside the OGC standardisation process. A community-standard endorsement signifies that a specification is mature, established, widely used, and implemented into reference software. This is a big step for Zarr 2.0, showing that it is now a de-facto way to access multi-dimensional data sets over the Web.
A community standard usually represents a snapshot of a specification under constant development. The Zarr community already works on advancements to the existing standard, eventually resulting in a new Zarr 3.0 specification and proposed to the OGC as a new community standard. Other work includes an extension to the Zarr 2.0 specification formilising how georefrenced grids should be represented in Zarr.
