10x Faster, No Limits: A Complete Rework of Geometry Assembly in ohsome-planet

With a new approach to transforming geometries, ohsome-planet now transforms OpenStreetMap data from PBF to GeoParquet faster and without relation-size constraints.

This release presents a complete rework of our approach to assemble geometries for complex mutlipolygon [sic] relations. Our new approach tackles two challenges at the same time. First, processing performance was a bottleneck for relations with a very high number of members. As a consequence we had to limit geometry building to relations with a fixed member count, e.g. below 600. Although most relations in OSM have fewer members, there are still enough common cases, such as country boundaries and lakes, that we weren’t able generate in previous versions. Second, the correct handling of invalid geometries and edge cases made processing even slower.

With our new geometry assembler, we can process the global OSM planet history file about 10x faster than before. You will notice this speed-up effect also for smaller regional extracts. On a Lenovo ThinkPad P14s Gen 5 with Intel® Core™ Ultra 7 165H × 22 and 64GB RAM transforming Germany latest pbf into Parquet took just 27 minutes.

Impressive update from the folks at HeiGIT.