Lonboard Effortlessly Renders 3 Million Points in Jupyther
Python library Lonboard promises super fast visualisation of huge geospatial datasets in Jupyter notebooks. The demo renders over three million data points in under three seconds; a load that brings other libraries to their knees.
We’re sharing lonboard, a new python library, to fill this need. On a dataset with 3 million points, ipyleaflet crashed after 3.5 minutes, pydeck crashed after 2.5 minutes, but lonboard successfully rendered in 2.5 seconds.
Impressive speed, all without clustering or downscaling the data. Lonboard renderes exactly the amount of features that it finds in the data set. How is this possible you ask? Lonboard employs efficient binary data encodings, as opposed to more traditional text-based formats like GeoJSON:
Lonboard is so fast because it moves data from Python to JavaScript (in your browser) and then from JavaScript to your Graphics Processing Unit (GPU) more efficiently than ever before. Other Python libraries for interactive maps encode data as GeoJSON to copy from Python to the browser. GeoJSON is extremely slow to read and write, resulting in a very large data file that must be copied to the browser.
With lonboard, the entire pipeline is binary. In Python, GeoPandas to GeoArrow to GeoParquet avoids a text encoding like GeoJSON, resulting in a compressed binary buffer that can be efficiently copied to the browser. In JavaScript, GeoParquet to GeoArrow offers efficient decoding (in WebAssembly). Then deck.gl can interpret the raw binary buffers of the GeoArrow table directly without any parsing (thanks to @geoarrow/deck.gl-layers).