In partnership with the OpenJS Foundation, Foursquare is pleased to share an update on kepler.gl. Many thanks to Ib Green, Distinguished Engineer at Foursquare and Shan He, Senior Director of Engineering, for their many contributions to this effort.
Version 3.0 of kepler.gl is now available, bringing new geospatial features to kepler.gl application users and dramatic improvements to developers using the kepler.gl libraries.
What’s New in the kepler.gl 3.0 Application?
Post-Processing Effects
The post-processing effects feature from Foursquare Studio has been upstreamed into the open source kepler.gl, bringing lights and shadows, and a library of image processing effects.
For more information, see Elevate Your Maps with Foursquare’s Artistic Geospatial Tools, which describes the feature set in Foursquare Studio that has now been merged back to kepler.gl.
Apache Arrow brings speed and memory usage improvements
kepler.gl 3.0 starts adding support for the Apache Arrow format, enabling dramatic increases in loading performance and maximum dataset sizes.
In this release, kepler.gl can load GeoArrow files, which can be visualized with the PolygonLayer. See the roadmap section below for more general Apache Arrow file support and support for other layers.
vs. ~20 seconds with GeoJson format.
The trick behind the massive performance boost is that Apache Arrow files are loaded as binary data. Binary data loads:
- avoid time-consuming parsing that is required for textual formats like GeoJSON
- allow kepler.gl to skip some costly data conversion steps when preparing the data for rendering on the GPU
Binary data support in kepler.gl has been a long time in the works and required deep refactoring of the kepler.gl core logic. For some additional information, see the DataContainer bullet in the kepler.gl library updates below and the roadmap.
Substantial parts of the GeoArrow support was contributed by the Center for Spatial Data Science at the University of Chicago.
Foursquare integration
A new Foursquare cloud plugin is available in kepler.gl, letting users save, load and publish kepler.gl maps via their Foursquare accounts. It also enables users to open kepler maps in Foursquare Studio, from which additional functions such as sharing, web publishing, and much more is available.
Foursquare is building a full suite of freemium tools on top of kepler.gl, and kepler.gl can now be used as a part of the Foursquare Studio platform through this new integration.
The Foursquare cloud plugin is an addition to the existing CARTO and Dropbox plugins, giving the user a wider range of options for how to store kepler.gl maps.
Naturally, cloud plugins are optional to use, and none of your data is shared with any cloud provider without the user’s explicit login and save action.
Maplibre Base Map Renderer
The default base map renderer has been changed from Mapbox to Maplibre to ensure that kepler.gl is using an open source basemap that is actively supported by a strong community.
The maplibre integration was provided by Birk Skyum.
Minor features
Apart from the headline features listed above, users may want to be on the lookout for numerous small performance and usability improvements in areas such as:
- Animations
- Map styles
- Dataset handling
- Viewports
- Layers
- Tooltips
What’s new for developers in kepler.gl 3.0?
The vast majority of kepler.gl 3.0 development work has taken place under the hood, improving the kepler.gl frameworks that power all kepler.gl based applications.
kepler.gl is a framework that lets developers quickly build their own customized tools for visual geospatial analytics. In fact, the popular geospatial analytics application hosted at https://kepler.gl is simply a small demo application built on top of the kepler.gl framework.
TypeScript
The entire kepler.gl code base has been converted to TypeScript. This helps mitigate code base complexity and has increased developer velocity:
- Type checks catch mistakes and provide a safety net when making changes to the code.
- Developers can inspect type definitions to confirm expected data formats are.
The TypeScript migration was provided by Foursquare in collaboration with Action Engine.
Modules
The kepler.gl libraries are published as a suite of namespaced npm modules. This makes kepler.gl more modular and composable, and enforces stricter separation of functionality, which should help both developers and maintainers.
| Module | Description |
| @kepler.gl/actions | Redux actions |
| @kepler.gl/cloud-providers | Cloud provide infrastructure |
| @kepler.gl/components | React components |
| @kepler.gl/constants | Various constants |
| @kepler.gl/deckgl-layers | Custom deck.gl layers |
| @kepler.gl/effects | Post-processing effects |
| @kepler.gl/layers | kepler.gl layers (configuration) |
| @kepler.gl/localization | |
| @kepler.gl/processors | Data processors |
| @kepler.gl/reducers | Redux reducers |
| @kepler.gl/schemas | |
| @kepler.gl/styles | CSS styles |
| @kepler.gl/table | Data Container |
| @kepler.gl/tasks | Redux tasks |
| @kepler.gl/types | Various types |
| @kepler.gl/utils | Various utils |
The modularization of kepler.gl was implemented by Foursquare in collaboration with Action Engine.
DataContainer
A new DataContainer interface encapsulates access to data tables. Before the DataContainer was introduced, all data loaded by kepler had to be converted into a common table format defined by JavaScript arrays and objects.
The DataInterface lets developers change how data tables are stored and organized internally, without having to modify all the places where.kepler.gl accesses table data.
The new DataInterface is not just an abstract capability. It is used by the new GeoArrow loader, which can now store the loaded binary tables in a compact and efficient binary form in memory instead of having to convert them to a traditional, bulkier, non-binary JavaScript format.
The DataContainer was developed by Foursquare.
Customizability
Many additional React UI components now export “factories” making them overridable.
When building applications on top of the kepler.gl library, it is often desirable to override components and behaviors of kepler with custom implementations.
Dependency Upgrades
Kepler.gl 3.0 also upgrades a number of its dependencies to the latest versions. This brings in a wide range of bug fixes and performance improvements. Examples of upgraded dependencies are:
- deck.gl 8.9
- loaders.gl 4.0
- React 18
- Node 18
Where is kepler.gl headed?
kepler.gl is under open governance in the OpenJS Foundation, and for the last few years kepler.gl has been maintained by Foursquare, where the original creators are using kepler as an open core for building location platform products such as Foursquare Studio.
While it took a long time for kepler.gl 3.0 to be released, development has never stopped. More than 800 commits have been made to the kepler.gl master branch after the release of kepler.gl v2.0 in February 2020. Though admittedly most of these commits have been focused on strengthening the foundations of kepler.gl rather than new features.
The current goal of the kepler.gl team is to increase release cadence, ideally to a quarterly release and also increase the amount of end-user facing geospatial features in each release.
If you’d like to participate in kepler.gl discussions or if you would consider contributing in any way, it is easy to engage with the kepler team through the OpenJS Foundation at https://www.openvisualization.org/.
Roadmap
As mentioned above, the kepler.gl developer team would like to ship quarterly releases. Some of the major items we hope to work on for those release are:
Binary Data (kepler.gl 3.x) Adding Apache Arrow support to all layers (beyond the PolygonLayer), and extending binary data support to other file formats, so that users can benefit from massive performance gains in more situations.
Cloud Native Geospatial File Formats (kepler.gl 3.x) An exciting trend in the geospatial industry is the momentum around cloud-native geospatial file formats. We’d like to keep adding support for loading from and exporting to these formats, including GeoParquet, FlatGeobuf, PMTiles, etc.
WebGPU (kepler.gl 4.0) Kepler.gl builds on deck.gl, and deck.gl v9 development is focused on WebGPU support. Upgrading kepler.gl to deck.gl v9 once it becomes available will bring opportunities to build even more performant and capable GPU visualizations and compute capabilities.
How to Engage
- Try out kepler.gl today
- Download kepler.gl libraries
- Check out kepler.gl documentation
- Engage with us at the OpenJS Foundation
