Welcome to Foursquare, Shan He! Last month, Foursquare acquired Unfolded, a next generation platform for geospatial analytics. Shan He is a co-founder of Unfolded, the author of one of Unfolded’s core technologies (Kepler.gl), and a rockstar in the world of data engineering and geospatial analytics.
To get to know more about Shan and reap the benefits of her deep expertise and experience in the field, we asked Shan a few questions:
Tell us a bit about your background and expertise.
I was trained as an architect and received Bachelor’s and Master’s degrees in Architecture Design. While designing buildings allows me to exercise my artistic skill, I am always intrigued by engineering and technical challenges that require logic and systematic thinking. After working for two years as an architect, I applied for the Master in Design and Computation program at MIT and was accepted. The two year study introduced me to computer science and data visualization, which I immediately fell in love with. It is the perfect domain for me because it lies at the intersection of technology and design.
After graduating MIT, I was hired as the first person on Uber’s data visualization team. At Uber, I worked on building tools and visualizations that help people make sense of the large amount of geospatial data collected on our platform.
Tell us about how Unfolded got started.
Unfolded was founded by four of us who were at different teams of Uber working with geospatial data. Ib and I were engineers at the data visualization team, building tools and frameworks for visual exploring and analyzing geospatial data. On the other side of the data pipeline, Isaac was an engineer on the marketplace team. He was the project lead for H3, Uber’s open source hierarchical hexagonal discrete global grid system.
The one who connects the dots is Sina. He was a data scientist on the geospatial intelligence team, and he used the tools and library Ib, Isaac and I built to apply advanced geo analytics models to solve some of the hardest market optimization problems at Uber. All four of us spent a total of twenty years thinking and solving the hard problems of all things geospatial. When we finally sat in the same room thanks to Sina, we echoed each other that the knowledge we had can be applied to unfold deeper insights in geospatial not only in the ride sharing industry but many others. The rest is history.
What were the main challenges and/or opportunities that Unfolded set out to solve or fulfill?
The landscape of geospatial data has rapidly changed over the recent years thanks to GPS technology. Geographic information is no longer just about the land, rivers and mountains in the physical world, but about how humans move and reside in the built environment. This data comes in terabytes, in many different formats and is transient in nature. Traditional GIS software – built for desktop with expensive licenses and steep learning curves – is no longer adequate to handle and make sense of it. The Unfolded team set out to build a geospatial platform that is easy to use and can handle large amounts of data
What is unique about Unfolded’s technology?
Unfolded is building a geospatial analytics platform for big geospatial data analytics and seamless spatial-temporal unification that is accessible through REST APIs, SDKs and Unfolded Studio.
For data scientists, Unfolded SDKs enable maps to be embedded into and controlled from Jupyter notebooks, and Unfolded is offering a growing library of integrations with key geospatial Python packages, soon to include machine learning.
Unfolded’s H3 based platform also makes it easy for users to do geospatial joins of data across the full range of geospatial formats, from tables with points, polygons or zip codes, to vector tiles and satellite imagery.
Finally, the platform includes Unfolded Studio which complements Unfolded analytics service APIs by providing and intuitive user experience for analysts,data scientists as well as operations use cases. Unfolded Studio takes full advantage of the end-user computer’s GPU to enable in-browser visualizations with industry leading performance. You can fluidly filter, aggregate, and playback million-row datasets directly in your browser.
Tell us more about Kepler.gl.
I built kepler.gl when I was at Uber’s data visualization team. Kepler.gl is a data-agnostic, high-performance web-based application for visual exploration of large-scale geospatial datasets. Built on top of deck.gl, kepler.gl can render millions of points representing thousands of trips and perform spatial aggregations on the fly. Using kepler.gl, a user can drag and drop a CSV or GeoJSON file into the browser, visualize it with different map layers, explore it by filtering and aggregating it, and eventually export the final visualization as a static map or an animated video. Instead of spanning multiple browsers and consuming weeks of work at a time, the entire trial and error process occurs in one user interface and can take as little as 10 minutes, no code required. Uber open sourced kepler.gl and donated it to the Urban Computing Foundation under the Linux Foundation.
What opportunities do you see for partners and/or customers to use FSQ + Unfolded technologies? What business challenges can be better-solved using Unfolded’s visualization and analysis capabilities and Foursquare’s industry leading data?
Foursquare has global scale location data, and Unfolded has the technology to analyze and visualize it. The partnership will make sure Foursquare customers not only have high quality, easy to use location data but also the tools and platforms to make sense of it. Geospatial data has tremendous potential to transform the way we understand the market and business operations. It can help answer questions such as the health of an operation geographically, where to pick the next store location, how to deliver services to underserved areas, etc.
What are the benefits and opportunities of an open source software community?
Software products in the same space tend to duplicate a substantial amount of functionality. Every new update to e.g. a geospatial format requires years of engineering work across many companies. But the value that companies offer to customers is not in table-stakes features but in the innovation each company brings. Open source lets us and other users of our frameworks share the cost of this engineering “tax” and more efficiently use our resources on building our actual products.
Development of world class technology stacks requires sustained work by many people for long periods of time, and in an open source environment multiple organizations and individuals can collaboratively contribute. In particular, technical leadership (which is the lifeblood of any complex software project) is much easier to expand and replenish when a wide pool of talented engineers across the globe are working on the same project.
Today, many companies also prefer to use open software, especially when they are able to participate in the open governance of the projects.
Finally, and most importantly, end users overwhelmingly favor the open core approach to product development, and this generates a positive halo around products built on great open source foundations.
What are you most looking forward to about working at Foursquare?
I am excited about what insights we can unlock with Unfolded’s technology using Foursquare data. It will be interested to work a planetary scale dataset that changes over time, and combining Foursquare POI data with other geospatial data formats – like satellite imagery, traffic movement or census demographics to draw correlations and help businesses make informed decisions. By combining Unfolded’s innovative technologies in merging, unifying, enriching and visualizing geospatial data with Foursquare’s comprehensive location and POI data, we can deliver a smart, rich and integrated solution to unleash the power of geospatial.
Want to learn more about how Unfolded and Foursquare’s technologies can be used together to tackle your needs? Reach out using the form below!
