Are you looking into building a location-based service with robust geodata and mapping functionalities? There are several options you can choose from when it comes to map data providers, including off-the-shelf APIs by Google Maps or Mapbox. You can also opt for custom development with an open-source alternative OpenStreetMap. Both options have their pros and cons, and they vary greatly from the pricing perspective. Overall, OpenStreetMap is a great option for businesses looking into building robust, flexible and scalable solutions, however, it also carries a number of risks. One of them is related to geospatial data updates. In the following sections, I explain the data-related challenges you will have to face when it comes to building a solution with OpenStreetMap and how to overcome them.
Geospatial data updates
Map data needs to be updated on a regular and ongoing basis. This isn’t just because a restaurant may shut down and a new one will open in its place, but because administrative divisions may change, for instance, or infrastructure works may affect traffic, thereby affecting the way routes will be determined in the routing system.
When you opt for ready-to-use APIs, map data updates are performed automatically by the provider. Building a system with Google Maps, Mapbox, TomTom or any other provider means you don’t have to take care of the updates yourself.In contrast, a solution powered with OpenStreetMap has to be updated by the owner. OpenStreetMap releases map data updates here on a daily basis. Regardless of the type of tool you're building and the target audience, you should implement these updates into your system as soon as possible.
Challenges in updating OpenStreetMap data
We have been working extensively with Trans.eu, one of the leading logistics platforms in Europe, for the past 14 years. As part of this cooperation, we built the entire cross-platform solution on the open-source geographic data provided by OpenStreetMap. It supports search and navigation in all the languages of the 72 countries covered by the system.
We opted for open-source data because we needed to build a robust, custom logistics platform that indicates traverses, buildings and enables search via postcodes and country codes. We’re currently working on a functionality that indicates warehouses, along with their entrances for delivery vehicles. The solution required functionalities that couldn’t have been built with off-the-shelf APIs. Above all, opting for OpenStreetMap also allowed us to save hundreds of thousands USD on a monthly basis. Using paid APIs offered by Google Maps, Mapbox or other popular providers would incur hefty usage bills (you can read more about it here).
All in all, we needed to build a system with open source data to allow the customer to meet their business objectives. However, this option isn’t free of downsides.
Manual updates
As mentioned, OpenStreetMap does update its data on a daily basis, but it’s the owner’s obligation to implement the updates into their system. To this end, we have implemented an automatic update solution.
Data errors
Since we are dealing with open-source data, it is subject to errors. It did happen in the past that Warsaw, Poland, was accidentally deleted from the data file for several hours. Such a scenario is unacceptable in the logistics industry that relies heavily on the availability of detailed coordinates of the supporting solution. That incident led us to a realization that any solution built on OpenStreetMap must undergo consistent validity checks for the data updates.
Building an automated updates mechanism for OpenStreetMap
To overcome this issue, we built a custom verification process for OpenStreetMap data updates. Here’s what the system dashboard looks like:
What does this automated process involve? There is no point in describing each stage, but I’m listing the key elements below to give you an idea of the complexity of this process.
Keeping the geographic names up to date
Trans.eu is a tool that works across borders and is used by different nationalities. As part of our verification mechanism, we update those names in the system with GeoNames - an open-source resource with community-driven translations. With that, we can be sure that typing in London, Londres or Londyn (in English, French and Polish, respectively) in the search bar will always yield the same search result. The search mechanism itself is powered by ElasticSearch and supports contextual browsing and suggestions.
Embedding the OSM data package
OSM data is processed and tailored to the needs of Trans.eu. This is because every country has a different mapping system, and all of them have to be unified for the tool to be functional and searchable. For that reason, it was important to keep the map data separate for each country. Our updates mechanism includes a script that converts the country-specific data to a unified format used in Trans.eu.
Verification check
The key component of the verification process is a script that double checks whether the new or modified data entries were implemented correctly. If they weren’t, we receive a notification that something requires additional verification. In the case of a recent update, the process discovered errors regarding the map data for Belarus (as indicated by the red ‘validate’ box). In such a case, we receive a note that details out what requires an additional check and corrections.
With this dedicated validation mechanism, we can also know when bigger modifications, such as changes in administrative division, are introduced within a country. Without the validation check, we wouldn’t have known about such changes and the modified data wouldn’t be searchable in this system. With 72 separate geospatial databases for every country that include thousands of territorial units and millions of locations, it wouldn’t be possible to manually check whether such changes were implemented.
Updating map data for solutions powered with OpenStreetMap
When you opt to build a location-based service powered with OpenStreetMap, you will have to develop your own mechanism for updating the map database on a regular basis. Dealing with open-source data is prone to errors, so you should also ensure you validate that date before updating the system. This process can be automated; what we built for Trans.eu can also be implemented in other solutions. The point is: if you want to offer a reliable application, you will require a database verification mechanism. If you’re having doubts about the exact requirements, contact me magda@rst.software and I will connect you with a technical expert who will be able to advise you regarding the exact needs.
