Most new traffic routes and cable trays are nowadays planned using digital maps, but creating these maps is a complex process: Necessary planning information must be added to a conventional building area map. For example, where are there trees, streetlights or garbage cans? What has been used to pave the road? Could there be any railway lines to look out for? All these snippets of information are collected during on-site visits using 3D measurement technology. Experts then manually sift through, interpret and add them to the maps. It’s a long, drawn out process. An AI based solution, developed by Fraunhofer IPM, now allows for both automated data interpretation and integration into smart maps for the first time. These maps lay the foundations for a fully automated planning process.

A fount of knowledge: a worldwide, one-of-a-kind training data set for urban development and architecture

How does the software recognize houses, signs, trees and curbstones from camera images and scanner data? And how does it differentiate them from other objects? To achieve this, the Fraunhofer IPM team combines conventional geometric and feature-based object recognition with deep learning methods. The deep learning approach is based on artificial neural networks (ANN). These are trained to recognize specific objects. This aspect of machine learning is classed as an artificial intelligence method. A globally unique, comprehensive training data set has been specially developed for ANN training, specifically in the field of infrastructure planning. It includes more than 30 types of objects at different times of day, in different seasons and lighting conditions as well as taking into consideration regional differences.

Deep understanding of 3D data is key to success

As preparation for the automated interpretation of measurement data, the 2D camera data are combined with the 3D scan data so that geometric information as well as the color and texture of surfaces can be retrieved from the data set. This makes it possible to for example differentiate types of trees and surfaces such as asphalt, gravel and grass. “We originally come from the world of hardware,” explains Professor Alexander Reiterer, responsible for the project at Fraunhofer IPM. He continues, “Our laser scanners are amongst the fastest in the world. Over the years, we have developed a deep understanding of data quality and when error sources crop up, we know how to deal with them.” This data understanding as well as newly acquired expertise in the field of AI form the basis for the software tool. This tool generates semantically enriched, smart planning maps automatically from measurement data – something which previously, would never even have been deemed possible. This innovation led to the researchers being crowned “AI Champions BW”.