We have been discussing with a colleague how it would be possible to automatically detect all potholes in large road networks. Last week at the INTERGEO exhibition, I discovered three companies which offer this service. In this market survey, I summarise what I found out about them.
I returned last week from the 25th INTERGEO trade show from Stuttgart. INTERGEO combines a trade fair with a conference over three days. It claims to be the world’s largest event for geodesy, geoinformation and land management. It is held in a different city of Germany every year.
For most people, asset management means managed investments in sovereign and corporate bonds, stocks, and investment funds. However, it is also the emerging name of the activity of administering and optimising the complete life cycle of physical assets and infrastructure, such as transport systems and utilities, distribution networks, production and service plants. Among the 705 exhibitors, I found three which are active in road asset management: Intermap from Budapest, Kaios.AI from Bedum (just north of Groningen), and Vialytics from Stuttgart.
Intermap
Intermap was founded in 1999. Its CEO, István Nikl has an informatics background, not one of surveying. They have been developing geoinformatics systems, desktop and web mapping and reconstruction solutions. More recently, Intermap added software for photogrammetrical and GPS-track-recorded road surveys, for panorama and also three-dimensional surveys.
In addition to georeferenced photos, they can provide point clouds and measurements with an accuracy of a few centimetres. Moreover, Intermap can support their customers throughout the whole workflow by providing surveying hardware and support in both surveying and data analysis.
Their pothole and rut detection algorithm is built using a standard and reliable convolutional neural network architecture and a well-known software library. From what I heard from their developer, I can confirm that they did all the right things.
The detection of street lamps, utility poles and traffic signs can also form part of the task. The CEO has detailed it to me that traffic-sign recognition for autonomous driving has very different requirements from what they do. They have plenty of computational time and multiple images of the same sign but their goal is absolute reliability. For autonomous driving, you need to quickly extract the best possible information.
Kaios.AI
In contrast to Intermap, the Dutch Kaios.AI focuses exclusively on artificial intelligence solutions for road asset management: the detection of traffic signs, road markers (road lines), and different types of road damage. They too use two-dimensional convolutional neural networks on georeferenced images.
The major sticking point in developing machine learning solutions is often not algorithmics or the availability of data but specifically the availability of labelled data. I learnt from Kaios.AI that road agencies had been recording their roads, labelling road damages manually on images with circumscribed polygons, and had been saving them in databases for some time. Kaios.AI uses such data for training their models. (Intermap has worked with geography university students to complete this task.) Given such labelled data, the detection of cracks in the road surface and of potholes is no longer particularly hard at all.
They also have a vision whereby their users can opt in to contribute their data to the further training of Kaios.AI’s learning models. Kaios.AI would not store their customers’ data but its models would change and become better through learning from customer data.
Kaios.AI is a neighbour of and works closely with Horus – View and Explore. Horus is a mobile mapping company which has developed an independent multisensor collection platform. They can fuse information from a 360-degree camera and forward-looking, high-resolution cameras. Their software can create digital maps, georeferenced images and video from the data. They are collecting georeferenced street-level imagery to feed the detection, recognition and classification algorithms of Kaios.AI.
Vialytics
Vialytics was present at EnBW’s smart cities-themed area. One gets the impression that Vialytics is incubated by the utilities company, if it is not outright or partially owned by it.
Their approach differs significantly from that of the previous two. They would send you an iPhone with a windscreen mount for data collection. You would attach the phone from inside to the windscreen of a van or a street sweeper, and record pictures every 4 metres as you drive. Importantly, it would also use the accelerometer of the phone to survey road surface smoothness. In this regard, the philosophy is more in the spirit of current data science than that of the other two providers: get data, any data, even if it’s from a consumer smartphone, and get the most out of it.
The customer is requested to drive all routes at least once every half year. For a price of €100 per kilometre per year, Vialytics evaluates these streets biannually and displays them in a web GIS application, categorised into five groups according to street surface quality. Their demo image shows that they can detect manhole covers, storm drains, cracks and potholes, but it is not their goal to detect traffic signs or road assets broadly.
They claim forty local and district councils among their customers. Being a German start-up, they emphasise their anonimisation capabilities: they blacken out all personal information in the uploaded images (people, cyclists, cars). Vialytics promises to use computational infrastructure located in Germany, but their website is also only available in German. They do not yet seem to have international customers.
Gaps or potholes in the market?
There is a US patent (US9416499B2, System and method for sensing and managing pothole location and pothole characteristics) by a company called Heatwurx Inc. for a very broad workflow of pothole detection and repair using some data sensing approach (mobile and airborne mapping) and some GIS system.
What we had wanted to find out, and still do not know, is if it is possible to detect potholes on satellite imagery. It’d be nice to know at least if this were theoretically possible some time in the future or if an insurmountable physical limit on image resolution prevents this. The use case we are interested in is monitoring long-distance road networks in remote, sparsely populated locations for substantial pothole formation, not merely for cracks.
This post benefited from research by Sebastian Ohse (Geospin GmbH).
Update (T+4 days): The highly complex and classified United States optical spy satellites of the latest KH-11 (Key Hole) type are estimated to have a ground sample distance of 6-10 centimetres. Their unit cost is measured in billions of US dollars!
Bence Mélykúti 