The Center for Data Innovation spoke with Michael Kopp, Director of Data Science and Head of Research of HERE Technologies, a provider of mapping services and location data based in the Netherlands. Kopp discussed how HERE uses AI-driven systems and data to improve urban mobility and to ensure automated and autonomous driving becomes safer.
Eline Chivot: What is your mission at HERE?
Michael Kopp: My mission is to innovate and bring new technological solutions which help improve our current products, and shape our future products. I do this through a dual role: I’m head of research, together with my colleague Moritz Neun, as well as a founding director at the Institute of Advanced Research in Artificial Intelligence (IARAI), located in Vienna and independent of HERE. There, we develop open innovation based on data that HERE shares with the institute, and we make it public. This really reflects the way our company thinks about how we can harness AI today.
Chivot: What are the differences between automated driving and autonomous driving, and how do HERE’s technologies and applications cater to both? In particular, can you explain how HERE uses mapping in the development of automated and fully autonomous vehicles?
Kopp: The difference between automated driving and autonomous driving is the level of automation. With automated driving, a system assists a presumed human driver who makes the ultimate decision, whereas with autonomous driving there is no human in the loop, so the car drives all by itself and makes its own decisions.
We are already implementing the technologies needed for fully autonomous cars. For self-driving technology to work, any autonomous agent needs to be able to understand the current context in which the vehicle operates. We have a variety of products that support advanced driver-assistance systems (ADAS), helping drivers make decisions: These systems provide information about speed limits, hazards on the road, and traffic conditions and which are actually paving the way to fully autonomous driving. In particular, HERE developed its HD Live Map—it is a live representation of the world in high-definition, which acts as the brain of the vehicle. The role of mapping as an assistive system that helps with decision-making is crucial to automated and autonomous driving, enabling a vehicle to understand the context it operated in it.
AI, and machine learning, have the ability to essentially change all decisions one makes based on data. Obviously, as a data-rich company, this is of high relevance to our business. Using AI, we do a lot to improve urban mobility—for instance by punching more data through our map and automating processes, we can produce a better map which is fresher and more accurate. Looking at edge computing, our AI-based Live Sense software development kit provides a mobile application that processes data on the edge to make suggestions that keeps drivers informed of unexpected road hazards. We are also making continuous improvements in fleet management and in traffic prediction.
In the future, we could do more towards achieving true multimodal routing, if we could use the same technology that underlines the hope of self-driving—which is advanced control theory—we could use it to manage an entire fleet, so rather than moving just a single vehicle from A to B, we could move an entire fleet. Choosing one element of that fleet to move somewhere else can have a great impact on the rest, an impact that is very difficult to forecast. But this can provide a significant technological advantage.
This technology could also give us the ability to route entire smart cities with a newly defined goal—we would still help people move from A to B, but in a sustainable way, generating lower CO2 emissions. Or, one can think of and apply this to the same problem in various sectors, for instance, in transport and logistics. That is, I believe, where a massive revolution is coming our way.
The key innovation that has given great hopes to achieve self-driving stems from advances in reinforcement learning, which has spurred advanced control. To put it simply, this allows you to solve an enormous problem by just building a simulator for what you want to achieve; the AI then trains itself in that simulator, and comes up with a solution. That might sound too good to be true, but it is actually achievable.
Where we stand right now in self-driving is that it turns out to be much harder than initially thought to get a good simulator. That is partially due to the fact that if you want to simulate real road conditions, nearly all situations you will have to deal with are unlikely to occur—so nearly all your training samples need to be exceptional in different ways. For example, cyclists that take a wrong turn—as a driver you would have the ability to anticipate and know that. We encounter many such situations everyday, some only a few times in your life, but we deal with those everyday. It is very difficult to train and build simulators that are realistic to that level, simulators in which an AI can learn anything we humans know based on our past experiences and a few hundred thousand years of evolution. The lofty aim or expectation we originally had for self-driving cars to be on our roads today hasn’t been achieved. Having said that, the field has massively advanced: Simulators and self-learning algorithms that self-learn in simulators have become better, and there are advances on how these two interact. You will always need machine learning to build the simulator, but maybe you won’t need a full simulator, and only snapshots. In general, I would say that we are well on our way. I believe autonomous driving currently represents a key technological driver in our industry—especially looking at the amount of investment that goes in that direction. And there is more to come, though maybe not at the speed we initially hoped.
I think the hope of self-driving is that it can operate within the cities we have. If we have to reconstruct these, this would have to be done in completely different ways. I’m not sure how a two-traffic system, with separated pedestrian or bicycle lanes and car lanes, would be viable in cities that have an old medieval center. Grafting external lanes onto it wouldn’t be possible, whereas perhaps in cities of the developing world, such as Mexico City, you could see a greater impact and find those types of vehicles on the road sooner.
In addition, two-traffic systems are no panacea: An autonomous vehicle can react in a way that is different from the way in which a human-driven vehicle would react, and that can also happen on dedicated lanes. Accidents normally happen when things out of the ordinary occur.
Chivot: What is the role of publicly-owned data in your sector, and how could it improve access to and sharing of location and safety data, hence improve autonomous driving?
Kopp: Given that the fuel of AI is data, in HERE’s view, universal access to public data is crucial to enable any kind of innovation, create value for businesses, citizens, and our society as a whole. This data can include location data, safety data, etc. It is by using big data that we create real-time maps that can prevent accidents between autonomous vehicles. To be future-proof, we need more open data to be made available to us. By that, I don’t just mean available, I also mean accessible: Data should be open by default, and everyone should be able to know how to access it, how it was collected, what the biases are in it. And we should encourage a culture of sharing data in a privacy-conscious way. This should also come with minimal standards that can enable access to fresh data, and harmonizing these across the EU would make a significant impact.
HERE is part of the European Data Task Force, a public-private partnership supported by the European Commission and aiming at improving road safety by enhancing a whole EU-wide data sharing ecosystem. HERE plays the key role of data aggregator, meaning the HERE platform provides the required infrastructure which enables seamless and secure data exchange amongst partners of the initiative, i.e. car manufacturers, service providers, and member states.
We also collaborate with many other companies and players in the industry. Our main strategy here is our platform, which reflects the “industrialization” of these ideas. People can join, use our knowledge, our location intelligence, combine it with their data, and use parts of our toolchain to understand their data.
Chivot: Data can be collected from different car makers, sensors, and suppliers, and on the basis of different standards. How is this data processed?
Kopp: The dominant operating model today remains the one of a client-server architecture (in which services are provided to clients from a centralized server). This means the cars will extract features and clean them to comply with privacy standards. There is a move towards cars having more compute power, so you can already apply more computing and processing on a car itself. The question is then less about where the data is from a privacy point of view, as you probably want to “leave it” on the car, but more about where you compute what, and what is shared with whom—and that is in flux, driven by more hardware in cars, or on cell towers, where you can do intermediate computing. This is a good thing because, with the increasing “sensorization” we have, there is exponential growth in data. We will need more computing power and new, clever ways of where and how to process the data. This can help with standardization, too. It might be that there are certain data standards on certain types of vehicles but that can then be solved on the vehicle itself, and what you transmit can be the standard you determine. Not that the vehicle adopts it itself—it’s more about the translation into something that you recognize as a standard that can be done more effectively on the vehicle, provided there is more computing power. Wherever the processing is made, we need to understand the fundamental difference between raw sensor data and enriched versions of this data, which is the added value that a company like HERE can provide.
Chivot: What could help fast-forward progress in the use and deployment of ADAS and autonomous driving technologies in your sector in Europe? In what ways do you expect smart maps and location-based services will play a much more major role in the future of urban mobility?
Kopp: The biggest driver to accelerate the use and deployment of technologies would be smart regulation, harmonized at EU level but also taking into account the global picture and the work undertaken at the level of the United Nations Economic Commission for Europe (UNECE) for example. We should avoid by all means trying to overregulate technologies that are nascent and premature, the only consequence of it being to block innovation. But what could be helpful is a smart regulatory framework that provides security, predictability, and legal certainty. Another factor is social acceptance. And of course, investment always helps, it is needed—and it will follow.
I mentioned social acceptance. With the application of technologies pioneered in self-driving, such as advanced control theory (and likely through reinforcement learning), there are indeed failure modes and mechanisms. These occur for very rare cases that have not been simulated and when there are—very occasionally—failures and errors, these make it to the newspapers’ headlines. The point is that, like with any technological advantage, there is a downside to it, as well as benefits such as social good. The question is whether this downside is acceptable for society. We can think of making an analogy with the decision to give driving licenses to 18-year-olds. Statistics show pretty consistently that there is a trade-off, but we accept it, given the benefits allowing this provides. It is never an easy decision. But with any technology, there needs to be cohesion and clarity, a decision about what is politically and socially acceptable or not, and a way to test for this. There should be a clear and common understanding about the benefits and the potential downsides.
Regarding where I expect fully autonomous vehicles may be deployed at scale—I can think of separate lanes, or robotaxis: There is already infrastructure being built in ways that make this possible. But to reiterate, from both the investment and social cohesion points of view, this deployment will depend on the extent to which a legal and regulatory framework provides clarity as to where we draw the line, and defines the trade-offs that are acceptable.
About the future of smart maps and location-based services for urban mobility—they are not just there to recommend good restaurants. They already play, and will continue to play a critical major role in the future of mobility. With hazard warnings, automated assisting driving systems, or even services like HERE EV Charge points, we are already today contributing to the uptake of electric and hybrid vehicles in Europe. For instance, if you want to buy an EV but are afraid to use it on a journey because you are not sure whether and where there are loading or charging stations on the way, the technology can guide you through it. The same holds true for the development of smart deliveries and solutions to overcome the last-mile challenge, as mentioned before. We will also make autonomous driving safer by improving the way we build on live maps, to better anticipate the context a car will find itself in. As mentioned earlier—fleet management can reach a new level, which could enable the creation of cities that are truly smart. For example, it can help you make use of a fleet of publicly-owned vehicles, or taxis, a fleet you have control over, so as to regulate traffic in a certain way, slow down certain lanes, etc. And how you do this, how you control this, how you make those decisions, is enabled by the same technology we already are developing today. This provides an even bigger hope for the promise of self-driving. Furthermore, this can have an immense impact and benefit us all by enhancing sustainability: The technology could be used to reduce CO2 emissions but still get people from A to B. It could be that we develop completely new forms of logistics, so that the current service boundaries (which are about having your own fleet vehicles and distribution centers) could be broken up to allow for better distribution, given you can control and plan better. Of course, that comes with security implications, but I am confident these could be solved.
I mentioned that we could transform today’s cities through technology. We could route traffic more efficiently, and in a way that it is more robust. New modes of transport and systems, such as sharing or communally-owned cars, could be operated 24/7, if you have this combination of autonomous vehicles available and with advanced control theory applied to fleet management. But that’s only one aspect—the other is about how we plan a city. Today, the technology on which we decide to build billion-dollar projects, is something quite anemic from a data science point of view. It’s from the 1950s and has stood the test of time, but with the data and the capabilities we now have, we could already start planning and implementing these projects differently. The shape of future cities would lead to maybe something beyond my imagination, but what you’d want at least, is to be able to simulate and anticipate those secondary effects that are normally so hard to guess from data. We have the raw material to start planning 2.0 versions of our urban environment. There are many considerations to include, naturally, regarding individual preferences on how far from work and home people may want to live, how resilient this planning could be to take the hit of a pandemic, and so on. I look forward to seeing what it could look like, beyond science fiction-like pictures representing future cities.