Driving in major cities is characterized by constant stop-and-go traffic as vehicles navigate traffic lights, merge, separate, turn, and park. This frequent stopping and starting is highly inefficient, resulting in increased pollution and higher emissions, including greenhouse gases, per mile driven.

Figure 1. New Tool Evaluates Advancements in Reinforcement Learning.

One potential solution to this issue is eco-driving, a control system designed to enhance the efficiency of autonomous vehicles. Figure 1 shows New Tool Evaluates Advancements in Reinforcement Learning.

But how significant could the impact of such systems be? Would the reduction in emissions justify the investment in the technology? These are part of a broader set of optimization challenges that have proven difficult for researchers to address, as they involve numerous factors such as different vehicle types, weather conditions, road conditions, and traffic light timings.

To tackle a question with so many variables, the first step is to gather comprehensive data from various sources. For example, Wu highlights the need for the layout of the city's network topology, which includes a map of all intersections. Additional data, such as U.S. Geological Survey information on elevations, helps determine road grades. There are also important factors like temperature and humidity data, the mix of vehicle types and ages, and the types of fuels used.

Eco-driving focuses on making small adjustments to reduce unnecessary fuel consumption. For instance, Wu explains that when approaching a red light, "there’s no point in me driving as fast as possible to the red light." By coasting, the vehicle doesn't waste fuel or electricity. Furthermore, if an automated vehicle slows down as it approaches an intersection, conventional cars behind it will also be forced to slow down, amplifying the benefits of efficient driving.

The basic idea behind eco-driving is clear, but quantifying its impact is challenging. "These are complex optimization problems," Wu says, with many factors at play. Consequently, there's growing interest in using AI to solve these difficult control problems.

To address this challenge, Wu and her collaborators developed a new benchmark system called "Intersection Zoo," which focuses on urban eco-driving. This benchmark, outlined in a paper presented at the 2025 International Conference on Learning Representation in Singapore, is designed to help solve some of these complex optimization problems.

Wu points out that multi-agent deep reinforcement learning (DRL) is a promising approach to solving these problems. However, the lack of standard benchmarks to evaluate the effectiveness of DRL methods has hindered progress in the field.

The new benchmark is designed to address a critical issue identified by Wu and her team two years ago: with most existing deep reinforcement learning (DRL) algorithms, training for a specific situation (e.g., one particular intersection) does not result in effective performance when even minor changes are made, such as adding a bike lane or adjusting the timing of traffic lights. Even when the system is retrained for the modified scenario, it still struggles to adapt.

Wu emphasizes that this problem of non-generalizability is not limited to traffic-related tasks. “It goes back down all the way to canonical tasks that the community uses to evaluate progress in algorithm design,” she says. Most traditional benchmarks do not involve modifications, making it difficult to assess whether an algorithm is improving in terms of robustness.

While there are many benchmarks for evaluating DRL algorithms, Wu notes that eco-driving presents a unique challenge with its rich set of characteristics, especially from the perspective of generalizability. No existing benchmark adequately addresses these aspects. This is why Intersection Zoo, with its 1 million data-driven traffic scenarios, stands out as a tool that can significantly advance DRL generalizability. “This benchmark adds to the richness of ways to evaluate deep RL algorithms and progress,” Wu explains.

As for the original question about city traffic, ongoing work will apply the Intersection Zoo benchmark to estimate the impact of eco-driving on emissions in a city, particularly based on the percentage of automated vehicles deployed. However, Wu clarifies that the primary goal of this study is not to develop a system for large-scale eco-driving deployment but rather to support the development of general-purpose DRL algorithms. These algorithms can be applied not just to eco-driving but also to a variety of fields, including autonomous driving, video games, security, robotics, warehousing, and classical control problems.

Source: MIT NEWS

Cite this article:

Priyadharshini S (2025), New Tool Assesses Progress in Reinforcement Learning, AnaTechMaz, pp.124