Cooperative Decision-Making for CAVs at Unsignalized Intersections: A
MARL Approach with Attention and Hierarchical Game Priors
Abstract
The development of autonomous vehicles has shown great potential to
enhance the efficiency and safety of transportation systems. However,
the decision-making issue in complex human-machine mixed traffic
scenarios, such as unsignalized intersections, remains a challenge for
autonomous vehicles. While reinforcement learning (RL) has been used to
solve complex decision-making problems, existing RL methods still have
limitations in dealing with cooperative decision-making of multiple
connected autonomous vehicles (CAVs), ensuring safety during
exploration, and simulating realistic human driver behaviors. In this
paper, a novel and efficient algorithm, Multi-Agent Game-prior Attention
Deep Deterministic Policy Gradient (MA-GA-DDPG), is proposed to address
these limitations. Our proposed algorithm formulates the decision-making
problem of CAVs at unsignalized intersections as a decentralized
multi-agent reinforcement learning problem and incorporates an attention
mechanism to capture interaction dependencies between ego CAV and other
agents. The attention weights between the ego vehicle and other agents
are then used to screen interaction objects and obtain prior
hierarchical game relations, based on which a safety inspector module is
designed to improve the traffic safety. Moreover, we consider the
heterogeneity of human drivers in traffic environments and conduct a
series of comprehensive experiments. The proposed approach shows better
performance in terms of driving safety, efficiency and comfort than
conventional RL algorithms.