TAG: Temporal Attention Graph for heterogeneous traffic trajectory prediction

Heterogeneous traffic patterns are commonly observed in pedestrian rich public spaces and unregulated vehicular environments. This poses significant challenges for trajectory prediction due to their complex, dynamic inter-agent relationships. These environments feature diverse agent types whose motions continuously influence one another, creating evolving, non-Euclidean interaction structures that traditional models struggle to capture. To tackle this problem, we propose a novel framework that learns the time-varying importance of all agents in a scene, enabling the model to focus on contextually relevant interactions across time. Our approach incorporates an enhanced spatiotemporal attention mechanism, which avoids simplistic proximity-based or frame-wise weighting. Instead, it adaptively attenuates agent features based on their temporal influence. Influence is learned through a custom attention architecture integrated with Graph Convolutional Networks (GCNs) and Temporal Convolutional Neural Networks (TCNNs). This design helps to extract subtle motion patterns across heterogeneous agents and improves prediction quality. We validate our framework using the ApolloScape dataset, known for its multi-agent and dynamic environment, as well as the ETH and UCY pedestrian datasets. Results show that our method achieves state-of-the-art performance, particularly excelling in heterogeneous environments. The model’s adaptive attention and dynamic interaction encoding contribute to more accurate and generalisable trajectory forecasts.