Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer

Abstract

Coordinating Multi-Unmanned Surface Vehicle (USV) swarms in complex, adversarial maritime environments is a significant challenge, as existing multi-agent reinforcement learning (MARL) methods often fail to capture intricate spatiotemporal dependencies, leading to suboptimal policies. To address this, we propose Adv-TransAC, a novel Spatio-Temporal Meta-Reinforcement Learning framework. Its core innovation is a hybrid GAT-transformer architecture that decouples spatial and temporal reasoning: a Graph Attention Network (GAT) models instantaneous tactical formations, while a transformer analyzes their temporal evolution to infer intent. This is combined with an adversarial meta-learning mechanism to enable rapid adaptation to opponent tactics. In high-fidelity escort and defense simulations, Adv-TransAC significantly outperforms state-of-the-art MARL baselines in task success rate and policy robustness. The learned policies demonstrate the emergence of complex cooperative behaviors, such as intelligent risk-aware coordination and proactive interception maneuvers. The framework’s practicality is further validated by a communication-efficient federated optimization architecture. By effectively modeling spatiotemporal dynamics and enabling rapid adaptation, Adv-TransAC provides a powerful solution that moves beyond reactive decision-making, establishing a strong foundation for next-generation, intelligent maritime platforms.

Bibliographic Information