Abstract
Dynamic target tracking is a challenging task for autonomous ships due to the continuous variation of relative motion states and the requirement for coordinated control of position, heading, and speed. This paper proposes a task-oriented continuous decision-making framework based on Soft Actor-Critic (SAC) reinforcement learning for the autonomous approach and stable following of dynamic target vessels. A finite-history-enhanced SAC framework is developed by incorporating LSTM-based sequence encoding into the policy and value networks to capture recent evolution patterns of target motion and own-ship maneuvering responses. Furthermore, a sector-annular tracking region defined by distance and relative bearing constraints is constructed and a multi-component reward function is designed to integrate distance convergence, heading adjustment, region maintenance, speed matching, and control smoothness into policy learning. Simulation experiments under straight-line motion, curved motion, and randomized initial conditions demonstrate that the proposed SAC-LSTM method achieves improved task completion capability and control quality compared with SAC, PPO, and DDPG under the same task settings. Compared with standard SAC, SAC-LSTM improves the average success rate by 7.5 percentage points, reduces the average episode length by approximately 22.0%, and decreases the average heading error by approximately 35.5%. Additional sequence-length analysis, reward-component ablation, and multi-level disturbance tests further validate the effectiveness of the proposed design. The results indicate that the proposed method provides an effective solution for continuous decision-making in dynamic target tracking tasks.