Trajectory planning and control for unmanned surface vehicle based on deep reinforcement learning

This study aims to apply deep reinforcement learning to address the challenges of trajectory planning and control for unmanned surface vehicles. In trajectory planning, the Q-learning algorithm is employed to generate trajectories in realworld aquatic environments. For the design of the reward function, factors such as shallow water areas are taken into account, with an emphasis on minimizing the number of turning points along the path. For trajectory tracking control, we integrate the Soft Actor-Critic（SAC） algorithm with the Proportional-Integral-Derivative（PID） control method to alleviate the difficulties of manual parameter tuning associated with conventional PID controllers. This hybrid approach also mitigates the interpretability limitations often found in pure deep reinforcement learning methods. Comparative experiments involving the traditional PID algorithm, Genetic Algorithm（GA）, and Deep Deterministic Policy Gradient（DDPG） algorithm demonstrate the superiority of the proposed SAC-PID method. Simulation results show that the planned trajectories effectively incorporate multiple factors, including travel distance, shallow water regions, and number of turning point, the SAC-PID method achieves outstanding performance in trajectory tracking.