Abstract: In order to realize the intelligent detection of buoy lights at night, this paper proposes a video-based buoy lights patten detection network.In the network, a detection module is used to capture the color features of buoy lights, then a tracking module is used to track the detected color features and lock light targets, and a classification module is used to further identify the flashing sequence of targets, finally a target′s color and flashing sequence are combined into the detection result of buoy lights by a binary correlation method. The detection module is developed based on YOLOv5, with improvements of its backbone network and loss function（SIoU）, realizing light targets′ color detection for video single-frame images. The tracking module is developed based on DeepSort, and Kalman filter and Hungarian algorithm are combined to realize stable tracking of light targets appeared at interval. The classification module uses a simple network structure to quickly identify the flashing frequency of buoy light. 3 500 min videos are used to train and test the network module. The experimental results show that the detection accuracy of the proposed module for nighttime buoy light reaches 90%, and the detection speed also meets real-time requirements.