In this paper, we utilize the features of YOLOv3 combined with multi-size prediction of feature pyramid network, which fuses the feature information of multiple feature maps of different sizes through up-sampling from top to bottom to improve the resolution of feature maps of different sizes. The converged target classification loss function and target regression loss function are used to train the YOLOv3 algorithm in combination with the homemade defect mapping dataset to meet the design requirements of the surface defect detection algorithm. Finetuning is applied as a pre-training model to optimize the loss function of the improved YOLOv3 algorithm. Analyze the experimental performance of the improved YOLOv3 algorithm with different Image Size parameters, different numbers of Anchors, and different sizes of defect areas. Compare the index performance of the improved YOLOv3 algorithm proposed in this paper with the YOLO series algorithms. The detection precision and recall of the improved YOLOv3 algorithm for different defective regions are 0.9324 and 0.8589, respectively, and the improved algorithm meets the requirements of defect detection algorithm design.