Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

Accurate detection of surface defects for steel is essential to improve surface quality and service life. Deep learning (DL) used in steel surface defect detection can solve the problems of low efficiency and poor accuracy of traditional manual detection. The classic YOLOv5 as a DL method is used to accomplish defect detection tasks without attention mechanisms, resulting in a loss of global information. Besides, it is difficult to complete complex network detection tasks with low-configuration hardware, especially for surface defects with complex defect types and variable defect sizes. To solve these issues, this paper introduces an improved global feature reuse and hardware-aware YOLOv5 by using BoTNet, RepGhost, and EfficientRep model (BGE-YOLOv5). The multi-head self-attention layer is used to obtain global information and only part of the convolutional layers is replaced to avoid the excessive computational cost. The RepGhost model is introduced to extract the remaining feature information for feature reuse. EfficientRep is used to replace the original structure to achieve hardware-aware and to balance the detection veracity and efficiency. The distance IOU is replaced by SCYLLA-IOU to accelerate the iteration and improve stability. The results of the framework on the surface defect database (NEU-DET) show that BGE-YOLOv5 achieves a mean average precision of 79.5%, which is 10.3% greater than the baseline. The proposed BGE-YOLOv5 has a better performance in steel surface defect detection.

References

1.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
2.
Dalal
,
N.
, and
Triggs
,
B.
,
2005
, “
Histograms of Oriented Gradients for Human Detection
,”
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
,
San Diego, CA
,
June 20–25
, pp.
886
893
.
3.
Platt
,
J.
,
1998
, “
Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines
.”
4.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2012
, “
Imagenet Classification With Deep Convolutional Neural Networks
,”
Adv. Neural Inf. Process. Syst.
,
25
(
2
), pp.
1
9
. DOI:10.1145/3065386
5.
Girshick
,
R.
,
Donahue
,
J.
,
Darrell
,
T.
, and
Malik
,
J.
,
2014
, “
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Columbus, OH
,
June 23–28
, pp.
580
587
.
6.
Girshick
,
R.
,
2015
, “
Fast R-CNN
,”
Proceedings of the IEEE International Conference on Computer Vision (CVPR)
,
Boston, MA
,
June 7–12
, pp.
1440
1448
.
7.
Ren
,
S.
,
He
,
K.
,
Girshick
,
R.
, and
Sun
,
J.
,
2015
, “
Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks
,”
Adv. Neural Inf. Process. Syst.
,
28
, pp.
1
9
. DOI:10.1109/TPAMI.2016.2577031
8.
He
,
K.
,
Gkioxari
,
G.
,
Dollár
,
P.
, and
Girshick
,
R.
,
2017
, “
Mask R-CNN
,”
Proceedings of the IEEE International Conference on Computer Vision (CVPR)
,
Venice, Italy
,
Oct. 22–29
, pp.
2961
2969
.
9.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2014
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
Computer Science
. DOI:10.48550
10.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
June 27–30
, pp.
770
778
.
11.
Redmon
,
J.
,
Divvala
,
S.
,
Girshick
,
R.
, and
Farhadi
,
A.
,
2016
, “
You Only Look Once: Unified, Real-Time Object Detection
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (CVPR)
,
Las Vegas, NV
,
June 27–30
, pp.
779
788
.
12.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
, and
Polosukhin
,
I.
,
2017
, “
Attention Is All You Need
,”
Proceedings of the 31st International Conference on Neural Information Processing Systems(NIPS)
,
Long Beach, CA
,
Dec. 4–9
.
13.
Dosovitskiy
,
A.
,
Beyer
,
L.
,
Kolesnikov
,
A.
,
Weissenborn
,
D.
,
Zhai
,
X.
,
Unterthiner
,
T.
,
Dehghani
,
M.
,
Minderer
,
M.
,
Heigold
,
G.
, and
Gelly
,
S.
,
2020
, “
An Image is Worth 16 (16 Words: Transformers for Image Recognition at Scale
,”
Computer Science
. arxiv:2010.11929, 2020.
14.
Carion
,
N.
,
Massa
,
F.
,
Synnaeve
,
G.
,
Usunier
,
N.
,
Kirillov
,
A.
, and
Zagoruyko
,
S.
,
2020
, “
End-to-End Object Detection With Transformers
,”
European Conference on Computer Vision (ECCV)
,
Glasgow, UK
,
Aug. 23–28
.
15.
Chen
,
H.
,
Nie
,
Z.
,
Xu
,
Q.
,
Fei
,
J.
,
Yang
,
K.
,
Li
,
Y.
,
Lin
,
H.
,
Fan
,
W.
, and
Liu
,
X. J.
,
2023
, “
Intelligent Detection and Classification of Surface Defects on Cold-Rolled Galvanized Steel Strips Using a Data-Driven Faulty Model With Attention Mechanism
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
4
), p.
041001
.
16.
Wang
,
L.
,
Liu
,
X.
,
Ma
,
J.
,
Su
,
W.
, and
Li
,
H.
,
2023
, “
Real-Time Steel Surface Defect Detection With Improved Multi-scale YOLO-v5
,”
Processes
,
11
(
5
), p.
1357
.
17.
Liu
,
B.
,
Gao
,
F.
, and
Li
,
Y.
,
2023
, “
Cost-Sensitive YOLOv5 for Detecting Surface Defects of Industrial Products
,”
Sensors
,
23
(
5
), p.
2610
.
18.
Hong
,
X.
,
Wang
,
F.
, and
Ma
,
J.
,
2022
, “
Improved YOLOv7 Model for Insulator Surface Defect Detection
,”
2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC)
,
Chongqing, China
,
Dec. 16–18
, pp.
1667
1672
.
19.
Srinivas
,
A.
,
Lin
,
T. Y.
,
Parmar
,
N.
,
Shlens
,
J.
,
Abbeel
,
P.
, and
Vaswani
,
A.
,
2021
, “
Bottleneck Transformers for Visual Recognition
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Nashville, TN
,
June 19–25
, pp.
16519
16529
.
20.
Song
,
K.
, and
Yan
,
Y.
,
2013
, “
A Noise Robust Method Based on Completed Local Binary Patterns for Hot-Rolled Steel Strip Surface Defects
,”
Appl. Surf. Sci
,
285
(
21
), pp.
858
864
.
21.
Chen
,
C.
,
Guo
,
Z.
,
Zeng
,
H.
,
Xiong
,
P.
, and
Dong
,
J.
,
2022
, “
Repghost: A Hardware-Efficient Ghost Module Via Re-Parameterization
,”
Computer Science
. arxiv:2211.06088, 2022
22.
Han
,
K.
,
Wang
,
Y.
,
Tian
,
Q.
,
Guo
,
J.
,
Xu
,
C.
, and
Xu
,
C.
,
2020
, “
GhostNet: More Features From Cheap Operations
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Seattle, WA
,
June 14–19
, pp.
1577
1586
.
23.
Goodfellow
,
I.
,
Warde-Farley
,
D.
,
Mirza
,
M.
,
Courville
,
A.
, and
Bengio
,
Y.
,
2013
, “
Maxout Networks
,”
International Conference on Machine Learning
,
Atlanta, GA
,
June 16–21
.
24.
Ding
,
X.
,
Zhang
,
X.
,
Ma
,
N.
,
Han
,
J.
,
Ding
,
G.
, and
Sun
,
J.
,
2021
, “
REPVGG: Making VGG-Style Convnets Great Again
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
June 19–25
, pp.
13733
13742
.
25.
Weng
,
K.
,
Chu
,
X.
,
Xu
,
X.
,
Huang
,
J.
, and
Wei
,
X.
,
2023
, “
EfficientRep: An Efficient Repvgg-Style ConvNets With Hardware-Aware Neural Network Design
,”
Computer Science
. arxiv:2302.00386
26.
Rezatofighi
,
H.
,
Tsoi
,
N.
,
Gwak
,
J.
,
Sadeghian
,
A.
,
Reid
,
I.
, and
Savarese
,
S.
,
2019
, “
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Long Beach, CA
,
June 16–20
, pp.
658
666
.
27.
Zheng
,
Z.
,
Wang
,
P.
,
Liu
,
W.
,
Li
,
J.
,
Ye
,
R.
, and
Ren
,
D.
,
2020
, “
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
,”
Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)
,
New York
,
Feb. 7–12
, pp.
12993
13000
.
28.
Gevorgyan
,
Z.
,
2022
, “
SIoU Loss: More Powerful Learning for Bounding Box Regression
,”
Computer Science
.
29.
Düntsch
,
I.
, and
Gediga
,
G.
,
2019
, “
Confusion Matrices and Rough Set Data Analysis
,”
J. Phys. Conf. Ser.
,
1229
(
1
), p.
012055
.
30.
Liu
,
W.
,
Anguelov
,
D.
,
Erhan
,
D.
,
Szegedy
,
C.
,
Reed
,
S.
,
Fu
,
C.-Y.
, and
Berg
,
A. C.
,
2016
, “
SSD: Single Shot Multibox Detector
,”
Computer Vision–ECCV 2016: 14th European Conference (ECCV)
,
Amsterdam, The Netherlands
,
Oct. 10–16
,
Part I 14
.
31.
Tan
,
M.
,
Pang
,
R.
, and
Le
,
Q. V.
,
2020
, “
Efficientdet: Scalable and Efficient Object Detection
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
,
Seattle, WA
,
June 14–19
, pp.
10781
10790
.
You do not currently have access to this content.