Fault named entity recognition based on multi-headed self-attention mechanism

Abstract

Abstract: In the process of building a knowledge graph of fault diagnosis for turbine generator set, there are some problems, such as a lack of the open named entity annotated corpus datasets, the multiple source heterogeneity of the data in the casebook, and the difficulty of extracting association weight features for specialized vocabulary. This study constructs a named entity recognition annotated corpus dataset for turbine generator set fault diagnosis based on publicly available information of turbine generator set fault cases. Additionally, a named entity recognition method that integrates multi-headed self-attention and BERT-BiLSTM-CRF is proposed. The experimental results show that the proposed method can effectively identify fault entity categories in specialized fields, which is significantly better than other traditional named entity recognition methods. Therefore, it provides a guarantee for the establishment of a fault diagnosis knowledge graph and an intelligent auxiliary decision system for the turbine generator set.

Key words: turbine generator set, fault diagnosis, named entity recognition, multi-headed self-attention mechanism, knowledge graph

CLC Number:

TH183.1

WANG Jiang, YAN Chang-feng, LU Jia-wei, WANG Rui-min, ZHANG Yong-ming. Fault named entity recognition based on multi-headed self-attention mechanism[J]. Journal of Lanzhou University of Technology, 2025, 51(4): 33-42.

References

[1] 张映锋,郭振刚,钱成,等.基于过程感知的底层制造资源智能化建模及其适自应协同优化方法研究[J].机械工程学报,2018,54(16):1-10.
[2] 周济.智能制造——“中国制造2025”的主攻方向[J].中国机械工程,2015,26(17):2273-2284.
[3] 谢永慧,刘天源,张荻.新能源形势下的“智慧汽轮机”及其研究进展[J].中国电机工程学报,2021,41(2):394-409.
[4] SINGHAL A.Official google blog:introducing the knowledge graph: things,not strings[EB/OL].(2012-05-02)[2022-06-02].http://googleblog.blogspot.pt/2015/05/introducing-knowledge-graph-things-nit.html.
[5] 张栋豪,刘振宇,郏维强,等.知识图谱在智能制造领域的研究现状及其应用前景综述[J].机械工程学报,2021,57(5):90-113.
[6] 刘峤,李杨,段宏,等.知识图谱构建技术综述[J].计算机研究与发展,2016,53(3):582-600.
[7] LI J H,WANG Z Q,WANG Y,et al.Research on distributed search technology of multiple data sources intelligent information based on knowledge graph[J].Journal of Signal Processing Systems for Signal Image and Video Technology,2021,93(2/3):239-248.
[8] 饶子昀,张毅,刘俊涛,等.应用知识图谱的推荐方法与系统[J].自动化学报,2021,47(9):2061-2077.
[9] LU X L,PRAMANIK S,ROY R S,et al.Answering complex questions by joining multi-document evidence with quasi knowledge graphs[C]//Proceedings of the 42nd International ACM SIGIR Conference.New York:ACM,2019:105-114.
[10] CHEN Y,LIAO Z F,CHEN B,et al.Construction method of knowledge base for power grid-aided decision based on knowledge graph[C]//Proceedings of the International Conference on Intelligent Computing,Communication and Devices.Berlin:Springer,2021:356-361.
[11] LIU P C,HUANG Y L,WANG P,et al.Construction of typhoon disaster knowledge graph based on graph database Neo4j[C]//Proceedings of the 2020 Chinese Control and Decision Conference.[S.l.]:IEEE,2020:3612-3616.
[12] YANG Y Y,WEI Z Y,CHEN Q,et al.Using external knowledge for financial event prediction based on graph neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York:ACM,2019:2161-2164.
[13] 何成兵,顾煜炯,邢诚.短路故障时汽轮发电机组轴系弯扭耦合振动分析[J].中国电机工程学报,2010,30(32):84-90.
[14] FU X Q,JIA W T,XU H,et al.Imbalance-misalignment-rubbing coupling faults in hydraulic turbine vibration[J].Optik,2016,127(8):3708-3712.
[15] SILVA G C,CARVALHO E E O,CAMINHAS W M.An artificial immune systems approach to case-based reasoning applied to fault detection and diagnosis[J].Expert Systems with Application,2020,140:112906.
[16] LIU R N,YANG B Y,ZIO E,et al.Artificial intelligence for fault diagnosis of rotating machinery:a review[J].Mechanical Systems and Signal Processing,2018,108:33-47.
[17] WEI Y B,WANG X,KONG Y Y,et al.A probability uncertainty method of fault classification for steam turbine generator set based on Bayes and holospectrum[J].Proceedings of the Institution of Mechanical Engineers,Part C:Journal of Mechanical Engineering Science,2016,230(20):3767-3776.
[18] TAYARANI-BATHAIE S S,VANINI Z N S,KHORASANI K.Dynamic neural network-based fault diagnosis of gas turbine engines[J].Neurocomputing,2014,125:153-165.
[19] CHEN H,LIN Z J,DING G G,et al.GRN:gated relation network to enhance convolutional neural network for named entity recognition[J].Proceedings of the AAAI Conference on Artificial Intelligence,2019,33(1):6236-6243.
[20] ALSAARAN N,ALRABIAH M.Arabic named entity recognition:a BERT-BGRU approach[J].Computers,Materials and Continua,2021,68(7):471-485.
[21] HUANG Z H,XU W,YU K.Bidirectional LSTM-CRF models for sequence tagging[EB/OL].(2015-08-09)[2022-06-02].https://doi.org/10.48550/arXiv.1508.01991.
[22] ZHANG Y,YANG J.Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2018.
[23] COLLOBERT R,WESTON J B,BOTTOU L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[24] MA X Z,HOVY E.End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2016:1064-1074.
[25] STRUBELL E,VERGA P,BELANGER D,et al.Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2017:2670-2680.
[26] GUI T,MA R T,ZHANG Q,et al.CNN-based Chinese NER with lexicon rethinking[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence.San Francisco:Margan Kaufmann,2019:4982-4988.
[27] DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg,PA:Association for Computational Linguistics,2019:4171-4186.
[28] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st Conference on Neural Information Processing Systems.New York:Curran Associates Inc,2017:6000-6010.
[29] 杨培,杨志豪,罗凌,等.基于注意机制的化学药物命名实体识别[J].计算机研究与发展,2018,55(7):1548-1556.
[30] GAO W C,ZHENG X H,ZHAO S S.Named entity recognition method of Chinese EMR based on BERT-BiLSTM-CRF[J].Journal of Physics:Conference Series,2021,1848(1):012083.
[31] 刘雨可,周申培,石英,等.面向配网一次设备缺陷文本命名实体识别研究[J].武汉理工大学学报,2022,44(10):93-101.
[32] 张磊,柴彤.大型火力发电机组故障分析[M].北京:中国电力出版社,2007.
[33] YANG J,ZHANG Y,LI L W,et al.YEDDA:a lightweight collaborative text span annotation tool[C]//Proceedings of ACL 2018,System Demonstrations.Stroudsburg,PA:Association for Computational Linguistics,2018:31-36.
[34] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[35] STRUBELL E,VERGA P,BELANGER D,et al.Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2017:2670-2680.