面向多模态皮肤病语料库的可变形分区注意力黑色素瘤识别方法

兰州理工大学学报 ›› 2025, Vol. 51 ›› Issue (5): 92-99.

• 自动化技术与计算机技术 • 上一篇下一篇

面向多模态皮肤病语料库的可变形分区注意力黑色素瘤识别方法

林玉萍¹, 刘梦皎², 王明豪², 张栋², 许美凤^*3, 李策⁴

1.西安交通大学外国语学院, 陕西西安 710049;
2.西安交通大学人工智能学院, 陕西西安 710049;
3.西安交通大学第二附属医院皮肤科, 陕西西安 710004;
4.兰州理工大学自动化与电气工程学院, 甘肃兰州 730050

收稿日期:2025-03-14 发布日期:2025-10-25
通讯作者: 许美凤(1977-),女,福建漳州人,副主任医师.Email:xumf96@163.com
基金资助:
国家自然科学基金(62363025),陕西省社会科学基金(2021K014)

Deformable partition attention based melanoma recognition method for multimodal skin disease corpus

LIN Yu-ping¹, LIU Meng-jiao², WANG Ming-hao², ZHANG Dong², XU Mei-feng³, LI Ce⁴

1. School of Foreign Studies,Xi’an Jiaotong University, Xi’an 710049, China;
2. College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an 710049, China;
3. Department of Dermatology, Second Afflicated Hospital of Xi’an Jiaotong University, Xi’an 710004, China;
4. School of Automation and Electrical Engineering, Lanzhou University of Technology, Lanzhou 730050, China

Received:2025-03-14 Published:2025-10-25

摘要/Abstract

摘要： 针对黑色素瘤图像诊断问题,提出一种基于可变形分区注意力机制的黑色素瘤识别方法.该方法采用由粗到细的特征提取与识别策略准确区分黑色素瘤和普通痣并建立相应语义标签,在此基础上结合病例文本构建多模态皮肤病语料库.首先,为解决良性与恶性子类别间差异过大导致模型训练困难及识别效率低的问题,构建了一个从粗类到细类层级深入的学习架构;其次,针对病灶边缘模糊、分布不均以及特征提取难的问题,提出了一种融合注意力机制与可变形卷积的可变形分区注意力模块,通过由粗到细的特征提取策略实现了全局与局部特征的有效结合;此外,引入了联合损失函数优化模型识别精准性.实验结果表明,该算法在自建数据集上展现了高敏感性和高特异性,有效提升了病例文本和医学影像匹配构建多模态皮肤病语料库的准确性.

关键词: 医学图像处理, 黑色素瘤识别, 可变形卷积, 注意力机制, 深度学习, 多模态语料库

Abstract: For the problem of melanoma image diagnosis, this paper proposes a recognition method based on a deformable partition attention mechanism. This method adopts a coarse-to-fine feature extraction and recognition strategy to accurately distinguish melanoma from common moles and establish corresponding semantic labels. Based on this, a multimodal dermatological corpus is constructed by integrating case texts. First, to tackle issues such as blurred lesion boundaries, uneven distribution, and difficulty in feature extraction, this paper introduces a deformable partition attention module that combines attention mechanisms with deformable convolutions. Second, to address the large differences between benign and malignant subcategories that result in training difficulties and low recognition efficiency, this paper constructs a hierarchical learning framework that progresses from coarse to fine categories. In addition, a joint loss function is introduced to optimize the model’s recognition accuracy. Experimental results show that the proposed algorithm demonstrates high sensitivity and specificity on self-constructed dataset, effectively improving the accuracy of multimodal dermatological corpus construction by matching case texts with medical images.

Key words: medical image processing, melanoma recognition, deformable convolution, attention mechanism, deep learning, multimodal corpus

中图分类号:

TP391

林玉萍, 刘梦皎, 王明豪, 张栋, 许美凤, 李策. 面向多模态皮肤病语料库的可变形分区注意力黑色素瘤识别方法[J]. 兰州理工大学学报, 2025, 51(5): 92-99.

LIN Yu-ping, LIU Meng-jiao, WANG Ming-hao, ZHANG Dong, XU Mei-feng, LI Ce. Deformable partition attention based melanoma recognition method for multimodal skin disease corpus[J]. Journal of Lanzhou University of Technology, 2025, 51(5): 92-99.

参考文献

[1] BI L,FENG D D,FULHAM M,et al.Multi-label classification of multi-modality skin lesion via hyper-connected convolutional neural network [J].Pattern Recognition,2020,107:107502.
[2] WANG X,JIANG X,DING H,et al.Knowledge-aware deep framework for collaborative skin lesion segmentation and melanoma recognition [J].Pattern Recognition,2021,120:108075.
[3] BASAK H,KUNDU R,SARKAR R.MFSNet:a multi focus segmentation network for skin lesion segmentation [J].Pattern Recognition,2022,128:108673.
[4] GOODSON A G,GROSSMAN D.Strategies for early melanoma detection:approaches to the patient with nevi [J].Journal of the American Academy of Dermatology,2009,60(5):719-735.
[5] WANG Q,SUN L,WANG Y,et al.Identification of melanoma from hyperspectral pathology image using 3D convolutional networks [J].IEEE Transactions on Medical Imaging,2020,40(1):218-227.
[6] GOCERI E.CapsNet topology to classify tumours from brain images and comparative evaluation [J].IET Image Processing,2020,14(5):882-889.
[7] WU H,PAN J,LI Z,et al.Automated skin lesion segmentation via an adaptive dual attention module [J].IEEE Transactions on Medical Imaging,2020,40(1):357-370.
[8] LI Y,SHEN L.Skin lesion analysis towards melanoma detection using deep learning network [J].Sensors,2018,18(2):556-572.
[9] YAMAMOTO Y,TSUZUKI T,AKATSUKA J,et al.Automated acquisition of explainable knowledge from unannotated histopathology images [J].Nature Communications,2019,10(1):5642-5651.
[10] QU X,SHI Y,HOU Y,et al.An attention-supervised full-resolution residual network for the segmentation of breast ultrasound images [J].Medical Physics,2020,47(11):5702-5714.
[11] TANG Z,CHUANG K V,DECARLI C,et al.Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline [J].Nature Communications,2019,10(1):2173-2187.
[12] TOLKACH Y,DOHMGRGEN T,TOMA M,et al.High-accuracy prostate cancer pathology using deep learning [J].Nature Machine Intelligence,2020,2(7):411-418.
[13] JAIN M S,MASSOUD T F.Predicting tumour mutational burden from histopathological images using multiscale deep learning [J].Nature Machine Intelligence,2020,2(6):356-362.
[14] ZHANG D,HAN H,DU S,et al.MPMR:multi-scale feature and probability map for melanoma recognition [J].Frontiers in Medicine,2022,8:775587.
[15] GOCERI E.Analysis of deep networks with residual blocks and different activation functions:classification of skin diseases [C]//2019 Ninth International Conference on Image Processing Theory,Tools and Applications (IPTA).Istanbul,Turkey:IEEE,2019:1-6.
[16] GUO X,CHEN Z,YUAN Y.Complementary network with adaptive receptive fields for melanoma segmentation [C]//2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).Iowa City,USA:IEEE,2020:2010-2013.
[17] YADA S,NAKAMURA Y,WAKAMIYA S,et al.Real-MedNLP:overview of real document-based medical natural language processing task [C]//Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies.Tokyo,Japan:National Institute of Informatics,2022:285-296.
[18] GU Y,TINN R,CHENG H,et al.Domain-specific language model pretraining for biomedical natural language processing [J].ACM Transactions on Computing for Healthcare(HEALTH),2021,3(1):1-23.
[19] LAMBERT B,FORBES F,DOYLE S,et al.Trustworthy clinical AI solutions:a unified review of uncertainty quantification in deep learning models for medical image analysis [J].Artificial Intelligence in Medicine,2024,150:102830.
[20] ZENG J,BANERJEE I,HENRY A S,et al.Natural language processing to identify cancer treatments with electronic medical records [J].JCO Clinical Cancer Informatics,2021,5:379-393.
[21] GONG L,ZHANG Z,CHEN S.Clinical named entity recognition from Chinese electronic medical records based on deep learning pretraining [J].Journal of Healthcare Engineering,2020(1):8829219.
[22] HU J,SHEN L,SUN G.Squeeze-and-excitation networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake,USA:IEEE Computer Society,2018:7132-7141.
[23] LI X,WANG W,HU X,et al.Selective kernel networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:510-519.
[24] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition [C/OL].[2024-12-10].https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7780459.
[25] XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE Computer Society,2017:1492-1500.
[26] ZHANG H,WU C,ZHANG Z,et al.ResNeSt:split-attention networks [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:2736-2746.
[27] GOUTTE C,GAUSSIER E.A probabilistic interpretation of precision,recall and F-score,with implication for evaluation [C]//European Conference on Information Retrieval.Berlin,Heidelberg:Springer,2005:345-359.
[28] ZUO W,ZHOU F,HE Y,et al.Automatic classification of lung nodule candidates based on a novel 3D convolution network and knowledge transferred from a 2D network [J].Medical Physics,2019,46(12):5499-5513.
[29] BRADLEY A P.The use of the area under the ROC curve in the evaluation of machine learning algorithms [J].Pattern Recognition,1997,30(7):1145-1159.

面向多模态皮肤病语料库的可变形分区注意力黑色素瘤识别方法

Deformable partition attention based melanoma recognition method for multimodal skin disease corpus

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 13

编辑推荐

Metrics

本文评价

[1]	张玺君, 苏晋, 陈宣, 尚继洋, 崔勇. 基于Attention-T-GRU的短时交通流预测[J]. 兰州理工大学学报, 2025, 51(4): 107-113.
[2]	陈辉, 胡荣海, 席磊. 基于长短期记忆神经网络局部认知结构的最优非线性卡尔曼滤波[J]. 兰州理工大学学报, 2025, 51(4): 88-94.
[3]	王江, 剡昌锋, 卢家伟, 王瑞民, 张永明. 融合多头自注意力机制的故障命名实体识别[J]. 兰州理工大学学报, 2025, 51(4): 33-42.
[4]	赵宏, 张陈鹏, 王奡隆, 张扬. 基于细粒度字词特征的中文作者识别模型[J]. 兰州理工大学学报, 2025, 51(3): 99-106.
[5]	杜先君, 王紫阳. 电动汽车IGBT剩余使用寿命预测[J]. 兰州理工大学学报, 2024, 50(2): 77-86.
[6]	陈辉, 牛丽丽, 付辉, 张天佑, 席磊. 基于多尺度残差和注意力机制的图像去雾算法[J]. 兰州理工大学学报, 2024, 50(2): 69-76.
[7]	杜先君, 邱小彧. 基于改进LSTM神经网络的化工过程故障诊断[J]. 兰州理工大学学报, 2023, 49(6): 72-79.
[8]	王文婷, 刘姝君, 张耀聪, 杜小泽, 许潼. 基于Seq2Seq深度学习模型的焦炉煤气发生量预测方法研究[J]. 兰州理工大学学报, 2023, 49(5): 50-58.
[9]	王星, 晏榕璟. 深度可分卷积结合多通道注意力的垃圾图像快速分类模型[J]. 兰州理工大学学报, 2023, 49(3): 88-93.
[10]	王燕, 赵妮妮, 范林. 基于图神经网络特征交叉的协同过滤算法[J]. 兰州理工大学学报, 2023, 49(1): 94-102.
[11]	郭润兰, 史方青, 范雅琼, 何智. 基于卷积神经网络的壁面清洗机器人障碍物检测识别算法[J]. 兰州理工大学学报, 2022, 48(4): 83-89.
[12]	赵宏, 傅兆阳, 王乐. 基于特征融合的中文文本情感分析方法[J]. 兰州理工大学学报, 2022, 48(3): 94-102.
[13]	刘金霞. 基于深度学习的无线电信号分类[J]. 兰州理工大学学报, 2021, 47(4): 106-110.