Deformable partition attention based melanoma recognition method for multimodal skin disease corpus

Abstract

Abstract: For the problem of melanoma image diagnosis, this paper proposes a recognition method based on a deformable partition attention mechanism. This method adopts a coarse-to-fine feature extraction and recognition strategy to accurately distinguish melanoma from common moles and establish corresponding semantic labels. Based on this, a multimodal dermatological corpus is constructed by integrating case texts. First, to tackle issues such as blurred lesion boundaries, uneven distribution, and difficulty in feature extraction, this paper introduces a deformable partition attention module that combines attention mechanisms with deformable convolutions. Second, to address the large differences between benign and malignant subcategories that result in training difficulties and low recognition efficiency, this paper constructs a hierarchical learning framework that progresses from coarse to fine categories. In addition, a joint loss function is introduced to optimize the model’s recognition accuracy. Experimental results show that the proposed algorithm demonstrates high sensitivity and specificity on self-constructed dataset, effectively improving the accuracy of multimodal dermatological corpus construction by matching case texts with medical images.

Key words: medical image processing, melanoma recognition, deformable convolution, attention mechanism, deep learning, multimodal corpus

CLC Number:

TP391

LIN Yu-ping, LIU Meng-jiao, WANG Ming-hao, ZHANG Dong, XU Mei-feng, LI Ce. Deformable partition attention based melanoma recognition method for multimodal skin disease corpus[J]. Journal of Lanzhou University of Technology, 2025, 51(5): 92-99.

References

[1] BI L,FENG D D,FULHAM M,et al.Multi-label classification of multi-modality skin lesion via hyper-connected convolutional neural network [J].Pattern Recognition,2020,107:107502.
[2] WANG X,JIANG X,DING H,et al.Knowledge-aware deep framework for collaborative skin lesion segmentation and melanoma recognition [J].Pattern Recognition,2021,120:108075.
[3] BASAK H,KUNDU R,SARKAR R.MFSNet:a multi focus segmentation network for skin lesion segmentation [J].Pattern Recognition,2022,128:108673.
[4] GOODSON A G,GROSSMAN D.Strategies for early melanoma detection:approaches to the patient with nevi [J].Journal of the American Academy of Dermatology,2009,60(5):719-735.
[5] WANG Q,SUN L,WANG Y,et al.Identification of melanoma from hyperspectral pathology image using 3D convolutional networks [J].IEEE Transactions on Medical Imaging,2020,40(1):218-227.
[6] GOCERI E.CapsNet topology to classify tumours from brain images and comparative evaluation [J].IET Image Processing,2020,14(5):882-889.
[7] WU H,PAN J,LI Z,et al.Automated skin lesion segmentation via an adaptive dual attention module [J].IEEE Transactions on Medical Imaging,2020,40(1):357-370.
[8] LI Y,SHEN L.Skin lesion analysis towards melanoma detection using deep learning network [J].Sensors,2018,18(2):556-572.
[9] YAMAMOTO Y,TSUZUKI T,AKATSUKA J,et al.Automated acquisition of explainable knowledge from unannotated histopathology images [J].Nature Communications,2019,10(1):5642-5651.
[10] QU X,SHI Y,HOU Y,et al.An attention-supervised full-resolution residual network for the segmentation of breast ultrasound images [J].Medical Physics,2020,47(11):5702-5714.
[11] TANG Z,CHUANG K V,DECARLI C,et al.Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline [J].Nature Communications,2019,10(1):2173-2187.
[12] TOLKACH Y,DOHMGRGEN T,TOMA M,et al.High-accuracy prostate cancer pathology using deep learning [J].Nature Machine Intelligence,2020,2(7):411-418.
[13] JAIN M S,MASSOUD T F.Predicting tumour mutational burden from histopathological images using multiscale deep learning [J].Nature Machine Intelligence,2020,2(6):356-362.
[14] ZHANG D,HAN H,DU S,et al.MPMR:multi-scale feature and probability map for melanoma recognition [J].Frontiers in Medicine,2022,8:775587.
[15] GOCERI E.Analysis of deep networks with residual blocks and different activation functions:classification of skin diseases [C]//2019 Ninth International Conference on Image Processing Theory,Tools and Applications (IPTA).Istanbul,Turkey:IEEE,2019:1-6.
[16] GUO X,CHEN Z,YUAN Y.Complementary network with adaptive receptive fields for melanoma segmentation [C]//2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).Iowa City,USA:IEEE,2020:2010-2013.
[17] YADA S,NAKAMURA Y,WAKAMIYA S,et al.Real-MedNLP:overview of real document-based medical natural language processing task [C]//Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies.Tokyo,Japan:National Institute of Informatics,2022:285-296.
[18] GU Y,TINN R,CHENG H,et al.Domain-specific language model pretraining for biomedical natural language processing [J].ACM Transactions on Computing for Healthcare(HEALTH),2021,3(1):1-23.
[19] LAMBERT B,FORBES F,DOYLE S,et al.Trustworthy clinical AI solutions:a unified review of uncertainty quantification in deep learning models for medical image analysis [J].Artificial Intelligence in Medicine,2024,150:102830.
[20] ZENG J,BANERJEE I,HENRY A S,et al.Natural language processing to identify cancer treatments with electronic medical records [J].JCO Clinical Cancer Informatics,2021,5:379-393.
[21] GONG L,ZHANG Z,CHEN S.Clinical named entity recognition from Chinese electronic medical records based on deep learning pretraining [J].Journal of Healthcare Engineering,2020(1):8829219.
[22] HU J,SHEN L,SUN G.Squeeze-and-excitation networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake,USA:IEEE Computer Society,2018:7132-7141.
[23] LI X,WANG W,HU X,et al.Selective kernel networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:510-519.
[24] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition [C/OL].[2024-12-10].https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7780459.
[25] XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE Computer Society,2017:1492-1500.
[26] ZHANG H,WU C,ZHANG Z,et al.ResNeSt:split-attention networks [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:2736-2746.
[27] GOUTTE C,GAUSSIER E.A probabilistic interpretation of precision,recall and F-score,with implication for evaluation [C]//European Conference on Information Retrieval.Berlin,Heidelberg:Springer,2005:345-359.
[28] ZUO W,ZHOU F,HE Y,et al.Automatic classification of lung nodule candidates based on a novel 3D convolution network and knowledge transferred from a 2D network [J].Medical Physics,2019,46(12):5499-5513.
[29] BRADLEY A P.The use of the area under the ROC curve in the evaluation of machine learning algorithms [J].Pattern Recognition,1997,30(7):1145-1159.