Research on segmentation and edge refinement of melanoma image

Abstract

Abstract: Segmentation of melanoma image has important clinical value in the diagnosis and treatment of skin diseases. However, due to the lack of annotation and serious category imbalance of data set, the accuracy of the segmentation map is low and the edge is rough. Therefore, a melanoma segmentation and edge refinement method is proposed, which can be implemented in three stages. In the first stage, after classifying the melanoma data set, multiple hierarchical class activation maps are extracted and processed by weakly supervised method. In the second stage, a UNet model is built to segment malignant samples in melanoma data set. In the third stage, the edge refinement of the results of the second stage is carried out by using the enhanced and overlaid class activation map obtained in the first stage. Experimental results on the ISIC melanoma dataset show that after three stages of processing, the segmentation map has finer edges, with a mean intersection over union value of 86.04%, a Dice similarity coefficient of 0.937, and a Jaccard similarity coefficient of 0.885.

Key words: image classification, medical image segmentation, weakly supervised learning, melanoma, image edge refinement

CLC Number:

TP389.1

ZHAO Hong, WANG Ao-long, ZHANG Chen-peng. Research on segmentation and edge refinement of melanoma image[J]. Journal of Lanzhou University of Technology, 2024, 50(4): 77-85.

References

[1] BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:a deep convolutional encoder-decoder architecture for image segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[2] ALTAF F,ISLAM S M S,AKHTAR N,et al.Going deep in medical image analysis:concepts,methods,challenges,and future directions [J].IEEE Access,2019,7:99540-99572.
[3] RONNEBERGER O,FISCHER P,BROX T.U-net:convolutional networks for biomedical image segmentation [C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2015,9351:234-241.
[4] ZHOU Z,SIDDIQUEE M M R,TAJBAKHSH N,et al.Unet++:a nested U-net architecture for medical image segmentation [C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support.Cham:Springer,2018:11045.
[5] OKTAY O,SCHLEMPER J,FOLGOC L L,et al.Attention U-net:learning where to look for the pancreas [DB/OL].(2018-04-11)[2022-04-25].https://arxiv.org/abs/1804.03999.
[6] ALOM M Z,HASAN M,YAKOPCIC C,et al.Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation [DB/OL].(2018-01-20)[2022-04-25].https://arxiv.org/abs/1802.06955.
[7] LI X,CHEN H,QI X,et al.H-DenseUNet:hybrid densely connected UNet for liver and tumor segmentation from CT volumes [J].IEEE Transactions on Medical Imaging,2018,37(12):2663-2674.
[8] TAJBAKHSH N,JEYASEELAN L,LI Q,et al.Embracing imperfect datasets:a review of deep learning solutions for medical image segmentation [J].Medical Image Analysis,2020,63:101693.
[9] 张粲,孟雪,王常青,等.基于对抗学习网络的半监督医学图像分割方法 [J].工业控制计算机,2021,34(9):57-59.
[10] YUAN Y.Automatic skin lesion segmentation with fully convolutional-deconvolutional networks [DB/OL].(2017-05-15)[2022-04-25].https://arxiv.org/abs/1703.05165.
[11] ZHANG X.Melanoma segmentation based on deep learning [J].Computer Assisted Surgery,2017,22(S1):267-277.
[12] 杨国亮,赖振东,喻丁玲.一种改进UNet++网络用于检测黑色素瘤皮肤病变 [J].中国医学影像技术,2020,36(12):1877-1881.
[13] AL-MASNI M A,AL-ANTARI M A,CHOI M T,et al.Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks [J].Computer Methods and Programs in Biomedicine,2018,162:221-231.
[14] XIE F,YANG J,LIU J,et al.Skin lesion segmentation using high-resolution convolutional neural network [J].Computer Methods and Programs in Biomedicine,2020,186:105241.
[15] ATTIA M,HOSSNY M,NAHAVANDI S,et al.Skin melanoma segmentation using recurrent and convolutional neural networks [C]//2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).Melbourne:IEEE,2017:292-296.
[16] GORRIZ M,CARLIER A,FAURE E,et al.Cost-effective active learning for melanoma segmentation [DB/OL].(2017-11-24)[2022-04-25].https://arxiv.org/abs/1711.09168.
[17] ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deep features for discriminative localization [DB/OL].(2016-06-27)[2022-04-25].https://openaccess.thecvf.com/content_cvpr_2016/papers/Zhou_Learning_Deep_Features_CVPR_2016_paper.pdf.
[18] SELVARAJU R R,CONGSWELL M,DAS A,et al.Grad-cam:visual explanations from deep networks via gradient-based localization [DB/OL].(2017-10-22) [2022-04-25].https://arxiv.org/abs/1610.02391.
[19] JIANG P T,ZHANG C B,HOU Q,et al.LayerCAM:exploring hierarchical class activation maps for localization [J].IEEE Transactions on Image Processing,2021,30:5875-5888.
[20] FENG X,YANG J,LAINE A F,et al.Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules [C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2017:568-576.
[21] GARCIA-GARCIA A,ORTS-ESCOLANO S,OPREA S,et al.A review on deep learning techniques applied to semantic segmentation [DB/OL].(2017-04-22) [2022-04-25].https://arxiv.org/abs/1704.06857.
[22] ISIC melanoma dataset [EB/OL].[2020-05-28].https://www.isic-archive.com/#!/onlyHeaderTop/gallery.
[23] ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[DB/OL].(2017-07-21)[2022-04-25].https://arxiv.org/abs/1612.01105.
[24] MIRIKHARAJI Z,HAMARNEH G.Star shape prior in fully convolutional networks for skin lesion segmentation [C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer,2018:737-745.
[25] JÉGOU S,DROZDZAL M,VAZQUEZ D,et al.The one hundred layers tiramisu:fully convolutional DenseNets for semantic segmentation [DB/OL].(2017-07-21) [2022-04-25].https://arxiv.org/abs/1611.09326.
[26] ÖZTÜRK Ş,ÖZKAYA U.Skin lesion segmentation with improved convolutional neural network [J].Journal of Digital Imaging,2020,33(4):958-970.