基于超复数小波和图像空域的卷积网络融合注视点预测算法

兰州理工大学学报 ›› 2021, Vol. 47 ›› Issue (5): 76-84.

• 自动化技术与计算机技术 • 上一篇下一篇

基于超复数小波和图像空域的卷积网络融合注视点预测算法

李策^*, 朱子重, 许大有, 高伟哲, 靳山岗

兰州理工大学电气工程与信息工程学院, 甘肃兰州 730050

收稿日期:2020-01-10 出版日期:2021-10-28 发布日期:2021-11-18
作者简介:李策(1974-),男,辽宁营口人,博士,教授,博导.Email:xjtulice@qmail.com
基金资助:
国家自然科学基金(61866022),甘肃省基础研究创新群体项目(1506RJIA031),国防基础科研项目(JCKY2018427C002)

Gaze prediction algorithm based on hypercomplex wavelet convolutional network

LI Ce, ZHU Zi-zhong, XU Da-you, GAO Wei-zhe, JIN Shan-gang

College of Electrical and Information Engineering, Lanzhou Univ. of Tech., Lanzhou 730050, China

Received:2020-01-10 Online:2021-10-28 Published:2021-11-18

摘要/Abstract

摘要： 针对已有注视点预测模型存在特征细节缺失、尺度单一和背景信息干扰严重导致的注视点预测精度偏低等问题,提出了一种基于超复数小波和图像空域的卷积网络融合注视点预测算法.首先,针对细节特征丢失问题,使用超复数小波变换在频域中提取图像的细节特征,与卷积网络提取的空域特征进行融合.然后,通过空洞空间金字塔池化模块,融合不同感受得到的特征图,有效解决了特征尺度单一的问题.最后,引入了残差卷积注意力模块,结合空间和通道的注意力机制,能够有效抑制背景信息的干扰,提高注视点预测精度.在SALICON数据集上,CC、sAUC和SIM评价指标下,该算法的性能达到0.884 7、0.769 3和0.778 0;在CAT2000数据集上,该算法在相应指标下的性能为0.735 5、0.870 1和0.664 5.主客观对比实验结果表明,该算法具有较好的注视点预测能力.

关键词: 注视点预测, 超复数小波变换, 空域特征, 卷积网络

Abstract: Gaze based prediction algorithms has a wide range of applications in object recognition, video compression, object tracking and so on. For existing gaze prediction models, the accuracy of gaze prediction is low due to the lack of feature details, single scale, and serious background information interference. This paper proposes a gaze prediction algorithm based on hypercomplex wavelet convolutional network. Firstly, aiming at the problem of loss of detailed features, the hypercomplex wavelet transform is used to extract the detailed features of the image in the frequency domain and fused with the spatial features extracted by the convolutional network. Then, through the atrous spatial pyramid pooling module, the feature maps obtained from different receptive fields are fused to effectively solve the problem of single feature scale. Finally, the proposed algorithm introduces a residual convolutional attention module, which combines spatial and channel attention mechanisms to effectively suppress the interference of background information and improve the accuracy of gaze prediction. On the SALICON datasets, CC, sAUC and SIM evaluation metrics, the performance of the proposed algorithm reaches 0.884 7, 0.769 3 and 0.778 0. On the CAT2000 datasets, the performance of the proposed algorithm is 0.735 5, 0.870 1, and 0.664 5. The experimental results show that the proposed algorithm has a good ability to predict fixation points.

Key words: gaze prediction, hypercomplex wavelet transform, spatial features, convolutional neural network

中图分类号:

TP393

李策, 朱子重, 许大有, 高伟哲, 靳山岗. 基于超复数小波和图像空域的卷积网络融合注视点预测算法[J]. 兰州理工大学学报, 2021, 47(5): 76-84.

LI Ce, ZHU Zi-zhong, XU Da-you, GAO Wei-zhe, JIN Shan-gang. Gaze prediction algorithm based on hypercomplex wavelet convolutional network[J]. Journal of Lanzhou University of Technology, 2021, 47(5): 76-84.

参考文献

[1] 姜波.基于卷积神经网络的注视点检测 [D].大连:大连理工大学,2017.
[2] ITTI L,KOCH C,NIEBUR E.A model of saliency-based visual attention for rapid scene analysis [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[3] HOU X,ZHANG L.Saliency detection:a spectral residual approach [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis,MN,USA:IEEE,2007:1-8.
[4] TORRALBA A,OLIVA A,CASTELHANO M S,et al.Contextual guidance of eye movements and attention in real-world scenes:the role of global features in object search [J].Psychological Review,2006,113(4):766-786.
[5] GUO C,MA Q,ZHANG L.Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Anchorage,AK,USA:IEEE,2008:1-8.
[6] 肖利梅,曹洁,李策.基于多尺度相位谱的显著性运动目标检测 [J].兰州理工大学学报,2011,37(4):90-94.
[7] LI J,LEVINE M D,AN X,et al.Visual saliency based on scale-space analysis in the frequency domain [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(4):996-1010.
[8] JUDD T,EHINGER K,DURAND F,et al.Learning to predict where humans look [C]//Proceedings of IEEE International Conference on Computer Vision.Kyoto,Japan:IEEE,2010:2106-2113.
[9] VIG E,DORR M,COX D.Large-scale optimization of hierarchical features for saliency prediction in natural images [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:2798-2805.
[10] KÜMMERER M,WALLIS T S A,GATYS L A,et al.Understanding low-and high-level contributions to fixation prediction[C]//Proceedings of IEEE International Conference on Computer Vision.Venice,Italy:IEEE,2017:4799-4808.
[11] CORNIA M,BARALDI L,SERRA G,et al.A deep multi-level network for saliency prediction [C]//Proceedings of IEEE International Conference on Pattern Recognition.Cancun,Mexico:IEEE,2016:3488-3493.
[12] HE S,TAVAKOLI H R,BORJI A,et al.Understanding and visualizing deep visual saliency models [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Long Beach,CA,USA:IEEE,2019:10206-10215.
[13] CORNIA M,BARALDI L,SERRA G,et al.Predicting human eye fixations via an LSTM-based saliency attentive model [J].IEEE Transactions on Image Processing,2018,27(10):5142-5154.
[14] CHAN W L,CHOI H,BARANIUK R G.Multiscale image disparity estimation using the quaternion wavelet transform [C]//Proceedings of IEEE International Conference on Image Processing.Atlanta,GA,USA:IEEE,2007:1229-1232.
[15] KRONER A,SENDEN M,DRIESSEN K,et al.Contextual encoder-decoder network for visual saliency prediction [CP/OL].(2019-08-10) [2020-01-05].https://github.com/alexanderkroner/saliency.
[16] WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module [C]//European Conference on Computer Vision.Munich,Germany:[s.n.],2018:3-19.
[17] JIANG M,HUANG S S,DUAN J Y,et al.SALICON:Saliency in context [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:1072-1080.
[18] BORJI A,ITTI L.CAT2000:A large scale fixation dataset for boosting saliency research [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:IEEE,2015:1-4.
[19] LIU N,HAN J.A deep spatial contextual long-term recurrent convolutional network for saliency detection [J].IEEE Transactions on Image Processing,2018,27(7):3264-3274.

基于超复数小波和图像空域的卷积网络融合注视点预测算法

Gaze prediction algorithm based on hypercomplex wavelet convolutional network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 10

编辑推荐

Metrics

本文评价

[1]	卢鹏丽, 陈云天. 基于网络拓扑和多种生物信息融合的关键蛋白质识别算法[J]. 兰州理工大学学报, 2022, 48(6): 96-103.
[2]	卢鹏丽, 许星舟. 基于精准k核的复杂网络节点重要性评估方法[J]. 兰州理工大学学报, 2022, 48(4): 90-98.
[3]	李策, 许大有, 靳山岗, 高伟哲, 陈晓雷. 基于通道注意力与空洞卷积的胸片肺气肿检测算法[J]. 兰州理工大学学报, 2022, 48(2): 81-89.
[4]	卢鹏丽, 周庚. 基于邻介熵和邻度熵的复杂网络中心性算法[J]. 兰州理工大学学报, 2021, 47(4): 91-98.
[5]	祝延波. 基于图论的中国高铁网节点中心性进化研究[J]. 兰州理工大学学报, 2021, 47(3): 97-104.
[6]	李策, 李兰, 靳山岗, 高伟哲, 许大有. 基于双树复数小波四元数卷积网络的Copy-move盲取证算法[J]. 兰州理工大学学报, 2021, 47(2): 87-96.
[7]	卢鹏丽, 周庚. 基于网络局部和全局特性的组合中心性度量措施[J]. 兰州理工大学学报, 2021, 47(1): 91-96.
[8]	卢鹏丽, 于洲. 基于度与H指数扩展的复杂网络节点排序方法[J]. 兰州理工大学学报, 2020, 46(5): 100-106.
[9]	董瑞洪, 闫厚华, 张秋余, 李学勇. 基于深度森林算法的分布式WSN入侵检测模型[J]. 兰州理工大学学报, 2020, 46(4): 103-109.
[10]	卢鹏丽, 郭旭东, 董璊, 曹乐. 基于介度熵的复杂网络节点重要度识别方法[J]. 兰州理工大学学报, 2020, 46(2): 111-115.