基于智能优化决策的目标跟踪传感器路径规划

兰州理工大学学报 ›› 2024, Vol. 50 ›› Issue (6): 92-98.

• 自动化技术与计算机技术 • 上一篇下一篇

基于智能优化决策的目标跟踪传感器路径规划

张文旭¹, 王晓晴¹, 陈辉^*1, 赵永红²

1.兰州理工大学电气工程与信息工程学院, 甘肃兰州 730050;
2.甘肃长风电子科技有限责任公司, 甘肃兰州 730070

收稿日期:2022-06-29 出版日期:2024-12-28 发布日期:2025-01-13
通讯作者: 陈辉(1978-),男,山西闻喜人,博士,教授,博导.Email:huich78@hotmail.com
基金资助:
国家国防基础科研项目(JCKY2018427C002),国家自然科学基金(62366031,61873116,51668039,61763029),甘肃省教育厅产业支撑计划(2021CYZC-02)

Path planning of target tracking sensor based on intelligent optimization decision

ZHANG Wen-xu¹, WANG Xiao-qing¹, CHEN Hui¹, ZHAO Yong-hong²

1. College of Electrical and Information Engineering, Lanzhou Univ. of Tech., Lanzhou 730050, China;
2. Gansu Province Changfeng Electronic Technology Co. Ltd., Lanzhou 730070, China

Received:2022-06-29 Online:2024-12-28 Published:2025-01-13

摘要/Abstract

摘要： 针对目标跟踪系统基于传感器管理的估计优化问题,提出一种在离散空间基于强化学习的传感器路径规划方法.首先,通过非线性最优滤波获得目标下一时刻的估计位置与协方差.然后,建立了基于强化学习的目标跟踪优化模型,基于SARSA算法计算传感器路径规划后的目标协方差的迹.最后,对下一时刻的传感器位置选择进行单步循环,根据每次循环中传感器路径规划前后协方差的迹的对比,获得传感器的最优移动位置.在仿真试验中对比分析了传感器在不同学习幕数中的运动轨迹,结果显示目标跟踪优化的效果得到明显提升.

关键词: 目标跟踪, 强化学习, SARSA, 传感器路径规划

Abstract: Aiming at the estimation and optimization problem of target tracking systems managed by sensor control, this paper proposes a sensor path planning method based on reinforcement learning in the discrete space. First, the estimated position and the covariance of the target at the next time are obtained by nonlinear optimal filtering. Then, a target tracking optimization model based on reinforcement learning is established, and the target covariance trace after sensor path planning is calculated based on the SARSA (state-action-reward-state-action) algorithm. Finally, a single-step cycle is performed on the sensor position selection the next time. The optimal moving position is obtained by comparison of the covariance traces before and after the sensor path planning in each cycle. In the simulation experiment, the motion trajectories of the sensor in different learning episodes were compared and analyzed, and the results show that the effect of target tracking optimization was significantly improved.

Key words: target tracking, reinforcement learning, SARSA, sensor path planning

中图分类号:

TP273

张文旭, 王晓晴, 陈辉, 赵永红. 基于智能优化决策的目标跟踪传感器路径规划[J]. 兰州理工大学学报, 2024, 50(6): 92-98.

ZHANG Wen-xu, WANG Xiao-qing, CHEN Hui, ZHAO Yong-hong. Path planning of target tracking sensor based on intelligent optimization decision[J]. Journal of Lanzhou University of Technology, 2024, 50(6): 92-98.

参考文献

[1] HERO A O,COCHRAN D.Sensor management:past,present,and future[J].IEEE Sensors Journal,2011,11(12):3064-3075.
[2] 陈辉,邓东明,韩崇昭.基于区间箱粒子多伯努利滤波器的传感器控制策略[J].自动化学报,2021,47(6):1428-1443.
[3] HASHEMI A,GHASEMI M,VIKALO H,et al.A randomized greedy algorithm for near-optimal sensor scheduling in large-scale sensor networks[C]//2018 Annual American Control Conference (ACC).Wisconsin:IEEE,2018:1027-1032.
[4] LIU F,JIANG C,XIAO W.Multistep prediction-based adaptive dynamic programming sensor scheduling approach for collaborative target tracking in energy harvesting wireless sensor networks[J].IEEE Transactions on Automation Science and Engineering,2020,18(2):693-704.
[5] PANG C,SHAN G.Sensor scheduling based on risk for target tracking[J].IEEE Sensors Journal,2019,19(18):8224-8232.
[6] PINTO S C,ANDERSSON S B,HENDRICKX J M,et al.Optimal minimax mobile sensor scheduling over a network[C]//2021 American Control Conference (ACC).New Orleans:IEEE,2021:1180-1185.
[7] ZHANG C,HWANG I.Decentralized multi-sensor schedulingformulti-targettrackingandidentitymanagement[C]//2019
18th European Control Conference (ECC).Piscataway:IEEE,2019:1804-1809.
[8] KAELBLING L P,LITTMAN M L,MOORE A W.Reinforcement learning:a survey[J].Journal of Artificial Intelligence Research,1996,4:237-285.
[9] FRANCOIS-LAVET V,HENDERSON P,ISIAM R,et al.An introduction to deep reinforcement learning[J].Foundations and Trends in Machine Learning,2018,11(3/4):219-354.
[10] CAMPBELL R H,CZECHOWSKI K,ERHAN D,et al.Model based reinforcement learning for Atari[C]//Proceedings of the International Conference on Learning Representations.New Orleans:[s.n.],2019:6-9.
[11] KUMAR A,ZHOU A,TUCKER G,et al.Conservative Q-learning for offline reinforcement learning[J].Advances in Neural Information Processing Systems,2020,33:1179-1191.
[12] RECHT B.A tour of reinforcement learning:the view from continuous control[J].Annual Review of Control,Robotics,and Autonomous Systems,2019,2:253-279.
[13] LEONG A S,RAMASWAMY A,QUEVEDO D E,et al.Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems[J].Automatica,2020,113:108759.
[14] YANG L,RAO H,LIN M,et al.Optimal sensor scheduling for remote state estimation with limited bandwidth:a deep reinforcement learning approach[J].Information Sciences,2022,588:279-292.
[15] YANG J,YOU X,WU G,et al.Application of reinforcement learning in UAV cluster task scheduling[J].Future Generation Computer Systems,2019,95:140-148.
[16] ZHAO D,WANG H,SHAO K,et al.Deep reinforcement learning with experience replay based on SARSA[C]//2016 IEEE Symposium Series on Computational Intelligence (SSCI).Athens:IEEE,2016:1-6.
[17] ALFAKIH T,HASSAN M M,GUMAEI A,et al.Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J].IEEE Access,2020,8:54074-54084.
[18] ASGHARI A,SOHRABI M K,YAGHMAEE F.Task scheduling,resource provisioning,and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm[J].The Journal of Supercomputing,2021,77(3):2800-2828.
[19] LAI X,YI W,CUI Y,et al.Capacity estimation of lithium-ion cells by combining model-based and data-driven methods based on a sequential extended Kalman filter[J].Energy,2021,216:119233.
[20] 孙照强,王志贵,孟飞,等.基于EKF及弹道方程的弹道目标跟踪滤波器设计[J].系统工程与电子技术,2022,44(10):3207-3212.
[21] 陈辉,刘雅婷,张双庆,等.多扩展目标跟踪中基于加权最优子模式分配距离的传感器管理方法[J].控制理论与应用,2022,39(5):887-896.

基于智能优化决策的目标跟踪传感器路径规划

Path planning of target tracking sensor based on intelligent optimization decision

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	陈辉, 王荆宇, 张文旭, 赵永红, 席磊. 基于蒙特卡罗策略梯度的雷达观测器轨迹规划[J]. 兰州理工大学学报, 2024, 50(5): 77-85.
[2]	陈辉, 刘雅婷, 王莉. 高斯混合标签多伯努利滤波器的传感器控制策略[J]. 兰州理工大学学报, 2022, 48(2): 73-80.
[3]	陈辉, 张星星, 杨文瑜. 厚尾噪声条件下星凸形扩展目标student’s t滤波器[J]. 兰州理工大学学报, 2021, 47(5): 85-92.