Path planning of target tracking sensor based on intelligent optimization decision

Abstract

Abstract: Aiming at the estimation and optimization problem of target tracking systems managed by sensor control, this paper proposes a sensor path planning method based on reinforcement learning in the discrete space. First, the estimated position and the covariance of the target at the next time are obtained by nonlinear optimal filtering. Then, a target tracking optimization model based on reinforcement learning is established, and the target covariance trace after sensor path planning is calculated based on the SARSA (state-action-reward-state-action) algorithm. Finally, a single-step cycle is performed on the sensor position selection the next time. The optimal moving position is obtained by comparison of the covariance traces before and after the sensor path planning in each cycle. In the simulation experiment, the motion trajectories of the sensor in different learning episodes were compared and analyzed, and the results show that the effect of target tracking optimization was significantly improved.

Key words: target tracking, reinforcement learning, SARSA, sensor path planning

CLC Number:

TP273

ZHANG Wen-xu, WANG Xiao-qing, CHEN Hui, ZHAO Yong-hong. Path planning of target tracking sensor based on intelligent optimization decision[J]. Journal of Lanzhou University of Technology, 2024, 50(6): 92-98.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://journal.lut.edu.cn/EN/

https://journal.lut.edu.cn/EN/Y2024/V50/I6/92

References

[1] HERO A O,COCHRAN D.Sensor management:past,present,and future[J].IEEE Sensors Journal,2011,11(12):3064-3075.
[2] 陈辉,邓东明,韩崇昭.基于区间箱粒子多伯努利滤波器的传感器控制策略[J].自动化学报,2021,47(6):1428-1443.
[3] HASHEMI A,GHASEMI M,VIKALO H,et al.A randomized greedy algorithm for near-optimal sensor scheduling in large-scale sensor networks[C]//2018 Annual American Control Conference (ACC).Wisconsin:IEEE,2018:1027-1032.
[4] LIU F,JIANG C,XIAO W.Multistep prediction-based adaptive dynamic programming sensor scheduling approach for collaborative target tracking in energy harvesting wireless sensor networks[J].IEEE Transactions on Automation Science and Engineering,2020,18(2):693-704.
[5] PANG C,SHAN G.Sensor scheduling based on risk for target tracking[J].IEEE Sensors Journal,2019,19(18):8224-8232.
[6] PINTO S C,ANDERSSON S B,HENDRICKX J M,et al.Optimal minimax mobile sensor scheduling over a network[C]//2021 American Control Conference (ACC).New Orleans:IEEE,2021:1180-1185.
[7] ZHANG C,HWANG I.Decentralized multi-sensor schedulingformulti-targettrackingandidentitymanagement[C]//2019
18th European Control Conference (ECC).Piscataway:IEEE,2019:1804-1809.
[8] KAELBLING L P,LITTMAN M L,MOORE A W.Reinforcement learning:a survey[J].Journal of Artificial Intelligence Research,1996,4:237-285.
[9] FRANCOIS-LAVET V,HENDERSON P,ISIAM R,et al.An introduction to deep reinforcement learning[J].Foundations and Trends in Machine Learning,2018,11(3/4):219-354.
[10] CAMPBELL R H,CZECHOWSKI K,ERHAN D,et al.Model based reinforcement learning for Atari[C]//Proceedings of the International Conference on Learning Representations.New Orleans:[s.n.],2019:6-9.
[11] KUMAR A,ZHOU A,TUCKER G,et al.Conservative Q-learning for offline reinforcement learning[J].Advances in Neural Information Processing Systems,2020,33:1179-1191.
[12] RECHT B.A tour of reinforcement learning:the view from continuous control[J].Annual Review of Control,Robotics,and Autonomous Systems,2019,2:253-279.
[13] LEONG A S,RAMASWAMY A,QUEVEDO D E,et al.Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems[J].Automatica,2020,113:108759.
[14] YANG L,RAO H,LIN M,et al.Optimal sensor scheduling for remote state estimation with limited bandwidth:a deep reinforcement learning approach[J].Information Sciences,2022,588:279-292.
[15] YANG J,YOU X,WU G,et al.Application of reinforcement learning in UAV cluster task scheduling[J].Future Generation Computer Systems,2019,95:140-148.
[16] ZHAO D,WANG H,SHAO K,et al.Deep reinforcement learning with experience replay based on SARSA[C]//2016 IEEE Symposium Series on Computational Intelligence (SSCI).Athens:IEEE,2016:1-6.
[17] ALFAKIH T,HASSAN M M,GUMAEI A,et al.Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J].IEEE Access,2020,8:54074-54084.
[18] ASGHARI A,SOHRABI M K,YAGHMAEE F.Task scheduling,resource provisioning,and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm[J].The Journal of Supercomputing,2021,77(3):2800-2828.
[19] LAI X,YI W,CUI Y,et al.Capacity estimation of lithium-ion cells by combining model-based and data-driven methods based on a sequential extended Kalman filter[J].Energy,2021,216:119233.
[20] 孙照强,王志贵,孟飞,等.基于EKF及弹道方程的弹道目标跟踪滤波器设计[J].系统工程与电子技术,2022,44(10):3207-3212.
[21] 陈辉,刘雅婷,张双庆,等.多扩展目标跟踪中基于加权最优子模式分配距离的传感器管理方法[J].控制理论与应用,2022,39(5):887-896.