针对机器人对未知目标的编队跟踪问题,建立机器人运动控制模型,提出了基于强化学习的目标跟踪与环围控制策略。在强化学习策略驱动下,机器人探索发现目标点位置并展开跟踪,使用环围编队运动模型对机器人跟踪策略进行实时优化,实现对逃逸目标点的动态跟踪与环围控制。搭建了多机器人运动测试环境,实验表明结合强化学习的方法能够缩短多机器人编队调节时间,验证了多机器人环围编队控制策略的有效性。
For the robot formation tracking problem of unknown target, a robot motion control model is established, and a target tracking and ring-around control strategy based on Reinforcement Learning(RL) is proposed to solve the problem. Driven by RL, the robot explore the location of the target point and initiate tracking. The robot tracking strategy is optimized in real time using the ring-around formation motion model to achieve dynamic tracking and ring-around control of the fleeing target point. A multi-robot motion control environment is established, and the experiments indicate that the combined RL can accelerate the multi-robot formation adjustment time and prove the efficiency of the multi-robot ring-around formation control strategy.
[1] YAN Z, JOUANDEAU N, CHERIF A A. A survey and analysis of multi-robot coordination[J]. International Journal of Advanced Robotic Systems, 2013, 10(12):399.
[2] QU Y, SUN Y, WANG K, et al. Multi-UAV Cooperative Search method for a Moving Target on the Ground or Sea[C] //2019 Chinese Control Conference (CCC). GuangZhou, China: IEEE, 2019: 40494054.
[3] KAMALAPURKAR R, ANDREWS L, WALTERS P, et al. Model-based reinforcement learning for infinite-horizon approximate optimal tracking[J]. IEEE transactions on neural networks and learning systems, 2016, 28(3): 753758.
[4] 路兰,殷水英. 基于空间交互作用的中国省际人口流动模型研究[DB/OL]. (2023-08-08)[2023-08-15].https://link.cnki.net/urlid/11.1115.F.20230808.1339.004.
LU L, YIN S Y. Study on the model of inter-provincial population flow in China based on spatial interaction[DB/OL]. https://link.cnki.net/urlid/11.1115.F.20230808.1339.004.
[5] MOHAN B M, SINHA A. The simplest fuzzy PID controllers: mathematical models and stability analysis[J]. Soft Computing, 2006, 10(10): 961975.
[6] 于欣波,贺威,薛程谦,等.基于扰动观测器的机器人自适应神经网络跟踪控制研究[J].自动化学报, 2019, 45(7):13071324.
YU X B, HE W, XUE C J, et al. Research on robot adaptive neural network tracking control based on disturbance observer [J]. Journal of Automation, 2019,45(7):13071324.
[7] 徐鹏,谢广明,文家燕,等.事件驱动的强化学习多智能体编队控制[J].智能系统学报, 2019,14(1):9398.
XU P, XIE G M, WEN J Y, et al. Event driven reinforcement learning multi-agent formation control [J]. Journal of Intelligent Systems, 2019,14(1):9398.
[8] YU Z, ZHANG Y, LIU Z, et al. Distributed adaptive fractional-order fault-tolerant cooperative control of networked unmanned aerial vehicles via fuzzy neural networks[J]. IET Control Theory & Applications, 2019, 13(17): 29172929.
[9] ZHANG B, SUN X, LIU S, et al. Adaptive differential evolution-based distributed model predictive control for multi-UAV formation flight[J]. International Journal of Aeronautical and Space Sciences, 2020: 21(2):538548.
[10] YIN S, XIAO B. Tracking control of surface ships with disturbance and uncertainties rejection capability[J]. IEEE/ASME Transactions on Mechatronics, 2016, 22(3): 11541162.
[11] ROVEDA L, PALLUCCA G, PEDROCCHI N, et al. Iterative learning procedure with reinforcement for high-accuracy force tracking in robotized tasks[J]. IEEE Transactions on Industrial Informatics, 2017, 14(4): 17531763.
[12] GAO S, SONG R, LI Y. Cooperative control of multiple nonholonomic robots for escorting and patrolling mission based on vector field[J]. IEEE Access, 2018, 6: 4188341891.
[13] CHOU C Y, JUANG C F. Navigation of an autonomous wheeled robot in unknown environments based on evolutionary fuzzy control[J]. Inventions, 2018, 3(1): 3.
[14] WANG M, LUO J, YUAN J, et al. Detumbling strategy and coordination control of kinematically redundant space robot after capturing a tumbling target[J]. Nonlinear Dynamics, 2018, 92(3): 10231043.
[15] YAO W, LU H, ZENG Z, et al. Distributed static and dynamic circumnavigation control with arbitrary spacings for a heterogeneous multi-robot system[J]. Journal of Intelligent & Robotic Systems, 2019, 94(3): 883905.
[16] LU C, WANG J, CUI X. Moving Target Tracking with Robot Based on Laser Range Finder[C] //2020 5th International Conference on Automation, Control and Robotics Engineering (CACRE). Dalian, China: IEEE, 2020: 2125.
[17] WANG Y, LU D, SUN C Y. Cooperative control for multi-player pursuit-evasion games with reinforcement learning[J]. Neurocomputing, 2020,412:101114
[18] GE H, SONG Y, WU C, et al. Cooperative deep Q-learning with Q-value transfer for multi-intersection signal control[J]. IEEE Access, 2019, 7: 4079740809.
[19] SAMPEDRO C, BAVLE H, Rodriguez-Ramos A, et al. Laser-Based Reactive Navigation for Multirotor Aerial Robots using Deep Reinforcement Learning[C] // 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018.
[20] NOGUCHI Y, MAKI T. Path Planning Method Based on Artificial Potential Field and Reinforcement Learning for Intervention AUVs[C] // 2019 IEEE Symposium on Underwater Technology (UT). Taiwan, China: IEEE, 2019:16.
