Abstract：For the robot formation tracking problem of unknown target, a robot motion control model is established, and a target tracking and ring-around control strategy based on Reinforcement Learning(RL) is proposed to solve the problem. Driven by RL, the robot explore the location of the target point and initiate tracking. The robot tracking strategy is optimized in real time using the ring-around formation motion model to achieve dynamic tracking and ring-around control of the fleeing target point. A multi-robot motion control environment is established, and the experiments indicate that the combined RL can accelerate the multi-robot formation adjustment time and prove the efficiency of the multi-robot ring-around formation control strategy.
韩艺琳, 王丽丽, 杨洪勇, 范之琳. 基于强化学习的多机器人系统的环围编队控制[J]. 复杂系统与复杂性科学, 2023, 20(3): 97-102.
HAN Yilin, WANG Lili, YANG Hongyong, FAN Zhilin. Ring-around Formation Control of Multi-robot Systems Based on Reinforcement Learning. Complex Systems and Complexity Science, 2023, 20(3): 97-102.
[1] YAN Z, JOUANDEAU N, CHERIF A A. A survey and analysis of multi-robot coordination[J]. International Journal of Advanced Robotic Systems, 2013, 10(12):399. [2] QU Y, SUN Y, WANG K, et al. Multi-UAV Cooperative Search method for a Moving Target on the Ground or Sea[C] //2019 Chinese Control Conference (CCC). GuangZhou, China: IEEE, 2019: 40494054. [3] KAMALAPURKAR R, ANDREWS L, WALTERS P, et al. Model-based reinforcement learning for infinite-horizon approximate optimal tracking[J]. IEEE transactions on neural networks and learning systems, 2016, 28(3): 753758. [4] 路兰,殷水英. 基于空间交互作用的中国省际人口流动模型研究[DB/OL]. (2023-08-08)[2023-08-15].https://link.cnki.net/urlid/11.1115.F.20230808.1339.004. LU L, YIN S Y. Study on the model of inter-provincial population flow in China based on spatial interaction[DB/OL]. https://link.cnki.net/urlid/11.1115.F.20230808.1339.004. [5] MOHAN B M, SINHA A. The simplest fuzzy PID controllers: mathematical models and stability analysis[J]. Soft Computing, 2006, 10(10): 961975. [6] 于欣波,贺威,薛程谦,等.基于扰动观测器的机器人自适应神经网络跟踪控制研究[J].自动化学报, 2019, 45(7):13071324. YU X B, HE W, XUE C J, et al. Research on robot adaptive neural network tracking control based on disturbance observer [J]. Journal of Automation, 2019,45(7):13071324. [7] 徐鹏,谢广明,文家燕,等.事件驱动的强化学习多智能体编队控制[J].智能系统学报, 2019,14(1):9398. XU P, XIE G M, WEN J Y, et al. Event driven reinforcement learning multi-agent formation control [J]. Journal of Intelligent Systems, 2019,14(1):9398. [8] YU Z, ZHANG Y, LIU Z, et al. Distributed adaptive fractional-order fault-tolerant cooperative control of networked unmanned aerial vehicles via fuzzy neural networks[J]. IET Control Theory & Applications, 2019, 13(17): 29172929. [9] ZHANG B, SUN X, LIU S, et al. Adaptive differential evolution-based distributed model predictive control for multi-UAV formation flight[J]. International Journal of Aeronautical and Space Sciences, 2020: 21(2):538548. [10] YIN S, XIAO B. Tracking control of surface ships with disturbance and uncertainties rejection capability[J]. IEEE/ASME Transactions on Mechatronics, 2016, 22(3): 11541162. [11] ROVEDA L, PALLUCCA G, PEDROCCHI N, et al. Iterative learning procedure with reinforcement for high-accuracy force tracking in robotized tasks[J]. IEEE Transactions on Industrial Informatics, 2017, 14(4): 17531763. [12] GAO S, SONG R, LI Y. Cooperative control of multiple nonholonomic robots for escorting and patrolling mission based on vector field[J]. IEEE Access, 2018, 6: 4188341891. [13] CHOU C Y, JUANG C F. Navigation of an autonomous wheeled robot in unknown environments based on evolutionary fuzzy control[J]. Inventions, 2018, 3(1): 3. [14] WANG M, LUO J, YUAN J, et al. Detumbling strategy and coordination control of kinematically redundant space robot after capturing a tumbling target[J]. Nonlinear Dynamics, 2018, 92(3): 10231043. [15] YAO W, LU H, ZENG Z, et al. Distributed static and dynamic circumnavigation control with arbitrary spacings for a heterogeneous multi-robot system[J]. Journal of Intelligent & Robotic Systems, 2019, 94(3): 883905. [16] LU C, WANG J, CUI X. Moving Target Tracking with Robot Based on Laser Range Finder[C] //2020 5th International Conference on Automation, Control and Robotics Engineering (CACRE). Dalian, China: IEEE, 2020: 2125. [17] WANG Y, LU D, SUN C Y. Cooperative control for multi-player pursuit-evasion games with reinforcement learning[J]. Neurocomputing, 2020,412:101114 [18] GE H, SONG Y, WU C, et al. Cooperative deep Q-learning with Q-value transfer for multi-intersection signal control[J]. IEEE Access, 2019, 7: 4079740809. [19] SAMPEDRO C, BAVLE H, Rodriguez-Ramos A, et al. Laser-Based Reactive Navigation for Multirotor Aerial Robots using Deep Reinforcement Learning[C] // 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018. [20] NOGUCHI Y, MAKI T. Path Planning Method Based on Artificial Potential Field and Reinforcement Learning for Intervention AUVs[C] // 2019 IEEE Symposium on Underwater Technology (UT). Taiwan, China: IEEE, 2019:16.
