Title: Heuristic deep reinforcement learning approach for deeply adaptive navigation in indoor dynamic environments

Authors: Walid Jebrane; Nabil El Akchioui

Addresses: Faculty of Science and Technology, University Abdelmalek Essaadi, Al Hoceima, Tetouan, Morocco ' Faculty of Science and Technology, University Abdelmalek Essaadi, Al Hoceima, Tetouan, Morocco

Abstract: Navigating mobile robots safely and efficiently through complex indoor environments populated by dynamic obstacles remains a challenging task. While traditional navigation techniques based on path planning and rule-based control struggle in such scenarios, deep reinforcement learning (DRL) offers a promising paradigm by enabling robots to learn adaptive behaviours directly from experience. This paper presents a DRL-based approach for autonomous indoor robot navigation integrated within the robot operating system (ROS) navigation stack. A proximal policy optimisation (PPO) agent is trained using laser scan and goal coordinate observations to output low-level velocity commands in real-time. To address limitations in global re-planning, we incorporate the computationally efficient D* Lite algorithm. Experiments in a Gazebo simulation combining the Flatland and PedSim simulators evaluate our approach. The environment models static obstacles and dynamic pedestrians using social forces modelling. Results demonstrate the ability of our trained agent to safely and efficiently navigate complex maps while negotiating pedestrian traffic. Comparisons with alternative global planners and navigation configurations provide insights into the benefits of integrating DRL with heuristic path planning. Our work contributes an adaptive solution for autonomous mobile robot navigation in dynamic indoor environments.

Keywords: deep reinforcement learning; DRL; D* Lite; proximal policy optimisation; PPO; autonomous navigation.

DOI: 10.1504/IJVP.2024.142076

International Journal of Vehicle Performance, 2024 Vol.10 No.4, pp.403 - 426

Received: 21 Jan 2024
Accepted: 06 Jun 2024
Published online: 07 Oct 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article