Title: Mobile robot navigation: neural Q-learning

Authors: S. Parasuraman; Soh Chin Yun

Addresses: School of Engineering, Monash University Sunway Campus, Bandar Sunway, 46150 Malaysia. ' School of Engineering, Monash University Sunway Campus, Bandar Sunway, 46150 Malaysia

Abstract: This paper presents the mobile robot navigation technique which utilises Reinforcement Learning (RL) algorithms and Artificial Neural Network (ANN) to learn in an unknown environment for mobile robot navigation. This research study is focused on the integration of multi-layer neural network and Q-learning as online learning control scheme. This process is divided into two stages. In the initial stage, the agent will map the environment through collecting state-action information according to the Q-learning procedure. Second training process involves neural network which utilises the state-action information gathered in the earlier phase of training samples. During final application of the controller, Q-learning would be used as primary navigating tool whereas the trained neural network will be employed when approximation is needed. MATLAB simulation was developed to verify and validate the algorithm before real-time implementation using Team AmigoBot™ robot. The results obtained from both simulation and real world experiments are discussed.

Keywords: reinforcement learning; artificial neural networka; ANNs; neural Q-learning; MATLAB; mobile robots; robot navigation; learning control; simulation.

DOI: 10.1504/IJCAT.2012.050118

International Journal of Computer Applications in Technology, 2012 Vol.44 No.4, pp.303 - 311

Published online: 29 Oct 2012 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article