Title: Meta Q-network: a combination of reinforcement learning and meta learning

Authors: Min Lu; Yi Wang; Wenfeng Wang

Addresses: School of Science, Shanghai Institute of Technology, Shanghai, 201418, China ' School of Science, Shanghai Institute of Technology, Shanghai, 201418, China ' Sino-Indian Joint Research Center of Artificial Intelligence and Robotics, Interscience Institute of Management and Robotics, Bhubaneswar, 752054, India; School of Electrical and Electronics Engineering, Shanghai Institute of Technology, Shanghai, 201418, China

Abstract: Deep reinforcement learning develops rapidly by using neural network to approximate the learning data of reinforcement learning, which makes the sequential decision in continuous space making achieve preliminary results. However, deep reinforcement learning is over-dependent on huge amount of training and requires accurate reward. For many problems in the real world, such as robot learning, there is generally no good reward and no unlimited training, which requires the ability to learn quickly. In this paper, we propose a deep reinforcement learning model with meta-learning, which we call meta Q-network (MQN). The model uses a LSTM-based meta-learner to update the Q-network. This optimises a series of problems such as the difficulty in stability of Q-network in deep reinforcement learning model, and we have proved this improved performance through experiments. It is not optimal, though, still the combination of meta-learning and reinforcement learning is very desirable.

Keywords: machine learning; reinforcement learning; meta-learning; artificial intelligence.

DOI: 10.1504/IJANS.2022.125303

International Journal of Applied Nonlinear Science, 2022 Vol.3 No.3, pp.179 - 188

Received: 31 May 2021
Accepted: 26 Jan 2022
Published online: 06 Sep 2022 *