Title: Condition-based maintenance plan for multi-state systems using reinforcement learning

Authors: Shuyuan Gan; Nooshin Yousefi; David W. Coit

Addresses: Department of Mechanical Engineering, Jiangsu University, Zhenjiang, Jiangsu, China ' Department of Industrial & System Engineering, Rutgers University, Piscataway, Jersey, USA ' Department of Industrial & System Engineering, Rutgers University, Piscataway, Jersey, USA

Abstract: In this paper, a dynamic condition-based maintenance model is presented for a degrading system using a reinforcement learning approach. The system utilises a critical machine with multiple production states, which are estimated and obtained by some measurable indicators. Within the framework of the suggested maintenance policy, both the production state and machine virtual age are considered during the decision-making process for maintenance actions. The system is inspected at periodic times, and at each inspection, an action is selected from those that are available regarding spare parts and maintenance. Replacement and different levels of imperfect repairs are both considered. Additionally, the ordering of spare parts is also involved. The problem is formulated by using a discrete time Markov decision process and solved by the Q-learning algorithm. Numerical examples are provided to illustrate the proposed method. Also, the results indicate that the proposed method is time-saving and more cost-efficient than traditional methods.

Keywords: dynamic maintenance; Markov decision process; Q-learning; spare parts; virtual age.

DOI: 10.1504/IJRS.2024.139213

International Journal of Reliability and Safety, 2024 Vol.18 No.2, pp.144 - 162

Received: 18 Sep 2023
Accepted: 05 Jan 2024
Published online: 26 Jun 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article