Title: Deep reinforcement learning-based collaborative computation offloading and caching decision for internet of things

Authors: Jianxin Li; Ke Yuan; Qian Wang; Siguang Chen

Addresses: School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210000, China ' School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210000, China ' School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210000, China ' School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210000, China

Abstract: Currently, as an extension of cloud computing, edge computing has attracted much attention for its ability to reduce delay and energy and bandwidth consumption. For satisfying the demands of resource-intensive and delay-sensitive applications and solving the problems of existing computation offloading algorithms, such as inability to handle massive amounts of data in time and the need for strong computing capability, an intelligent computation offloading, resource allocation and collaborative caching scheme with joint optimisation of delay and energy consumption is proposed. Specifically, we formulate an optimisation problem of minimising the weighted sum of all tasks' completion time and energy consumption under the constraints of delay, bandwidth, computing capability, and energy. To solve the above mixed integer nonlinear programming problem (MINLP), we develop a deep reinforcement learning (DRL)-based collaborative computation offloading and caching decision (DRL-CCOC) algorithm. The algorithm jointly optimises offloading decisions, caching decisions, the occupation ratio of the wireless channel bandwidth and the edge server's computing capability which is allocated to the task. It can generate the optimal policy and adapt to dynamic network environment with the ability of autodidacticism. Finally, the simulation results demonstrate that the DRL-CCOC can converge at a faster rate and reduce the total cost significantly compared with other methods, they also confirm the strong dynamic adaptability of our algorithm.

Keywords: computing; computation offloading; caching; deep reinforcement learning; DRL; resource allocation.

DOI: 10.1504/IJES.2024.144359

International Journal of Embedded Systems, 2024 Vol.17 No.3/4, pp.161 - 170

Received: 23 Oct 2022
Accepted: 07 Jan 2024
Published online: 10 Feb 2025 *