Title: Survey on iterative and incremental approaches in distributed computing environment

Authors: Afaf G. Bin Saadon; Hoda M.O. Mokhtar

Addresses: Faculty of Computers and Information, Cairo University, Giza, 12613, Egypt ' Faculty of Computers and Information, Cairo University, Giza, 12613, Egypt

Abstract: Iterative computation has become increasingly needed for a large and important class of applications such as machine learning and data mining. These iterative applications typically apply computations over large-scale datasets. So it is desirable to develop efficiently distributed frameworks to process data iteratively. On the other hand, data keeps growing over time as new entries are added and existing entries are deleted or modified. This incremental nature of data makes the previously computed results of iterative applications stale and inaccurate over time. It is hence necessary to periodically refresh the computation so that the new changes can be quickly reflected in the computed results. This paper presents the existing distributed systems that support iterative and incremental computations on large-scale datasets. It describes the main optimisations and features of these systems and identifies their limitations.

Keywords: big data; distributed systems; iterative computation; incremental processing.

DOI: 10.1504/IJDS.2019.098359

International Journal of Data Science, 2019 Vol.4 No.1, pp.18 - 30

Received: 02 May 2017
Accepted: 25 Aug 2017
Published online: 18 Mar 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article