Title: Real-time data warehouse: a solution and evaluation

Authors: Nickerson Ferreira; Pedro Furtado

Addresses: Departamento de Engenharia Informática, Faculdade de Ciências e Tecnologia, Universidade de Coimbra, Pólo II – Pinhal de Marrocos, 3030-290 Coimbra, Portugal ' Departamento de Engenharia Informática, Faculdade de Ciências e Tecnologia, Universidade de Coimbra, Pólo II – Pinhal de Marrocos, 3030-290 Coimbra, Portugal

Abstract: Real-time warehousing has been researched and tried in practice by several companies, but there should be a definition of an architecture and an evaluation of its merits. Based on both industry and theoretical insights, we define a data warehouse architecture for constant integration of data without compromising query performance, and we evaluate its capacity to provide realtime. For this real-time data warehouse (RTDW) we define a dynamic warehouse component and a static warehouse component to represent the recently integrated data and the rest of the data, respectively, with relevant choices concerning how the components merge together. We propose design choices concerning query computation mechanisms and evaluate the alternatives to conclude which is the most efficient implementation of those mechanisms. The real-time data warehouse architecture is evaluated with a real-time benchmark setup that considers online loading with simultaneous query workload processing. Results prove the validity of the architecture, compare different mechanisms and quantify its efficiency in the constant integration context. The performance is carefully analysed taking into account several important factors.

Keywords: data warehouses; real time data warehousing; data integration.

DOI: 10.1504/IJBIDM.2013.059046

International Journal of Business Intelligence and Data Mining, 2013 Vol.8 No.3, pp.244 - 263

Received: 14 Oct 2013
Accepted: 16 Oct 2013
Published online: 28 Jun 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article