Title: Application of fuzzy clustering for text data dimensionality reduction

Authors: Amir Karami

Addresses: College of Information and Communications, University of South Carolina, USA

Abstract: Large textual corpora are often represented by the document-term frequency matrix whose elements are the frequency of terms; however, this matrix has two problems: sparsity and high dimensionality. Four dimension reduction strategies are used to address these problems. Of the four strategies, unsupervised-feature transformation (UFT) is a popular and efficient strategy to map the terms to a new basis in the document-term frequency matrix. Although several UFT-based methods have been developed, fuzzy clustering has not been considered for dimensionality reduction. This research explores fuzzy clustering as a new UFT-based approach to create a lower-dimensional representation of documents. Performance of fuzzy clustering with and without using global term weighting methods is shown to exceed principal component analysis and singular value decomposition. This study also explores the effect of applying different fuzzifier values on fuzzy clustering for dimensionality reduction purpose.

Keywords: dimension reduction; fuzzy clustering; singular value decomposition; SVD; principal component analysis; PCA; text mining.

DOI: 10.1504/IJKEDM.2019.102487

International Journal of Knowledge Engineering and Data Mining, 2019 Vol.6 No.3, pp.289 - 306

Received: 25 Feb 2019
Accepted: 02 May 2019
Published online: 27 Sep 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article