Title: Q-grams-imp: an improved q-grams algorithm aimed at edit similarity join
Authors: Yunxia Liu; Zhaobin Liu; Zhiyang Li
Addresses: School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China ' School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China ' School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China
Abstract: Similarity join is more and more important in many applications and has attracted wide-spread attention from scholars and communities. Similarity join has been used in many applications, such as spell checking, copy detection, entity linking, pattern recognition and so on. Actually, in many web and enterprise scenarios, where typos and misspellings often occur, we need to find an efficient algorithm to handle these situations. In this paper, we propose an improved algorithm on q-grams called q-grams-imp that is aimed at solving edit similarity join. We use this algorithm in order to reduce the number of tokens and thus reduce space costs, so it is fit best for same size strings. But for different sizes of strings, we need to handle these strings in order to fit for the algorithm. Finally, we conclude and get the results that our proposed algorithm is better than the traditional method.
Keywords: similarity join; q-grams algorithm; edit distance.
DOI: 10.1504/IJCSE.2019.098537
International Journal of Computational Science and Engineering, 2019 Vol.18 No.3, pp.269 - 278
Received: 30 Jun 2016
Accepted: 08 Sep 2016
Published online: 26 Mar 2019 *