Title: Persian POS tagging using probabilistic morphological analysis

Authors: Hakimeh Fadaei, Mehrnoush Shamsfard

Addresses: NLP Research Laboratory, Faculty of Electrical and Computer Engineering, Shahid Beheshti University, Evin, Tehran, 1983963113, Iran. ' NLP Research Laboratory, Faculty of Electrical and Computer Engineering, Shahid Beheshti University, Evin, Tehran, 1983963113, Iran

Abstract: Part of speech (POS) tagging as a fundamental task in natural language processing (NLP) has attracted many research efforts and many taggers are developed with different approaches to reach high performance and accuracy. In many complex applications of NLP, tagged corpora are among essential resources and designing an algorithm to create or enrich these resources is of high importance. Handling unknown words is a challenge in POS tagging which usually decreases the performance of taggers. This paper presents a POS tagger for Persian. It exploits a hybrid approach which is a combination of statistical and rule-based methods to tag Persian sentences. The proposed tagger uses a novel probabilistic morphological analysis to tag unknown words. As a secondary result of this research a knowledge base of Persian morphological rules with their probabilities is built according to a corpus. Experimental results show that our method improves the tagging performance and accuracy.

Keywords: part-of-speech tagging; natural language processing; NLP; bigrams; probabilistic morphological analysis; lexical acquisition; Persian; unknown words.

DOI: 10.1504/IJCAT.2010.034527

International Journal of Computer Applications in Technology, 2010 Vol.38 No.4, pp.264 - 273

Published online: 07 Aug 2010 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article