Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG

• Published in: Nashrīyah-i mudīrīyat-i fannāvarī-i iṭṭilāʻāt Vol. 15; no. Special Issue; pp. 19 - 33
• Main Authors: M. Rajani Shree, Shambhavi B. R.
• Format: Journal Article
• Language: Persian
• Published: University of Tehran 01-01-2023
• Subjects: artificial intelligence; cyk parsing algorithm; natural language processing; probabilistic context free grammar; syntax parser
• ISSN: 2008-5893; 2423-5059
• DOI: 10.22059/jitm.2022.89408

In our research, we provide a statistical syntax parsing method experimented on Kannada texts, which is an official language of Karnataka, India. The dataset is downloaded from TDIL website. Using the Cocke-Younger-Kasami (CYK) parsing technique, we generated Kannada Treebank dataset from 1000 annotated sentences in the first stage. The Treebank generated in this stage contains 1000 syntactically structured sentences and it is used as input to train the syntax parser model in the second stage. We have adopted Probabilistic Context Free Grammar (PCFG) while training the parser model and extracting the Chmosky Normal Form (CNF) grammar from a Treebank dataset. The developed syntax parser model is tested on 150 raw Kannada sentences. It outputs with the most likely parse tree for each sentence and this is verified with golden Treebank. The syntax parser model generated 74.2% precision, 79.4% recall, and 75.3% F1-score respectively. The similar technique may be adopted for other low resource languages.