Skip to main navigation Skip to search Skip to main content

Developing Statistical Properties and Markov Chains for Arabic Language Diacritized Names.

Project: RESEARCH PRIORITY PROJECT

Project Details

Description

Statistical N-grams language models (LMs) have shown to be very effective in natural language processing (NLP), particularly in automatic speech recognition (ASR) and machine translation. In fact, the successful impact of LMs promote to introduce efficient techniques as well as different types models in various linguistic applications. The LMs mainly include two types that are grammars and statistical language models that is also called N-grams. The main difference between grammars and statistical language models is that the statistical language models are based on the estimation of probabilities for words sequences while the grammars usually do not have probabilities. Despite there are many toolkits that are used to create LMs, however, this work employs two well-known language modeling toolkits with focus on the Arabic text. The implementing toolkits include the Carnegie Mellon University (CMU)-Cambridge Language Modeling Toolkit and the Cambridge University Hidden Markov Model Toolkit (HTK) language modeling toolkits. For clarification, we used a small Arabic text corpus to compute the N-grams for 1-gram, 2-gram, and 3-gram. In addition, this paper demonstrates the intermediate steps that are needed to generate the ARPA-format LMs using both toolkits.

StatusFinished
Effective start/end date26/08/06 → 25/08/07

Keywords

  • Arabic Language Processing
  • Mathematical modeling
  • algorithm
  • translation.

Fingerprint

Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.
  • Exploring the language modeling toolkits for Arabic text

    Al-Anzi, F. S. & Abuzeina, D., 28 Jun 2017, 2017 International Conference on Electrical and Computing Technologies and Applications, ICECTA 2017. Institute of Electrical and Electronics Engineers Inc., p. 1-4 4 p. (2017 International Conference on Electrical and Computing Technologies and Applications, ICECTA 2017; vol. 2018-January).

    Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

    2 Scopus citations