The paper presents research on the feasibility and development of methods for the rapid creation of stopgap language technology resources for low-density languages: (i) related language bootstrapping can be used to port existing technology from a resource-rich language to its associated lower-density variant; and (ii) clever use of linguistic knowledge can be employed to scale down the need for large amount of training or development data. Based on Persian and Armenian languages, the paper illustrates several methods that can be implemented in each instance in the goal of reducing human effort and avoiding the scarce data issue faced by statistical systems.

The exponential growth of the Persian blogosphere and the increased number of neologisms create a major challenge in NLP applications of Iranian blogs. This paper describes a method for extracting and classifying newly constructed words and borrowings from Persian blog posts. The analysis of the occurrence of neologisms across six distinct topic categories points to a strong correlation between the topic domain and the type of neologism that is most commonly encountered. The results suggest that different approaches should be implemented for the automatic detection and processing of neologisms depending on the domain of application.

Low-density languages raise difficulties for standard approaches to natural language processing that depend on large online corpora. Using Persian as a case study, we propose a novel method for bootstrapping MT capability for a low-density language in the case where it relates to a higher density variant. Tajiki Persian is a low-density language that uses the Cyrillic alphabet, while Iranian Persian (Farsi) is written in an extended version of the Arabic script and has many computational resources available. Despite the orthographic differences, the two languages have literary written forms that are almost identical. The paper describes the development of a comprehensive finite-state transducer that converts Tajik text to Farsi script and runs the resulting transliterated document through an existing Persian-to-English MT system. Due to divergences that arise in mapping the two writing systems and phonological and lexical distinctions, the system uses contextual cues (such as the position of a phoneme in a word) as well as available Farsi resources (such as a morphological analyzer to deal with differences in the affixal structures and a lexicon to disambiguate the analyses) to control the potential combinatorial explosion. The results point to a valuable strategy for the rapid prototyping of MT packages for languages of similar uneven density.

This paper describes a two-level morphological analyzer for Persian using a system based on the Xerox finite state tools. Persian language presents certain challenges to computational analysis: There is a complex verbal conjugation paradigm which includes long-distance morphological dependencies; phonological alternations apply at morpheme boundaries; word and noun phrase boundaries are difficult to define since morphemes may be detached from their stems and distinct words can appear without an intervening space. In this work, we develop these problems and provide solutions in a finite-state morphology system.

This paper presents a complete formalization of Persian inflectional morphology using a unification-based framework. The morphological analyzer was developed for use in a Persian-English machine translation system; it computes the part of speech categories and returns all syntactically relevant inflectional features for a word. The morphological analyses are represented as feature structures, which can easily be used by a syntactic parser. The morphological formalism consists of a declarative description of rules utilizing typed feature structures. Persian morphotactics include a few prefixes and sequences of suffixes with co-occurrence constraints between non-adjacent morphemes. The verbal inflectional morphology is rich and is characterized by a complex system of conjugations. A morphological rule associates a regular expression describing a set of character strings to a typed feature structure. Rules can be combined using regular expression operators and they can be factorized in conjugation tables. The morphological engine is implemented as a finite-state transducer where the left projection is the input string and the right projection is a typed feature structure.

The Computing Research Laboratory (CRL) is developing a machine translation toolkit that allows a rapid deployment of translation capabilities. This toolkit has been used to develop several machine translation systems, including a Persian-English and a Turkish-English system, which will be demonstrated. We present the architecture of these systems as well as the development methodology.

The Computing Research Laboratory is currently developing technologies that allow rapid deployment of automatic translation. These technologies are designed to handle low-density languages for which resources, be that human informants or data in electronically readable form, are scarce. All tools are built in an incremental fashion, such that some simple tools (a bilingual dictionary or a glosser) can be delivered early in the development to support initial analysis tasks. More complex applications can be fielded in successive functional versions. The technology we demonstrate has first been applied to Persian-English machine translation within the Shiraz project and is currently extended to cover languages such as Arabic, Japanese, Korean and others.

This report provides a description of the structure of common Persian names from Iran, with an emphasis on automatic recognition of these personal names. Since the early parts of the 20th century, person proper names of Persian (Farsi) origin have been composed of a first name and a last name. There are no middle names. However, first and last names may each be a compound proper name, consisting of two subparts.

The Computing Research Laboratory (CRL) is developing a machine translation toolkit that allows a rapid deployment of translation capabilities. This toolkit has been used to develop several machine translation systems, including a Persian-English and a Turkish-English system, which will be demonstrated. We present the architecture of these systems as well as the development methodology.

This report describes the Shiraz project MT prototype for a Persian to English machine translation system using typed feature structures and unification. An overview of the linguistic properties of Persian is presented and the morphological and syntactic grammars developed within the Shiraz project are discussed. The underlying model for the system is a layered chart, capable of representing heterogeneous types of hypotheses in an integrated way.

This report provides a complete descriptive analysis of Persian inflectional morphology from a computational perspective. The parts of speech and the morphemes that appear on them as well as their corresponding morphotactics are presented in detail. The verbal paradigm is also described in this document. Since the morphological analyzer designed for this project uses a unification-based grammar with typed feature structures, the morphological information has been defined in terms of features and values. The report describes the current version of the morphological analyzer used in the Shiraz project and discusses any morphological elements that have not been included in this version, mostly due to the colloquial usage of these morphemes. Sample rules of Samba, the grammar specifying the morphological analyzer, as well as the feature specification for the Persian type definitions module are also described.

The highly ambiguous structure of the Persian Noun Phrase (NP) causes immense difficulties for automatic parsing of written text. Factors contributing to the ambiguity of the Persian NP structure include the lack of overt morphology to mark boundaries, the fact that short vowels are not written in Persian text, a relatively free word order and the optionality of the subject. This paper introduces the constituents forming a Noun Phrase in Persian. The syntax of NPs and relative clauses is described. A closer study of the Noun Phrase provides lexical and morphological clues for determining boundaries. To describe the NP rules in the Shiraz machine translation project, developed at CRL, a unification-based syntactic grammar was used, which operates on typed feature structures.

Prior to morphological analysis or syntactic parsing, a text needs to undergo tokenization, in order to determine sentence and word boundaries. This report describes the tokenizer used in the Shiraz Persian-English machine translation project at the Computing Research Laboratory. The Persian writing system and the methods that can be used in recognizing token boundaries in written text are presented. The system uses a low-level language-independent tokenizer, which outputs an unambiguous sequence of basic tokens. Difficulties arise in analysis of Persian text since certain detachable morphemes need to be reattached to the word before morphological analysis takes place. In addition, words are often concatenated in written form. These pre-processing tasks are accomplished by a post-tokenizer that contains language-specific information.