Unsupervised Learning of Morphology: Survey, Model, Algorithm and Experiments
Licentiate thesis, 2007
This thesis contains work on a specific problem in field of Language
Technology. The problem can be described as follows:
"Can a computer extract a description of word conjugation in a natural
language using only written text in the language?"
The problem is often referred to as Unsupervised Learning of
Morphology (ULM) and has a wide variety of Language Technology
applications, including Machine Translation, Document Categorization
and I nformation Retrieval. The ULM problem is also relevant for
linguistic theory, and can serve to boost empirical investigations in
subfields as Quantitative Linguistics and Linguistic Typology,
The first part of the thesis contains a comprehensive survey of work
done on the ULM problem. All the minor and major lines of work are
mentioned with a reference and a very brief characterization.
Different approaches that have been prevalent in the field as a whole
are highlighted and critically discussed. The general picture
resulting from the survey is that much work has been repeated over and
over, with little exchange and evolution of techniques.
The second part of the thesis describes a simple model of
concatenative affixation, i.e., how stems and affixes are stringed
together to form words. The model says that words consist of
high-frequency strings (``affixes'') attached to low-frequency strings
(``stems''), e.g., as in the English play-ing. Then it is
shown that from a set words constructed according to the model, the
affixes can be extracted with their correct segmentation. The
algorithm for extraction is impressionistically evaluated on a diverse
set of natural languages.
The affix extraction algorithm does not output a full-fledged description of
conjugational patterns -- it only produces a list of affixes. The third and
fourth parts of the thesis show how it can be used in further morphological
In the third part, an algorithm is presented that decides
if two given words are conjugations of the same stem. The key part is
the development of a metric for quantifying which endings tend to attach
to the same set of stems. The algorithm has no parameters or human input
and works equally well for languages with widely different morphological
typology. It achieves almost perfect accuracy on word pairs selected from
In the fourth part, the affix extraction model is exploited for the
written language identification problem, i.e., to decide which natural
language a given text is written in. Existing state-of-the-art
techniques to identify the language of a written text most often use a
3-gram frequency table as basis for 'fingerprinting' a language. While
this approach performs very well in practice (99\%-ish accuracy) if
the text to be classified is of size, say, 100 characters or more, it
cannot be reliably used to classify even shorter input, nor can it
detect if the input is a concatenation of text from several languages.
Therefore a more fine-grained model is presented which aims at reliable
classification of input as short as one word. In essence, the
language of an unseen word is guessed based on any salient affixes
that appear on it. Many practical applications do not need this fine
level of granularity, but Multilingual Information Retrieval is a
major target area where input is usually only one or a few words.
The algorithm is given a rigorous evaluation on a 32-language parallel
bible corpus showing competitive accuracy on short input as well as
multi-lingual input, and not only for a set of European languages with
similar morphological typology.