Afaan Oromo PartofSpeech Tagging using Hidden Markov Models

Afaan Oromo (also known as Oromo) is the most widely spoken Cushitic language in East Africa, with over 35million native speakers in Ethiopia, Kenya, and Somalia. Like many underresourced languages, reliable linguistic tools such as partofspeech (POS) taggers are scarce. This page explains how a classic statistical approachHidden Markov Models (HMMs)can be employed to build an effective POS tagger for Afaan Oromo.

Why POS Tagging Matters for Afaan Oromo

POS tagging assigns a grammatical category (noun, verb, adjective, etc.) to each word in a sentence. The information is essential for downstream tasks:

• Machine translation and texttospeech synthesis.
• Information extraction, sentiment analysis, and question answering.
• Linguistic research on morphosyntax and language teaching.

Afaan Oromo is an agglutinative language with rich suffixation. A single surface form can encode tense, aspect, mood, case, and agreement, making lexical ambiguity a real challenge. An HMMbased tagger can capture systematic patterns in these sequences without requiring deep linguistic resources.

Hidden Markov Models in a Nutshell

An HMM is a probabilistic model consisting of:

1. Hidden statesthe POS tags we want to predict.
2. Observationsthe actual words that appear in the text.
3. Transition probabilities\(P(t_i \mid t_{i-1})\) that describe how likely a tag follows another tag.
4. Emission probabilities\(P(w_i\mid t_i)\) that describe how likely a word is generated by a given tag.

The model assumes a firstorder Markov property: the current tag depends only on the previous tag, and the observed word depends only on its own tag. Decodingfinding the most probable tag sequence for a sentenceis performed with the Viterbi algorithm.

Data Preparation

Because annotated corpora for Oromo are limited, researchers typically start from one of the following resources:

• Afaan Oromo Treebank (small but manually annotated with Pennstyle tags).
• Parallel corpora (e.g., EnglishOromo subtitles) that can be projected using bilingual alignments.
• Webcrawled texts that are automatically preprocessed and then manually corrected for a subset.

Regardless of the source, the preparation steps are the same:

1. Tokenize the text (respecting clitics such as 'ni' for progressive aspect).
2. Map each token to a coarse tag set (e.g., Noun, Verb, Adj, Adv, Pron, Det, Prep, Conj, Punct).
3. Split the data into training (80%), development (10%), and test (10%) sets.

Estimating Model Parameters

Transition probabilities

From the tagged training data, count tag bigrams:

count(t_{i-1}, t_i) = number of times tag t_i follows tag t_{i-1}P(t_i | t_{i-1}) = count(t_{i-1}, t_i) / count(t_{i-1})

Smoothing (e.g., addone or KneserNey) is crucial because the training set is small and many tag pairs will be unseen.

Emission probabilities

Similarly, count how often each word appears with each tag:

count(w, t) = number of times word w is tagged as tP(w | t) = count(w, t) / count(t)

To handle outofvocabulary (OOV) words, use an UNKNOWN token and estimate P(UNKNOWN|t) from lowfrequency words (e.g., those occurring once).

Decoding with the Viterbi Algorithm

The Viterbi dynamic programming routine finds the tag sequence \(\hat{t}_1^n\) that maximizes:

argmax_{t_1^n} _{i=1}^{n} P(w_i|t_i)P(t_i|t_{i-1})

Implementation details:

• Work in logspace to avoid underflow.
• Maintain backpointers for each position to reconstruct the optimal path.
• Initialize with a special start tag START and end with END.

Evaluating the Tagger

Standard metrics are:

• Accuracypercentage of correctly tagged tokens.
• Precision/Recall/F1 per tag class (useful for lowfrequency categories).

On a modest Oromo test set (5000 tokens), a wellsmoothed HMM typically reaches 8590% accuracy, comparable to early English HMM taggers. Errors often involve:

• Ambiguous suffixes (e.g., -aa can mark both plural nouns and verb infinitives).
• Rare verb forms not observed in training.
• Incorrect handling of clitic boundaries.

Improving the Baseline HMM

While a pure HMM is simple and fast, several extensions can boost performance for Oromo:

1. Higherorder Markov models (trigrams) capture longer dependencies such as SubjectVerbObject patterns.
2. Morphological preprocessingsegmenting affixes before tagging reduces sparsity. Tools like Adamor can provide morpheme boundaries.
3. Featurerich emissionsinstead of wordonly likelihoods, combine word form, suffix, prefix, and orthographic features within a Maximum Entropy HMM.
4. Hybrid modelsuse a Conditional Random Field (CRF) or a neural sequence tagger to rescore the topk HMM hypotheses.
5. Semisupervised learningapply the BaumWelch (EM) algorithm on large unlabeled Oromo corpora to refine probabilities.

Sample Code (Python)

The following minimal example demonstrates training and decoding with the hmmlearn library. It assumes you have a list of sentences where each sentence is a list of (word, tag) tuples.

import numpy as npfrom collections import defaultdictfrom math import log# ---------- 1. Build vocabularies ----------word_counts = defaultdict(int)tag_counts  = defaultdict(int)bigram_counts = defaultdict(int)emit_counts = defaultdict(int)sentences = [...]   # list of [(w,t), (w,t), ...]for sent in sentences:    prev_tag = 'START'    for w,t in sent:        word_counts[w] += 1        tag_counts[t] += 1        emit_counts[(t,w)] += 1        bigram_counts[(prev_tag,t)] += 1        prev_tag = t    bigram_counts[(prev_tag,'END')] += 1# ---------- 2. Probabilities with Laplace smoothing ----------V = len(word_counts) + 1          # +1 for UNKNOWNT = len(tag_counts) + 1           # +1 for ENDdef trans_prob(prev, cur):    return (bigram_counts[(prev,cur)] + 1) / (tag_counts[prev] + T)def emit_prob(tag, word):    if (tag, word) in emit_counts:        return (emit_counts[(tag,word)] + 1) / (tag_counts[tag] + V)    else:        # treat as UNKNOWN        return 1 / (tag_counts[tag] + V)# ---------- 3. Viterbi ----------def viterbi(words):    tags = list(tag_counts.keys())    n = len(words)    v = np.full((len(tags), n), -np.inf)    bp = np.zeros((len(tags), n), dtype=int)    # initialization    for i,tag in enumerate(tags):        v[i,0] = log(trans_prob('START', tag)) + log(emit_prob(tag, words[0]))    # recursion    for j in range(1,n):        for i,cur in enumerate(tags):            best_score = -np.inf            best_k = 0            for k,prev in enumerate(tags):                score = v[k,j-1] + log(trans_prob(prev, cur)) + log(emit_prob(cur, words[j]))                if score > best_score:                    best_score = score                    best_k = k            v[i,j] = best_score            bp[i,j] = best_k    # termination    best_last = np.argmax([v[i,n-1] + log(trans_prob(tags[i],'END')) for i in range(len(tags))])    best_path = [tags[best_last]]    for j in range(n-1,0,-1):        best_last = bp[best_last,j]        best_path.append(tags[best_last])    best_path.reverse()    return best_path# Example usagesentence = ['ani', 'barii', 'bultii', 'dhufaa', 'jira']print(viterbi(sentence))

The script is deliberately simple: it uses addone smoothing, treats OOV words as UNKNOWN, and works with a flat tag set. For production use, replace the naive counts with more sophisticated smoothing and add featurebased emissions.

Future Directions

Research on Oromo POS tagging is moving beyond pure HMMs:

• Neural architectures (BiLSTMCRF, Transformers) have shown >95% accuracy when trained on augmented corpora.
• Crosslingual transferleveraging wellannotated languages such as Amharic or Swahili through multilingual embeddings.
• Active learningquerying native speakers for the most informative sentences reduces annotation cost.
• Integration with morphological analyzersjoint models that simultaneously segment and tag can resolve many of the ambiguities that pure HMMs struggle with.

Nevertheless, the HMM remains a valuable baseline. It is fast, interpretable, and requires only a modest amount of annotated dataattributes that are attractive for lowresource language communities and for educational purposes.

Key Takeaways

1. Afaan Oromos rich morphology makes POS tagging challenging but not intractable.
2. An HMM models tag transitions and word emissions, providing a probabilistic framework that works well even with limited training data.
3. Proper smoothing, handling of OOV words, and basic morphological preprocessing dramatically improve accuracy.
4. Extensions such as higherorder models, featureenhanced emissions, and semisupervised learning can push performance close to modern neural methods.

By implementing and experimenting with the HMM approach described here, researchers and developers can create a solid foundation for further NLP work on Afaan Oromo and contribute to the broader effort of supporting African languages in digital technologies.