Marathi, spoken by over 83 million people worldwide, has long been overlooked in natural language processing research, but a new dataset and trained models are beginning to change that. Researchers at L3 Cube Labs, PICT Pune, and IIT Madras have released L3Cube-MahaPOS, a gold-standard dataset for part-of-speech tagging in Marathi, addressing a fundamental gap in computational linguistics for Indian languages.

Part-of-speech tagging is a foundational task in natural language processing (NLP), the field that teaches computers to understand human language. By assigning grammatical categories like noun, verb, and adjective to every word in a sentence, POS tagging provides the structural foundation that downstream AI systems rely on for machine translation, information extraction, and question-answering.

Why Has Marathi Lagged So Far Behind in AI Language Resources?

Despite being the official language of Maharashtra and ranking among the top twenty most spoken languages globally, Marathi remains severely under-resourced in the world of computational linguistics. English, French, and Mandarin have achieved part-of-speech tagging accuracy above 97 percent, supported by massive treebanks and pre-trained language models. Marathi, by contrast, has received comparatively little attention from researchers.

The language presents unique computational challenges that make it harder to process with standard AI models. Marathi is morphologically rich, meaning a single verb can take dozens of inflected forms to express tense, aspect, mood, number, gender, and honorific level simultaneously. The language also exhibits relatively free word order, lacks capitalization conventions that English-based systems exploit, and experiences pervasive code-mixing with Hindi and English in contemporary written text.

What Makes This New Dataset Different?

The L3Cube-MahaPOS dataset comprises 32,354 manually annotated sentences drawn from real-world news text. Every sentence was annotated entirely by hand by a team of Marathi-proficient annotators following a 16-tag scheme aligned with Universal Dependencies, an international standard that ensures the dataset can be used alongside resources for other languages.

The researchers implemented a rigorous quality-assurance process that included Unicode normalization, Devanagari-aware tokenization, and noise filtering to ensure label consistency across all data splits. This structured preprocessing pipeline sets a reproducible standard that future corpus builders can adapt for other under-resourced languages.

How Well Do Current AI Models Perform on Marathi?

The research team benchmarked the dataset across six different model families, ranging from classical statistical approaches to state-of-the-art transformer-based architectures. The models tested included:

• HMM (Hidden Markov Models): Classical probabilistic sequence models that learn patterns from annotated data without hand-crafted rules.
• CRF (Conditional Random Fields): Discriminative models that combine diverse contextual features for improved accuracy over generative approaches.
• BiLSTM (Bidirectional Long Short-Term Memory): Neural networks that process text in both directions to capture context from surrounding words.
• BiLSTM+CharCNN: A hybrid approach combining bidirectional neural networks with character-level convolutional layers to handle morphological complexity.
• MuRIL: A multilingual transformer model designed for Indian languages.
• MahaBERT-v2: A Marathi-specific transformer model fine-tuned for the language.

The best-performing system achieved 88.67 percent token-level accuracy and a macro-F1 score of 81.67 percent across 15 evaluated tag classes. While this represents solid performance, it also highlights the remaining challenge: the gap between Marathi's 88.67 percent accuracy and English's 97 percent-plus accuracy underscores how much work remains to bring under-resourced languages to parity with high-resource languages.

How to Leverage This Dataset for Marathi NLP Development

Researchers and developers working on Marathi language technology can now take several concrete steps to advance the field:

• Fine-tune existing models: Use the L3Cube-MahaPOS dataset to adapt pre-trained multilingual or Marathi-specific transformer models like MahaBERT-v2 for downstream tasks such as named entity recognition, sentiment analysis, and machine translation.
• Build domain-specific taggers: Extend the news-based dataset with annotations from social media, technical documentation, or literary texts to create specialized POS taggers for different writing styles and contexts.
• Develop morphological analyzers: Leverage the annotated morphological information to build tools that can decompose Marathi words into their constituent parts, improving handling of inflected forms and rare words.
• Address code-mixing challenges: Use the dataset as a foundation for training models that can accurately tag mixed-language text, a critical capability for real-world Marathi NLP applications.

The research team has released the dataset, annotation guidelines, and trained model checkpoints publicly to foster further research in Marathi NLP. This open-source approach mirrors successful practices in high-resource language communities and signals a commitment to building computational linguistics infrastructure for Indian languages.

What Does This Mean for the Broader NLP Community?

The L3Cube-MahaPOS project addresses a critical imbalance in AI language resources. Prior computational work on Marathi has concentrated on tasks with immediate societal applications, such as sentiment analysis and named entity recognition, but foundational infrastructure like POS tagging datasets remained absent. By establishing this baseline, the researchers create a platform for building more sophisticated language understanding systems.

The detailed error analysis provided in the research reveals the principal sources of tagging difficulty in Marathi, offering guidance for future modeling efforts. This transparency helps the research community understand not just what works, but why certain approaches succeed or fail for morphologically rich, low-resource languages.

As AI systems become increasingly central to how people access information and communicate across languages, ensuring that under-resourced languages receive comparable investment in foundational datasets becomes a matter of both technical progress and linguistic equity. The L3Cube-MahaPOS dataset represents a meaningful step toward that goal, demonstrating that large-scale, high-quality language resources are achievable even for languages that have historically received minimal computational attention.