NVIDIA Launches Nemotron-pretraining-Indic-v1: 2.8T Tokens, 22 Indian Languages

Executive Overview

NVIDIA, in collaboration with AI4Bharat at IIT Madras, has announced Nemotron-pretraining-Indic-v1, the largest pretraining dataset for Indian languages, comprising 2.8 trillion tokens across 22 Indian languages and 34 text configurations. The release includes an open-source, GPU-accelerated data generation pipeline and is being published on Hugging Face. It targets 97% of India's population (approximately 1.2 billion citizens according to the 2011 census). Continuous pretraining on Nemotron 3.5 Super VL yielded improvements of 4.98 to 9.82 percentage points on five Indic knowledge and reasoning benchmarks.

Deep-Dive Features & Architecture

The dataset is split into 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, according to the Gemma-4 tokenizer count. It covers 34 language-text configurations, with 12 languages available in both native and romanized scripts and 10 languages in native script only. The primary data source is 15 subsets of Nemotron-CC-v2, including web text, Q&A, math, and code.

The pipeline operates in four stages: full document translation, quality evaluation using the MQM framework (an open standard for translation error classification), repair of low-scoring outputs, and conversion of a subset to romanized script. Gemma-4-31B-it was used for translation and evaluation, while Indic Translate from Bodhan AI (a non-profit center incubated at IIT Madras) was used for the other ten languages. A total of 438.4 billion tokens were recovered through repair operations, already included in the dataset total.

Benchmark & Competitive Performance

Continuous pretraining (CPT) for Nemotron 3.5 Super VL added approximately 10 billion Indian tokens out of a 100 billion token budget (10%), while preserving 90% of the previous training mix. The model showed improvement on all five reported Indic knowledge and reasoning benchmarks: MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy, and Indic GSM8K, with gains ranging from 4.98 to 9.82 percentage points. It also outperformed Nemotron 3 Super on five of six task benchmarks covering knowledge, reasoning, math, coding, and translation, with a slight decline of 2.04 points on IFEval 1.5. The model improved across all four IndiVibe dimensions: helpfulness, fluency, language and script accuracy, and appropriate verbosity.

Industry Impact & Enterprise Adoption

Although Nemotron-pretraining-Indic-v1 is exclusively for the 22 Indian languages and does not support Arabic, its methodology offers direct lessons for Arabic developers. The open pipeline (NeMo Data Designer + Gemma-4-31B-it) is adaptable to any language, opening the door for Arabic development teams to build similar datasets for Arabic and its dialects. The pricing model here is completely free via Hugging Face, reducing development costs for startups in the Arab world to nearly zero. The quality threshold of 75/100 per MQM and full provenance tracking represent a standard that can be replicated to ensure data quality. This release sets a new benchmark for open-source, large-scale multilingual dataset creation, with potential ripple effects across low-resource language communities globally.

Conclusion

NVIDIA and AI4Bharat's Nemotron-pretraining-Indic-v1 represents a significant leap forward for Indian language AI, delivering 2.8 trillion tokens, 22 languages, and a fully open, GPU-accelerated pipeline. The measurable gains on Indic benchmarks demonstrate the value of targeted pretraining. While Arabic is not supported, the open methodology and free availability provide a blueprint for other language communities. The dataset is releasing soon on Hugging Face, with the pipeline already open-source.

Media Source: Hugging Face | البيان الرسمي للشركة: المصدر الأصلي | Fact Verification & Analysis: AI Tools Oasis