Teach an open language model better Nepali

Published 12 October 2026 · prices on this page are live

Open models such as Qwen, Gemma and Llama have read some Nepali, but they make grammar mistakes, slip into Hindi and drift into English. Fine-tuning on good Nepali examples is the cheapest way to fix that: with QLoRA, one rented 24 GB GPU is enough for a 7B–14B model.

1. Pick a base model by testing it

Write 20 to 30 prompts that look like your real use, in Nepali, and try two or three instruction-tuned models on them before you train anything. Judge with native speakers: fluency, correct facts, and whether the model answers in Nepali when asked in Nepali. A model that starts better needs less training.

To try models before you fine-tune, rent a GPU with the Ollama + Open WebUI software and chat with them in your browser.

2. Get Nepali training data

• Your own examples are worth the most: real questions from your users, with answers written or checked by a native speaker. A few thousand good pairs go a long way.
• Translated instruction sets help with format and breadth, but machine translation adds errors, so have samples checked.
• Nepali web text, such as the Nepali part of CulturaX on Hugging Face, teaches vocabulary and grammar through continued pre-training, which needs far more text and GPU time than instruction tuning.

Keep everything in Unicode Devanagari, remove duplicates, and check the licence of anything you did not write yourself.

3. Mind the tokens

Most tokenizers split Devanagari into many more tokens per word than English, so the same sentence uses more memory and time. Count your data with the model's own tokenizer and set the maximum sequence length from that, not from English rules of thumb.

4. Train with QLoRA

QLoRA loads the base model in 4-bit and trains small LoRA adapters on top, which is why a 7B–14B model fits on a 24 GB card:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig

base = "Qwen/Qwen2.5-7B-Instruct"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(base, quantization_config=bnb, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(base)
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, target_modules="all-linear", task_type="CAUSAL_LM")

Then train with TRL's SFTTrainer on your examples, formatted with the model's own chat template. Two or three epochs at a learning rate around 2e-4 is a common starting point for LoRA; hold back 5–10% of the examples to watch for overfitting.

Which GPU to rent

A 24 GB card such as the RTX 3090 or RTX 4090 handles QLoRA on 7B–14B models; 48 GB (RTX A6000, L40S) handles 30B-class models; 80 GB cards (A100, H100) are fastest and fit 70B-class QLoRA. Fine-tuning a 7B model with LoRA on about 10,000 examples for three epochs costs, on today's cheapest hosts:

5. Evaluate and ship

Run your test prompts through the base model and the fine-tuned one, and have native speakers compare the answers without knowing which is which. Then serve the model: the vLLM software on PowerPod gives you an OpenAI-compatible API, and Ollama + Open WebUI a chat page in the browser.

Questions

More guides