PowerPod GPUsUse casesCase studiesGuidesBlogFor collegesHelp Sign inCreate account

Fine-tune Llama 3.1 8B on your own data

Published 27 October 2026 · prices on this page are live

GPU prices in India (INR) · Nepal (NPR) · across Asia · everywhere else.

Llama 3.1 8B Instruct is a good first model to fine-tune: small enough for one 24 GB GPU with QLoRA, and strong enough to learn your format, your tone or your field. The same steps work for Qwen, Gemma and newer Llama models.

Fine-tune, or give the model your documents?

Fine-tuning changes how a model answers: the format, the style, the vocabulary of your field, a classification it should make. If the model must know facts that change, such as prices or your documents, retrieval (RAG) is usually the better tool; many projects use both.

Pick a method and a GPU

MethodGPU memory for 8BGood cards
QLoRA: the model in 4-bit, small adapters trained on topabout 10–16 GB, by sequence length and batch16–24 GB: RTX 4090, RTX 3090
LoRA in 16-bitabout 20–30 GB40–48 GB: L40S, RTX A6000, A100
Full fine-tuningabout 16 bytes per parameter with Adam, so 130 GB or moreSeveral 80 GB GPUs; not covered here

QLoRA is where most people start: it costs the least and, for teaching a format or a style, comes close to full fine-tuning.

What a run costs

LoRA on about 10,000 examples for three epochs, estimated for a 7B model; an 8B model takes a little longer. On today's cheapest hosts:

GPUTypical timeUSDINRNPR
A100 SXM42 hUSD 1.30INR 122NPR 192
A100 PCIe2.2 hUSD 1.43INR 135NPR 212
RTX 30905 hUSD 1.50INR 140NPR 220
RTX 40903 hUSD 1.80INR 168NPR 264
RTX 50902.5 hUSD 1.88INR 175NPR 278

1. Prepare your data

Write your examples as conversations, one per line in a JSONL file, each with a list of messages: an optional system message, the user's turn and the answer you want. A few hundred to a few thousand clean, consistent examples teach a format or a tone; quality matters far more than quantity. Hold back 5–10% to test on.

2. Rent the GPU

Choose the PyTorch + JupyterLab software and a 24 GB GPU, with about 60 GB of disk for the model, your data and checkpoints. Open JupyterLab from the console.

3. Get the model

Llama 3.1 is gated on Hugging Face: accept Meta's licence on the model page, create an access token, and log in on the machine with hf auth login (or huggingface-cli login on older versions). Then install the training libraries with pip install -U transformers peft trl bitsandbytes datasets.

4. Load it for QLoRA

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig

base = "meta-llama/Llama-3.1-8B-Instruct"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(base, quantization_config=bnb, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(base)
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, target_modules="all-linear", task_type="CAUSAL_LM")

Then train with TRL's SFTTrainer on your JSONL file: it applies Llama's chat template to the messages. Two or three epochs at a learning rate around 2e-4 is a common start for LoRA, with a batch that fits the card and gradient accumulation for the rest. Watch the loss on your held-back examples: when it stops falling, stop.

5. Test it and run it

Llama 3.1 comes with Meta's Llama 3.1 Community License: read it before you use the model, or your fine-tune, in a product.

Create accountSee all GPU prices

Questions

Can I fine-tune Llama 3.1 8B on a 16 GB GPU?

Yes, with QLoRA, sequences up to about 2,048 tokens and a small batch with gradient accumulation. A 24 GB card gives more room for longer examples.

How many examples do I need?

To teach a format or a tone, a few hundred to a few thousand good examples often do. Consistent, correct examples matter more than a large, noisy set.

How long does it take?

About two to five hours on a 24 GB card for 10,000 examples and three epochs of LoRA, and about an hour on an H100.

Can I use Qwen or Gemma instead?

Yes. The same steps work; pick the base model that does best on your own test prompts before you train.

Case study: Train a chatbot on your own data: cost and steps, with the steps and what it costs today.

More guides

Build Nepali speech-to-text with WhisperMake a Nepali text-to-speech voiceTeach an open language model better NepaliWhich GPU do I need for AI?Google Colab alternatives: when to switch, and what it costsThe cheapest cloud GPUs right nowGPUs for students: the free options and the cheap onesRun ComfyUI on a cloud GPUH100 price per hour, cloud by cloudA100 price per hour, cloud by cloudHow to rent a GPU online, step by stepWhat can you do with USD 1, USD 5 or USD 20 of GPU time?IndiaAI compute portal: who qualifies, what it costs, how to applyWhat do H100, H200, A100 and T4 cards cost to buy, and to rent?