Fine-tune Llama on an H100: what it costs in rupees, and when the big card is cheaper
Tutorial · Published 2 November 2026 · prices on this page are live
The H100 is the card everyone asks about and the one most people assume they cannot afford. For a fine-tune, that assumption is often wrong: the H100 costs several times an RTX 4090 per hour, but it finishes the same LoRA run in a fraction of the time, and you pay by the second. Here are today’s numbers in rupees, and a step-by-step run.
The same job on three cards
A LoRA fine-tune of Llama 3.1 8B on about 10,000 examples for three epochs, the most common first project, on today’s cheapest hosts:
| GPU | Typical time | USD | INR | NPR |
|---|---|---|---|---|
| A100 SXM4 | 2 h | USD 1.30 | INR 122 | NPR 192 |
| A100 PCIe | 2.2 h | USD 1.43 | INR 135 | NPR 212 |
| RTX 3090 | 5 h | USD 1.50 | INR 140 | NPR 220 |
| RTX 4090 | 3 h | USD 1.80 | INR 168 | NPR 264 |
| RTX 5090 | 2.5 h | USD 1.88 | INR 175 | NPR 278 |
Read the table as a total, not as an hourly rate. The RTX 4090 is the cheapest per hour and the slowest; the H100 is the most expensive per hour and the fastest. For a run this size the totals land close together, and the H100 gives you the result in your lunch break. Where the H100 wins outright is bigger models and longer context: a 70B model with QLoRA does not fit a 24 GB card at all.
Hourly, for reference
| GPU | Memory | USD | INR | NPR |
|---|---|---|---|---|
| RTX 4090 | 24 GB | USD 0.60 | INR 56 | NPR 88 |
| A100 SXM4 | 80 GB | USD 0.65 | INR 61 | NPR 96 |
| H100 SXM | 80 GB | USD 3.74 | INR 349 | NPR 556 |
Step by step
- Prepare the data on your laptop. A JSONL file, one example per line: an instruction, optionally an input, and the answer you want. Ten thousand clean examples beat a hundred thousand scraped ones. Keep a few hundred aside to test with.
- Rent the card with PyTorch + JupyterLab. Create an account, add credits (from USD 5.00, INR 200 or NPR 200), pick PyTorch + JupyterLab and the card from the live board. For an 8B model: 24 GB for QLoRA, 48 GB for full LoRA with long context, 80 GB for speed or for 70B-class models.
- Install the trainer and log in to Hugging Face. In a notebook cell:
pip install -U "transformers>=4.45" peft trl datasets bitsandbytes accelerate huggingface-cli login # accept the Llama licence on huggingface.co first - Upload the data and train. Drag the JSONL file into JupyterLab, then run a TRL SFT script with LoRA rank 16, learning rate 2e-4, batch size to fill the memory, three epochs. The H100 handles bf16 and larger batches; on a 24 GB card load the base model in 4-bit (QLoRA).
- Test, then save. Generate answers for the examples you held back and compare with the base model. Save the adapter (a few hundred megabytes) and download it, or merge it into the base weights for serving.
- Stop the machine. Billing ends at once; keep the disk if you plan another run.
When the H100 is the cheaper choice
- The model does not fit otherwise. 70B-class models with QLoRA need 48 GB; full LoRA on 13B-class models with long context needs 48 to 80 GB.
- Your time is the cost. A run that takes three hours on an RTX 4090 and under an hour on an H100 frees your afternoon for the next experiment.
- Many runs. Hyperparameter sweeps multiply the hours; the fast card multiplies the savings.
- Otherwise, the RTX 4090. For one small fine-tune with no deadline, the cheapest 24 GB card is the cheapest total.
The full fine-tuning guide has the complete script and the settings for each memory size; the H100 price guide compares this price with the big clouds.
Questions
How long does fine-tuning Llama 3.1 8B take on an H100?
A LoRA run on about 10,000 examples for three epochs takes roughly an hour on an H100 SXM, two hours on an A100 and three hours on an RTX 4090. More data, more epochs or longer examples scale the time up.
Can I fine-tune Llama 70B?
With QLoRA, yes, on a 48 GB card (RTX A6000, L40S) or an 80 GB card (A100, H100). Expect many hours and a larger adapter; serve the result on a 48 GB card with 4-bit weights.
What does an H100 cost per hour in rupees?
An H100 SXM starts at USD 3.74 an hour on PowerPod today; the table above has the same price in rupees. The price is billed by the second and locked when you rent.
Do I need to accept the Llama licence?
Yes. Accept it once on Hugging Face under your account, then log in from the notebook with huggingface-cli; the download works after that.