Which GPU do I need for AI?
Published 12 October 2026 · prices on this page are live
The first question is memory, not speed. If the model and its working data do not fit in the GPU's memory (VRAM), the job fails or crawls; if they fit, a faster card only saves time. Find your job below, then pick the cheapest card with enough memory.
Running language models
A model needs about 2 bytes of GPU memory per parameter at 16-bit precision, 1 byte at 8-bit and a little over half a byte at 4-bit, plus room for the conversation (the KV cache), which grows with context length and the number of users.
| Model size | 16-bit | 8-bit | 4-bit |
|---|---|---|---|
| 7–8B | About 14–16 GB | About 7–8 GB | About 4–5 GB |
| 13–14B | About 26–28 GB | About 13–14 GB | About 8–9 GB |
| 30–34B | About 60–68 GB | About 30–34 GB | About 18–21 GB |
| 70–72B | About 140–144 GB | About 70–72 GB | About 40–42 GB |
These are the weights alone; add 10–30% for context. A 24 GB card runs 7–8B models at 16-bit and 30B-class models at 4-bit; 70B-class models need 48 GB at 4-bit, or 80 GB to be comfortable.
Fine-tuning language models
| Method | 7–8B | 13–14B | 30–34B | 70B |
|---|---|---|---|---|
| QLoRA (4-bit model + adapters) | 12–16 GB; 24 GB comfortable | 24 GB | 48 GB | 80 GB |
| LoRA (16-bit model + adapters) | 24–40 GB | 40–48 GB | 80 GB | Several GPUs |
| Full fine-tuning | About 16 bytes per parameter with the Adam optimiser: a 7B model needs more than 100 GB, so several GPUs | |||
Images and video
| Model | To generate | To train a LoRA |
|---|---|---|
| Stable Diffusion 1.5 | 4–6 GB | 8–12 GB |
| SDXL | 8–12 GB | 16–24 GB |
| FLUX.1 | About 24 GB at 16-bit; 12–16 GB with 8-bit or quantised weights | 24 GB with 8-bit weights; 48 GB comfortable |
| Video models | 24 GB and up; the largest want 48–80 GB | 48–80 GB |
Speech
- Whisper large-v3 needs about 10 GB to transcribe, large-v3-turbo about 6 GB and whisper-small about 2 GB.
- Fine-tuning Whisper large-v3 with LoRA works on 16 GB with small batches; 24 GB is comfortable. See Nepali speech-to-text.
- Fine-tuning a text-to-speech model of about half a billion parameters is comfortable on 24 GB. See Nepali text-to-speech.
The cheapest card for each memory size today
| Memory | Cheapest GPU today | USD | INR | NPR |
|---|---|---|---|---|
| 8 GB or more | GTX 1080 | USD 0.08 | INR 8 | NPR 12 |
| 12 GB or more | RTX 3060 | USD 0.10 | INR 10 | NPR 16 |
| 16 GB or more | Tesla V100 | USD 0.14 | INR 13 | NPR 21 |
| 24 GB or more | Tesla V100 | USD 0.14 | INR 13 | NPR 21 |
| 48 GB or more | A100 SXM4 | USD 0.65 | INR 63 | NPR 99 |
| 80 GB or more | A100 SXM4 | USD 0.65 | INR 63 | NPR 99 |
Beyond memory: what makes a card faster
- Memory bandwidth matters most for running language models: datacenter cards with HBM memory, such as the A100 and H100, generate text much faster than consumer cards running the same model.
- Tensor cores speed up training and 16-bit or 8-bit maths. GTX cards, including the GTX 16 series, do not have them, so they are cheap but slow for modern AI.
- Newer number formats: Ada and Hopper cards support FP8, and Blackwell adds FP4, which speed up models built to use them.
- Every GPU page shows a typical machine's memory bandwidth, compute and CUDA version.
Questions
How much GPU memory do I need to run Llama 3 8B?
About 16 GB for the weights at 16-bit, or about 5 GB with 4-bit weights, plus room for context. A 24 GB card runs it at 16-bit with room to spare.
Can I use two smaller GPUs instead of one big one?
Many frameworks can split a model across GPUs, but one card with enough memory is simpler and usually faster.
Is a gaming GPU good for AI?
Yes. GeForce cards such as the RTX 3090 and RTX 4090 have 24 GB of memory and fast tensor cores, and they are the best value for fine-tuning and for running models up to about 30B parameters with 4-bit weights.