Train a chatbot on your own data: 10,000 support tickets for USD 1.30 of GPU time
Example project: fine-tune an open model on 10,000 support tickets and serve it as a private API, with today's GPU prices for every step.
- Fine-tuning Llama 3 8B on 10,000 support tickets takes about 2 hours of GPU time: USD 1.30 at today's lowest price.
- A 24 GB GPU is enough for an 8B model with LoRA; you do not need an H100.
- Serving it through office hours costs about USD 52.80 a month, and nothing at night.
The challenge
A small software company answers the same customer questions every day. General chatbots give vague answers because they do not know the product, and pasting tickets into a closed AI service raises privacy questions.
Why a rented GPU
- A fine-tune runs for a few hours. A GPU bought for it would sit idle the rest of the month.
- The tickets and the trained model stay on a machine you rent and control, and they are deleted with it.
How it works, step by step
- Export about 10,000 solved tickets as question and answer pairs (a JSONL file), with names and email addresses removed.
- Rent a 24 GB GPU with the PyTorch + JupyterLab software and fine-tune Llama 3 8B with LoRA. The fine-tuning guide walks through it.
- Test the answers on tickets the model has not seen, and fix the training data where it goes wrong.
- Serve the result with vLLM, which answers through an OpenAI-compatible API, and point your help desk at it.
- Stop the machine when you are done: a stopped machine pays only for its disk.
Which GPU to choose
The cheapest way today is A100 SXM4: 2 hours for USD 1.30.
In a hurry, H100 SXM does it in 1.2 hours for USD 4.49.
A 13B model needs about 40 GB with LoRA, and a 70B model 80 GB or more; check the memory before you rent.
What it costs today
| Step | GPU | Time | Cost |
|---|---|---|---|
| Fine-tune on 10,000 tickets (LoRA, 3 passes) | A100 SXM4 | 2 h | USD 1.30 |
| Try it with the team for an evening (vLLM) | RTX 3090 | 4 h | USD 1.20 |
| Run it through office hours for a month (8 h × 22 days) | RTX 3090 | 176 h | USD 52.80 |
What you end up with
- A private model that answers in your product's own words.
- An API your help desk or website widget can call, on hardware nobody else shares while you rent it.
- Fine-tune once, then pay only for the hours the model is online.
Common mistakes to avoid
- Training on tickets that still contain customers' names and email addresses: remove them first.
- Renting a GPU with too little memory, so the run stops halfway: 24 GB for 8B, 80 GB for 70B.
- Leaving the machine running after the fine-tune: set a timer when you rent.
Keep the bill where you want it
- Billing by the second; a stopped machine pays only for its disk.
- Daily and monthly spending limits, a timer on each machine and an idle stop.
- Prices and payment in your currency, by card.
Questions
How much does it cost to train a chatbot on your own data?
On PowerPod today, fine-tuning an 8B model on 10,000 support tickets costs about USD 1.30 of GPU time on A100 SXM4. Serving it costs only the hours it is online.
How much data do I need to fine-tune an LLM?
A few thousand good examples already help; 10,000 solved tickets is a solid start. Clean, correct answers matter more than the number.
Is our data safe on a rented GPU?
The machine is yours while you rent it, and its disk is deleted when you destroy it. The hosts are verified data centres and operators; for confidential data pick a data-centre host and encrypt what you upload.