Self-hosted LLM API with vLLM: USD 52.80 a month of office hours
Example project: an agency serves an open model behind an OpenAI-compatible API for a client app, with monthly costs from today's prices and the rent-or-buy sum.
- A 13B model on vLLM costs about USD 0.30 an hour on RTX 3090: USD 52.80 for a month of office hours.
- vLLM answers through the same API shape as OpenAI's, so the app code barely changes.
- One RTX 4090 at its launch price would pay for about 30 months of office-hours serving.
The challenge
An agency builds an AI feature for a client who will not send customer data to a closed AI service. The client wants a private model, a predictable bill, and no hardware to look after.
Why a rented GPU
- vLLM serves open models such as Llama, Qwen or Gemma behind the same API shape as OpenAI's, so the app code barely changes.
- Renting by the second means the model runs during the client's working hours and costs nothing at night.
How it works, step by step
- Rent a 24 GB or larger GPU with the vLLM software and choose the model.
- Point the client's app at the machine's API address with an API key.
- Set a timer or a daily spending limit so it stops after office hours, and start it again in the morning (the disk keeps the model).
- Keep a fallback for the rare hour a host is down: a second machine, or a hosted API.
Which GPU to choose
The cheapest way today is RTX 3090 at about USD 0.30 an hour.
More users at once need more GPU memory for the cache: move up to a 48 GB card before you add a second machine.
What it costs today
| Step | GPU | Time | Cost |
|---|---|---|---|
| An evening of testing with the client | RTX 3090 | 4 h | USD 1.20 |
| Office hours for a month (8 h × 22 days) | RTX 3090 | 176 h | USD 52.80 |
| Around the clock for a month | RTX 3090 | 720 h | USD 216.00 |
What you end up with
- Buying one RTX 4090 at NVIDIA's launch price (USD 1,599.00, before import duty, tax and the rest of the computer) would pay for about 30 months of office-hours serving.
- A private model the client can audit, with no data sent to a closed AI service.
- A bill that scales with use: pause it any day it is not needed.
Common mistakes to avoid
- Running it around the clock when the app is used in office hours: a timer halves the bill.
- Exposing the API without a key: always set one.
- Promising 100 % uptime on one machine: hosts can go offline, so plan a fallback.
Keep the bill where you want it
- Billing by the second; a stopped machine pays only for its disk.
- Daily and monthly spending limits, a timer on each machine and an idle stop.
- Prices and payment in your currency, by card.
Questions
How much does a self-hosted LLM cost?
About USD 0.30 an hour for a 13B model on vLLM on PowerPod today: USD 52.80 for a month of office hours, USD 216.00 around the clock.
Is there an uptime guarantee?
No. Hosts are independent and can go offline, and billing pauses when they do. For a production app keep a fallback: a second machine or a hosted API for the rare hour a host is down.
Can we automate it?
Yes. API keys with scopes let a script rent, start, stop and destroy machines; see the API reference.