Local or hosted: where the crossover actually is

The break-even between a GPU you own and an API you rent is a division, not an argument. Here is the calculation, with the terms people leave out.

2026-06-09 · 3 min read

"Self-hosting is cheaper" and "just use the API" are both defensible, and which one is true for you is a division you can do in ten minutes.

The calculation

Hosted cost is a rate:

hosted monthly = requests × (tokens_in × rate_in + tokens_out × rate_out)

Local cost is a fixed part plus a small variable part:

local monthly = hardware ÷ amortisation_months + electricity + your_time

Break-even is where they meet. The terms people forget are all in the second equation.

Hardware, amortised. A £900 card over 24 months is £37.50/month. Pick a real horizon and stick to it — "it's a one-off cost" is how a £900 purchase disappears from a comparison it should dominate.

Electricity, both states. Not just inference. A machine that idles at 70W so it can answer within a second costs about £14/month at 28p/kWh before it does any work at all. Add the load draw on top — measure both with a plug meter, because the manufacturer's TDP is not what the wall sees.

Your time. The honest killer. Driver updates, a runtime that changes its quantisation format, the model that stops loading after a kernel upgrade. Two hours a month is conservative. At any consulting rate at all, this term is larger than the electricity and often larger than the hardware.

A worked comparison

A classification task: 800 tokens in, 60 out, on a small hosted model at roughly £0.12 per million input and £0.45 per million output.

Per request: 800 × 0.12/1e6 + 60 × 0.45/1e6 = £0.000123. Call it £0.12 per thousand requests.

Local: an 8B model at Q4 on a £900 card, 2 hours/month of maintenance at £70:

hardware   £900 ÷ 24        = £37.50
idle       70W × 730h       = £14.30
load       (say 200W × 40h) = £2.24
time       2h × £70         = £140.00
                             ───────
local monthly (fixed)        = £194.04

Break-even: £194.04 ÷ £0.000123 ≈ 1.58 million requests/month. About 2,200 an hour, continuously.

That is the number that surprises people, and it is why the honest default for most projects is hosted. The fixed cost of running your own is dominated by a maintenance term that does not shrink when your volume does.

Drop the time term to zero — genuinely set-and-forget, which does happen — and break-even falls to about 440,000 requests a month. Still substantial.

When local wins anyway, and it is not about cost

The interesting cases are the ones where the money was never the argument:

The data cannot leave. Medical, legal, or a client contract that says so. This is not a cost comparison, it is a constraint, and it ends the discussion.
Latency in a room. A physical device that must respond in 200ms cannot afford a round trip to another continent, and no amount of API discount fixes the speed of light.
It must work with the network down. A greenhouse, a boat, a rural site. Availability, not price.
Cost predictability. £194 every month regardless of volume is sometimes worth more than a lower average with a tail risk, particularly when you have quoted a client a fixed fee — see cost per run before you quote.
You want to learn the stack. A completely legitimate reason that nobody puts in a spreadsheet. Just do not file it under cost saving.

The recommendation

Start hosted. Instrument spend per request from day one, so the crossover is a fact you will observe rather than a thing you speculate about. If you cross a few hundred thousand requests a month, or if one of the five constraints above applies, run the arithmetic again with your real numbers.

What you should not do is buy a card and then work out what it saved you. That is the same error as quoting a job before costing it, in a different currency.

Local or nothing does this properly across three modules — including the part where you measure your own tokens per second, because the break-even above assumes your hardware keeps up with the volume you are throwing at it, and that assumption deserves its own measurement.

Take it further

Local or nothing — Run open-weight models on hardware you own, and know what they cost you in watts and seconds. (5 lessons, 6 min, 2 free)
Where the energy actually goes — Joules, not vibes. Measure what your own work costs before joining an argument about data centres. (3 lessons, 65 min, 3 free)
Price the work — Scope it, cost it, quote it — and know the number below which the job loses you money. (3 lessons, 80 min, 2 free)