Self-Hosting an LLM vs Using an API in 2026: When Open-Weight Models Actually Save a Startup Money (Break-Even Math)

Self-hosting an LLM is cheaper than using an API only when your traffic is high and steady. For most startups in 2026, it isn't. Two rented H100 GPUs cost about $5,560 a month whether anyone uses them or not. On my arithmetic, that only beats a mid-tier closed API above roughly 214,000 requests a month, and only beats the same open model bought per token from a host above roughly 594,000. Below those numbers, you are paying for idle GPUs.

I'm Kunal Vohra. I've been the technical co-founder six times, and I run ColadAI, a multi-LLM platform, so the question of which model runs where, and who pays for it, is one I deal with every month. My recent posts on running AirLLM on a 16GB Mac mini were about what open models can do on hardware you already own. This one is about what they cost when real users show up.

Self-hosting an LLM vs an API: the short answer

Use an API until you can prove three things: your volume is high, it is steady around the clock, and you have someone to run the GPUs. If you can't prove all three, the API is cheaper, faster to ship, and easier to change.

The comparison most founders make is wrong. They put "our own GPU" next to OpenAI or Claude prices and see a saving. The real competitor to self-hosting is not a closed model at all. It is the same open-weight model, sold per token by a hosting company, with no GPUs for you to run. Once that option is on the table, the case for running your own hardware gets much narrower.

What an API costs per request in 2026

API prices span about a hundredfold, from $0.10 to $10 per million input tokens, and hosted open models sit at the cheap end of that range. All prices below are per million tokens, input then output, as of October 2026. The closed-model prices come from my comparison of the major AI APIs; the hosted open-model prices are Together AI's serverless rates.

Closed models:

• Frontier tier: GPT-6 Astra at $10 / $50, Claude Opus 5.5 at $4 / $20.
• Mid tier: GPT-6.1 Sol and Claude Sonnet 5.5, both $2 / $10.
• Fast tier: Gemini 3.8 Flash at $0.75 / $3.75 (rising to $1.50 / $7.50 on January 1, 2027).
• Small tier: GPT-6 Luna and Claude Haiku 5.5, both $0.10 / $0.50.

Open-weight models, hosted per token:

• DeepSeek V4 Pro: $1.32 / $3.96.
• Llama 3.3 70B: $1.04 / $1.04.
• Gemma 4 31B: $0.39 / $0.97.
• gpt-oss-120B: $0.15 / $0.60.
• DeepSeek V4 Flash: $0.14 / $0.28.

To compare these with a GPU, you need a cost per request. Take a retrieval-heavy chat turn: 8,000 tokens in (instructions, retrieved documents, the user's message) and 1,000 tokens out. That request costs about $0.026 on a mid-tier closed model, about $0.0094 on hosted Llama 3.3 70B, and about $0.0013 on a small closed model. Batch processing and prompt caching cut the closed prices further, often by half or more, if you design for them.

What a GPU really costs

A rented H100 costs a median of $3.81 an hour in October 2026, which is about $2,780 a month if it runs all the time. GPU prices have gone up, not down: the median H100 rose about 28 percent over the past year. Where you rent matters a lot. On-demand prices I found ran from about $4 an hour at a GPU cloud like Lambda to about $11 an hour on Google Cloud, with spot capacity as low as about $2.50.

One GPU is rarely enough. A 70B model in 8-bit precision takes about 70GB of an H100's 80GB, which leaves little room for the memory each conversation needs, and a single GPU is a single point of failure. So the realistic minimum for serving a 70B-class model to paying users is two GPUs: about $5,560 a month at the median, before anything else.

What does that buy? Published benchmarks put Llama 3.3 70B at roughly 900 to 1,850 output tokens a second per H100 on this kind of workload, depending on how fast each user's answer has to stream. I use 1,000 tokens a second per GPU in the math. At that rate, two GPUs could serve about 5.3 million of the requests above every month. Capacity isn't the problem. Utilisation is.

The break-even math, step by step

The break-even point is your fixed monthly GPU cost divided by what one request would cost on the API. Here it is with the numbers above:

1. Fixed cost: 2 GPUs × $3.81 an hour × 730 hours = about $5,563 a month.
2. Against a mid-tier closed API: $5,563 ÷ $0.026 per request = about 214,000 requests a month.
3. Against hosted Llama 3.3 70B: $5,563 ÷ $0.0094 per request = about 594,000 requests a month.
4. Against a small closed API: $5,563 ÷ $0.0013 per request = about 4.3 million requests a month, close to what two GPUs can serve at all.

The line I'd want every founder to remember: two rented GPUs cost the same $5,560 a month at zero users as at half a million requests, so self-hosting only wins once you are busy enough to keep them working. 214,000 requests a month is about 7,000 a day, every day. Plenty of AI startups get there. Very few get there in their first year, and fewer still get there with traffic that is steady at 3 a.m.

You can redo this with your own numbers in five minutes: your tokens per request, your provider's price, your GPU quote. Change the request size and the break-even moves; a short request with little context pushes it much further out, because the API bill per request drops while the GPU bill doesn't.

What the break-even math leaves out

The GPU bill is the smallest part of self-hosting at startup scale. The arithmetic above flatters it in four ways.

The engineer. Someone has to set up and tune the serving stack (vLLM or similar), handle model updates, watch memory and latency, and get paged when it falls over. That is a meaningful slice of a senior engineer's time, and at startup salaries it can exceed the GPU bill on its own.

Idle hours. Traffic has a daily shape. If your users are in one region, your GPUs sit mostly idle for a third of the day, and you pay for them anyway. APIs only charge for what you use.

Quality. A self-hosted 70B open model is not a mid-tier closed model. For some tasks it is close enough; for others it isn't, and the cheaper option only counts if it does the job. Measure cost per successful task, not cost per token.

Speed of change. New models arrive every few months. On an API, switching is a config change. On your own hardware, it is a deployment, a benchmark, and sometimes a different GPU.

The benchmark figure I used is also an optimistic ceiling: the page it came from was unclear on precision, and real workloads rarely hit benchmark throughput. If the true number is lower, the GPU line stays where it is and capacity drops, which makes the picture for self-hosting worse, not better.

When self-hosting an open-weight model is right anyway

The strongest reasons to self-host are about control, not cost. Each is real, and each has a price worth naming.

Data that can't leave your infrastructure. Healthcare, defence, some fintech and some government buyers need the model inside their walls or yours. The closed providers' zero-retention and data-residency options close part of this gap, so check them first; my startup security checklist and the DPDP Act explainer cover where the obligations actually sit.

A customer contract that requires it. Some enterprise buyers ask for a private deployment in their own cloud account. That is a strong reason, and usually a separate deployment per customer.

Fine-tuned weights you own. Fine-tune an open model and the artefact is yours; no vendor can retire it. That only matters once fine-tuning is the right call, which is later than most founders think. I covered the order of operations in RAG vs fine-tuning for startups.

Using DeepSeek's models safely. DeepSeek's own API stores data in China. Its MIT-licensed weights, run on your servers or a US host, avoid that.

Offline and on-device products. If the product has to work without a network, there is no API option. My AirLLM setup guide shows how far a 16GB Mac can be pushed, and why that is a development and demo setup rather than production serving.

Real scale. Once you are well past the break-even with steady traffic, the savings are real and compound every month. That is a good problem, and a reason to plan the move, not to make it on day one.

The middle path most startups miss: hosted open models

A hosted open-model API gives you most of the benefits of open weights without running a GPU. You pick an open model, pay per token like any API, and keep the option to move it onto your own hardware later without changing the model your product depends on.

That path solves the lock-in worry (Gemini Flash doubling its price on January 1, 2027 is the live example of why people have it), lets you test whether an open model is good enough for your task before committing to hardware, and keeps your costs at zero when nobody is using the product. Most hosts also sell dedicated endpoints, a reserved GPU billed by the hour, which is the natural next step once volume is steady.

Check the licence before you build on it. Qwen, gpt-oss, Gemma 4 and Mistral ship under Apache 2.0, and DeepSeek's recent models under MIT: you can use them commercially without strings. Llama's community licence restricts very large products (above 700 million monthly users), and some others, such as Kimi and MiniMax, restrict commercial hosting or require a separate agreement. The models that fit on a single rented GPU (Qwen at around 27B parameters, Gemma 4 31B, gpt-oss-120B) are where self-hosting is realistic for a startup. The 500B+ models need a multi-GPU node and a team to run it.

How I'd make the call for an AI startup

Here is the sequence I'd follow:

1. Ship on an API. A closed model or a hosted open model, behind a thin layer in your code that makes the provider swappable.
2. Log cost per request from day one. Tokens in, tokens out, model, outcome. My AI MVP cost guide explains the per-user arithmetic.
3. Test an open model on your own eval set through a hosted API. If it isn't good enough, the self-hosting question is closed.
4. Run the break-even with your real request size, your real traffic shape, and a GPU quote. Add the engineer.
5. Move to a dedicated endpoint or your own GPUs only when you are clearly past the break-even, or when data or a contract requires it.

The decision is less about GPUs than about who owns it. A non-technical founder can't easily judge a self-hosting proposal from an engineer who wants to build one, and that is the kind of call a senior technical lead should make with numbers in hand. That rarely needs a full-time CTO at $180,000 to $260,000 a year. A technical advisor retainer ($750 a month for 15+ hours, held through December 2026) can check the model and infrastructure plan; a fractional CTO ($2,500 a month for 50+ hours on the same terms) can own the AI roadmap, the vendors and the build. The fractional CTO cost guide shows market rates beside mine. If you are still at the build stage, my team and I build MVPs for AI startups, and the MVP cost calculator gives you a first range.

Frequently asked questions

Is it cheaper to self-host an LLM than to use an API? Only at high, steady volume. With two rented H100s at the October 2026 median of $3.81 an hour, self-hosting a 70B model breaks even with a mid-tier closed API at roughly 214,000 requests a month (8,000 tokens in, 1,000 out each), and with the same open model hosted per token at roughly 594,000. Below that, the API is cheaper, before you count the engineer's time.

How much does it cost to run an LLM on your own GPU? A rented H100 costs a median of $3.81 an hour, about $2,780 a month running continuously, with on-demand prices from about $4 to $11 an hour depending on the provider. A 70B-class model realistically needs two GPUs, so about $5,560 a month, plus the engineering time to run it.

What is the cheapest way to use an open-source LLM? A hosted open-model API, paid per token. gpt-oss-120B runs about $0.15 / $0.60 per million tokens and DeepSeek V4 Flash about $0.14 / $0.28 on Together AI as of October 2026, with no GPUs to rent and no cost when idle.

When should a startup self-host an open-weight model? When traffic is clearly past the break-even and steady around the clock, when data or a customer contract requires the model inside your infrastructure, or when you have fine-tuned weights to serve. Even then, a dedicated endpoint from a host is often the step before your own hardware.

Can I run a production LLM on a Mac? For development, demos and personal use, yes; small models run well on Apple Silicon. For serving paying users, no. My AirLLM tests on a 16GB Mac mini measured an 8B model at 19.6 seconds per token, which is fine for an experiment and nowhere near production.

Are open-weight models free to use commercially? Many are: Qwen, gpt-oss, Gemma 4 and Mistral use Apache 2.0, and DeepSeek's recent models use MIT. Llama's licence restricts very large products, and some others require a separate agreement for commercial hosting. Read the licence for the exact version you deploy.

If you're weighing this for your own product, book a 30-minute call. Bring your request size and rough traffic, and I'll run the break-even with you and tell you whether self-hosting is worth a conversation yet. Or get in touch if you'd rather write first.