Open-weight vs hosted models
When to rent a model through an API and when to run open weights yourself, with the licence, hardware and privacy trade-offs spelled out.
Updated 2026-11-19. "(source N)" refers to the numbered list at the end.
Two ways to get a model
A hosted model lives on the vendor's servers. You send text over an API and pay per token. Claude, GPT and Gemini work this way.
An open-weight model publishes its trained weights, the large files that hold what the model learned. You can download them and run the model on your own machine or on a server you rent. Llama, Qwen, DeepSeek, Gemma and some Mistral models work this way. OpenAI also publishes two open models, gpt-oss-120b and gpt-oss-20b (source 5).
Open weights are not the same as open source. You usually get the finished model, not the training data or the full recipe, and the licence sets the rules.
Licences differ more than you think
Read the licence before you build a product on a model. A few current examples:
- Qwen3.8-27B uses Apache 2.0, a permissive licence that allows commercial use (source 2).
- DeepSeek V4.1 Flash uses the MIT licence for both code and weights (source 3).
- Llama 4 uses Meta's own community licence. Companies whose products had more than 700 million monthly users must ask Meta for a separate licence (source 1). That will not affect a small business, but the licence carries other terms too.
- Mistral mixes licences. Mistral Small 4 is Apache 2.0, while Mistral Large 4 uses a custom open-weight licence (source 4).
The hardware reality
Open does not mean small. Many top open models are huge, and running them well takes data-centre graphics cards.
DeepSeek V4.1 Flash has hundreds of billions of parameters, though only about 8 to 16 billion are active for each token because of how it is built (source 3). Llama 4 Scout has 109 billion parameters in total. Meta says it fits on a single H100 card, but only after squeezing the weights down to 4-bit precision (source 1). An H100 is a server card, not something in a laptop.
Smaller open models are a different story. Tools like Ollama let you download and run a model on an ordinary computer from the command line (source 10). Its library shows the most pulled models include Llama 3.1, Qwen and Gemma, and the newest Qwen3.8 builds are already listed (source 6). For serving a model to many users on your own servers, the open-source vLLM engine is a common choice (source 7).
Why pick hosted
For most small teams, a hosted API is the sensible default.
- No hardware to buy, patch or keep cool.
- You get the strongest models, which are mostly not open.
- You pay only for what you use, which suits uneven workloads.
- Paid tiers usually keep your prompts out of training. Google, for example, says its paid Gemini API tier does not use prompts to improve its products (source 8).
Why pick open weights
There are good reasons to run your own.
- Data never leaves your machines. That can matter for client files, health data or contracts that forbid third-party processing.
- No surprise price changes or retired models. You keep the exact version you tested.
- Very high, steady volume can be cheaper on your own hardware than per-token billing.
- You can fine-tune the model on your own examples.
The middle path
You do not have to choose all or nothing. Many hosts rent out open models by the token, so you get open-model prices without running servers. DeepSeek's own API, for example, serves V4.1 Flash at $0.15 to $0.30 per million input tokens and $0.60 to $1.20 per million output, with half-price off-peak hours (source 9). Mistral's platform serves its own open models and third-party ones like GLM and DeepSeek (source 4).
Check where such a host is based and what its data terms say. A cheap API in another country may not meet your privacy needs, even if the model itself is open.
Be careful with free tiers in general. Google's terms, for instance, say prompts sent on the free Gemini API tier may be used to improve its products and read by human reviewers (source 8). Paid tiers are usually stricter.
A quick decision guide
Whichever you pick, keep your app able to switch. Serving engines such as vLLM and SGLang can expose an OpenAI-compatible API (source 3), so moving between hosted and self-run models can be a configuration change rather than a rewrite.
- Just starting, low volume, no strict data rules: use a hosted API.
- Strict rule that data must stay in-house: run a small open model locally, or a larger one on a private server.
- High steady volume on a narrow task: test an open model on rented hardware and compare the monthly bill.
- Want open-model prices without the servers: use a host that serves open weights, after checking its data terms.
Sources
Written by Agents Lookup in our own words. Facts come from these pages:
- 1. Hugging Face: Llama-4-Scout-17B-16E-Instruct model card: huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
- 2. Hugging Face: Qwen3.8-27B model card: huggingface.co/Qwen/Qwen3.8-27B
- 3. Hugging Face: DeepSeek-V4.1-Flash model card: huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
- 4. Mistral AI: Models overview: docs.mistral.ai/getting-started/models/models_overview
- 5. Hugging Face: OpenAI organisation page (gpt-oss models): huggingface.co/openai
- 6. Ollama: Model library: ollama.com/library
- 7. vLLM: Documentation: docs.vllm.ai/en/latest
- 8. Google AI Studio: Gemini API additional terms of service: aistudio.google.com/docs/terms
- 9. DeepSeek: Models and pricing: api-docs.deepseek.com/quick_start/pricing
- 10. GitHub: ollama/ollama: github.com/ollama/ollama