Open-Source vs Commercial Models

Phase 5 · Document 01 · Model Selection Prev: 00 — Model Selection Framework · Next: 02 — Local vs Cloud

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

"Should we use a commercial API or an open-weight model?" is one of the highest-leverage decisions you'll make — it shapes cost, data governance, ops burden, customization, and time-to-market all at once. Get it wrong and you either bleed money at scale (API when you should self-host) or drown in GPU ops you didn't need (self-host when an API would do). The terms are also widely confused: "open source," "open weights," and "free to use commercially" are three different things. This doc gives you a clear comparison and a decision rule, feeding the local-vs-cloud detail in 02.

2. Core Concept

Plain-English primer (the licensing words people conflate)

• Closed / commercial / API-only — you can't download the model; you call it over an API (OpenAI, Anthropic, Gemini). You pay per token.
• Open-weight — the weights are downloadable, so you can run/self-host the model. But the license on those weights decides what you may do.
• Open source — strictly, both the weights and training code/data are released under a permissive license. Most "open" LLMs are open-weight, not fully open-source.
• License types you'll meet: permissive (Apache-2.0, MIT — commercial use fine), community/custom (e.g. Llama's license — mostly fine but with conditions/usage caps), research-only / non-commercial (can't ship a product), source-available (visible but restricted). Always read the license — "open weights" ≠ "free for my commercial product." → Phase 1.02
• Self-hosted vs managed-open — you can run open weights yourself (self-hosted) or pay a provider to host the same open model for you (managed inference, e.g. an open model served via OpenRouter/Together). So "open-weight" doesn't force you to run GPUs. Caveat: with managed-open, the host chooses the precision/quantization and context cap — the same open model can be served at lower quality by a cheaper host. This "same ID ≠ same model" issue is a major open-weight risk; see 10 — Provider Variance and Serving Fidelity.

The two paths, compared

The decision is usually "and," not "or"

Most production systems are hybrid: commercial APIs for the frontier-quality minority and for spiky/low-volume traffic; self-hosted open weights for sensitive data, high steady volume, or customization. The selection process (00) plus routing (Phase 8) is how you combine them: route by data sensitivity and task, not religion.

The economics crossover (the quantitative core)

Per-token API cost grows linearly with usage; self-hosting is a (large) fixed GPU cost regardless of usage. So there's a break-even volume above which self-hosting is cheaper:

api_cost(V)      = V × per_token_price
selfhost_cost(V) = monthly_gpu_cost + ops_cost          (≈ fixed up to capacity)
break_even: the V where these cross — below it, API wins on cost; above it, self-host can.

Most teams are below break-even early (API wins) and approach it as volume grows — which is why "start on an API, revisit self-hosting at scale" is common advice (02, Phase 16).

3. Mental Model

THREE DIFFERENT WORDS:  open source ⊃ open weights ≠ free-commercial-use   → READ THE LICENSE

COMMERCIAL/API ──────────────────────────────  OPEN-WEIGHT
  frontier quality, zero ops, per-token,        strong quality, your data stays in,
  fast TTM, vendor lock-in                       full control/customization, GPU ops, $0/token+fixed cost

DECISION = usually HYBRID + ROUTING:
  sensitive data / high steady volume / customization → open-weight (self-host or managed-open)
  frontier quality / spiky / low volume / fast TTM    → commercial API

COST: api = V × price (linear) ; selfhost ≈ fixed → BREAK-EVEN volume decides the $ argument.

4. Hitchhiker's Guide

What to decide first: is there a hard constraint (data can't leave your environment, must be offline, must fine-tune)? That forces open-weight regardless of cost. Otherwise it's a cost/quality/TTM trade-off.

What to ignore at first: ideology ("open good / closed bad"). Decide on requirements and economics.

What misleads beginners:

• "Open weights = free to ship commercially" — check the license.
• "Self-hosting is cheaper" — only above break-even, and only if you can run GPUs reliably.
• "Open models are too weak" — top open models are strong; measure on your eval.
• "It's open vs closed" — it's usually a hybrid with routing.

How experts reason: hard constraints first (data/offline/fine-tune → open-weight); else estimate the break-even volume and weigh ops capability and TTM. They often start commercial, design for portability (OpenAI-compatible interface), and revisit self-hosting as volume grows.

What matters in production: data-governance fit, realistic total cost (incl. ops/GPU idle), the ability to actually operate GPUs, and an exit/fallback (portable interface so you can move).

How to verify: run your eval on a top open model vs your commercial pick; compute the break-even volume with real prices and GPU costs (Phase 1.08).

Questions to ask: What does the license permit for my product? What's the provider's data-retention/region policy? Can my team operate GPUs (Phase 7)? What's my break-even volume? How portable is my integration?

What silently gets expensive/unreliable: license violations discovered late; underestimating GPU ops/idle cost; vendor lock-in + deprecation; assuming open quality without measuring.

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• Open source ⊃ open weights ≠ free commercial use — read the license (Phase 1.02).
• Open-weight doesn't force self-hosting — you can use a managed-open host (per-token, no GPU ops).
• Top open models are strong (≈ last-gen frontier) — but measure on your eval.
• Commercial wins on TTM/quality/spiky volume; open wins on data control/customization/high steady volume.
• Self-hosting is cheaper only above a break-even volume — and only if you can run GPUs reliably.
• The real answer is usually hybrid + routing by data sensitivity and task.
• Commercial brings lock-in and deprecation risk; design for portability (OpenAI-compatible interface).
• Data governance often dominates the decision over cost for regulated workloads.

9. Observations from Real Systems

• Hybrid is the norm: teams use a commercial frontier model for hard/sensitive-to-quality tasks and a self-hosted/managed open model for bulk or private data, combined via a gateway (Phase 8).
• Managed-open hosts (Together, Fireworks, OpenRouter) let you use Llama/Qwen/Mistral per-token — open weights without running GPUs.
• License gotchas (non-commercial, usage caps, acceptable-use clauses) are a recurring late-stage blocker — caught by reading the license (Phase 3.04).
• OpenAI-compatible APIs everywhere (Phase 1.06) make commercial→open migration feasible — portability as an exit strategy.
• Break-even reality: most startups stay on APIs early (below break-even) and revisit self-hosting as volume and data needs grow (Phase 16).

10. Common Misconceptions

11. Engineering Decision Framework

Hard constraints first:
  data can't leave env / must be offline / must fine-tune deeply → OPEN-WEIGHT (self-host).
  none of those → it's a cost/quality/TTM trade-off, continue.

Cost argument:
  estimate break-even volume: api = V×price vs selfhost ≈ fixed GPU+ops.
  below break-even or spiky/uncertain volume → COMMERCIAL (or managed-open).
  well above break-even AND can run GPUs reliably → SELF-HOSTED open-weight.

Quality/TTM:
  need frontier quality now / fast TTM → COMMERCIAL.
  top open model passes YOUR eval → open-weight is on the table.

Default: HYBRID + routing (Phase 8). Design portable (OpenAI-compatible) to avoid lock-in.

12. Hands-On Lab

Goal

Make the open-vs-commercial call for one real use case with data, not opinion: compare quality on your eval and compute the break-even volume.

Prerequisites

• A use case + golden set (Phase 1.07); access to one commercial model and one open model (self-hosted via Phase 6 or a managed-open host); a GPU cost estimate.

Steps

1. License check: confirm the open model's license permits your use (Phase 3.04). If not, it's out.
2. Quality: run your golden set on both; record scores (Phase 1.07).
3. Break-even: compute it.

def break_even_tokens_per_month(api_price_per_1m, monthly_gpu_plus_ops):
    # tokens/month where self-host fixed cost equals API variable cost
    return monthly_gpu_plus_ops / api_price_per_1m * 1e6

# Example: API blended $0.50/1M; a GPU box + ops ≈ $2,000/month
be = break_even_tokens_per_month(0.50, 2000)
print(f"break-even ≈ {be/1e9:.1f}B tokens/month")
# below this volume → API cheaper; above → self-host can win (if you can run it)

4. Decide: combine license (gate), quality (eval), volume vs break-even, and your team's GPU ability into a recommendation — likely a hybrid/routing rule.
5. Memo: record it (Phase 3.07).

Expected output

A recommendation backed by an eval comparison and a break-even number — not vibes.

Debugging tips

• Open model fails the license gate → stop; pick another open model or go commercial.
• Break-even seems impossibly high → your volume is below it; API is the cost-winner for now.

Extension task

Add a managed-open option (open model, per-token) as a third column — often the pragmatic middle (open weights, no GPU ops).

Production extension

Express the outcome as a routing rule (sensitive→self-hosted, bulk→managed-open, hard→commercial) and estimate blended cost vs single-provider cost.

What to measure

Quality (eval) per option; break-even volume; your current volume vs break-even; blended cost (extension).

Deliverables

• A license/quality/cost comparison for one use case.
• A break-even calculation with your real prices.
• A recommendation/memo (likely hybrid + routing).

13. Verification Questions

Basic

1. Distinguish open source, open weights, and free-for-commercial-use.
2. Name two situations that force open-weight regardless of cost.
3. What is a managed-open host, and what problem does it solve?

Applied 4. Compute break-even: API at $1/1M blended; self-host box+ops at $3,000/month. At what monthly token volume does self-host break even? 5. A top open model matches your commercial pick on your eval. What else decides the choice?

Debugging 6. You shipped on an open model and got a legal flag. What did you skip? 7. Your self-hosting bill exceeded the API it replaced. Name two likely causes.

System design 8. Design a hybrid architecture routing by data sensitivity and task complexity across open and commercial models.

Startup / product 9. Explain to investors why you start commercial and plan a self-hosting path as volume grows — with the break-even logic.

14. Takeaways

1. Open source ⊃ open weights ≠ free commercial use — read the license first.
2. Commercial wins on quality/TTM/spiky volume; open-weight wins on data control/customization/high steady volume.
3. Self-hosting is cheaper only above break-even and only if you can run GPUs.
4. Open-weight ≠ must self-host — managed-open hosts exist.
5. The production answer is usually hybrid + routing by sensitivity and task.
6. Measure open quality on your eval; design portable to avoid lock-in.

15. Artifact Checklist

• License check for one open model vs your use.
• Eval comparison (open vs commercial) on your golden set.
• Break-even calculation with real prices + GPU cost.
• Recommendation/memo (likely hybrid + routing).
• (Extension) routing rule + blended-cost estimate.
• Notes: the three-words distinction and the break-even logic.

Next: 02 — Local vs Cloud