Phase 4 — Model Catalogs and Trend Reading
How to navigate thousands of fast-changing models without drowning: use catalogs to shortlist, read benchmarks and pricing skeptically, track releases/deprecations, and maintain one automated watchlist that feeds your decisions.
Why this phase matters
Phase 3 taught you to read a single model's documents. Phase 4 zooms out to the market: comparing many models efficiently and keeping current as it churns weekly. The output is operational — a catalog explorer, a release detector, a benchmark-skeptic's checklist, a cost/margin calculator, and a maintained watchlist — all feeding model selection (Phase 5) and the gateway (Phase 8).
Documents
| # | Document | What you'll be able to do |
|---|---|---|
| 00 | How to Use models.dev | Filter thousands of models to candidates; import the catalog |
| 01 | Column-by-Column Guide | Read every catalog column; flag the binding constraint |
| 02 | How to Track Model Releases | Run a tiered system; triage releases; catch deprecations |
| 03 | How to Read Benchmarks | Read benchmarks skeptically; decide with your eval |
| 04 | How to Read Pricing Pages | Turn a pricing page into cost-per-request and margin |
| 05 | How to Build a Model Watchlist | Maintain one automated, dated, eval-backed watchlist |
How to work through it
Read in order — 00/01 are the catalog mechanics, 02–04 the skeptical reading skills, and 05 the capstone that ties them into a maintained artifact. The labs compound: the catalog importer (00) → column reader (01) → release diff (02) → benchmark + cost tools (03/04) → all feed the watchlist (05), which feeds the Phase 8 gateway.
Phase 4 artifacts
- A models.dev → SQLite explorer + candidate shortlist
- A
read_rowcatalog reader that flags the binding constraint - A catalog-diff release/deprecation detector + 4-question triage
- A trust-weighted benchmark reading + golden-set comparison
- A pricing → cost/margin calculator with caching/reasoning/batch scenarios
- A maintained
watchlist.yaml+ updater (feeding gateway routing/fallback)
Related
- Broad market snapshot: References — Trending LLMs Landscape
- The keep-up method: Phase 0.02 — How to Study Fast-Moving AI