What is a token, and why does it affect AI cost?
Understand tokenisation and pricing basics before using an API.
100-page programme
All 100 briefs, visible and accountable
The full 100-page programme is complete. All 100 briefs have been drafted, editorially reviewed, and integrated into the live prototype. This page is the accountable record of every brief in the programme — use it to verify coverage, not as a working queue.
100total briefs
48Cache
25Run
21Diff
48 briefs
Cache
Concepts, ideas, explainers, comparisons and knowledge snippets.
Context windows explained: why bigger is not always better
Decide whether long-context models solve a product problem.
Temperature, top-p and deterministic outputs: what the settings actually do
Configure generation settings for consistency or creativity.
Embeddings explained for business search and RAG
Understand embeddings before buying a vector database.
Latency in LLM apps: first token, total time and user experience
Diagnose why an AI feature feels slow.
Rate limits explained: requests, tokens, tiers and hidden launch risks
Plan capacity before launching an AI feature.
Prompt length, output length and why AI bills surprise teams
Explain unexpected token usage and cost spikes.
System prompts, developer prompts and user prompts: who controls what?
Understand prompt hierarchy and instruction conflicts.
JSON mode and structured outputs: what reliability does and does not mean
Get valid machine-readable outputs from LLMs.
Multimodal models explained: text, images, audio and video in practical products
Assess whether multimodal AI fits a workflow.
Inference vs training vs fine-tuning: three terms operators confuse
Understand what kind of AI work a project actually needs.
Model parameters and sizes: why 7B, 70B and MoE labels can mislead
Interpret model-size claims and open-model labels.
lm-eval-harness explained for non-researchers
Learn what lm-eval-harness can and cannot test.
HELM-style evaluation: why transparency matters as much as scores
Understand holistic model evaluation.
Promptfoo vs lm-eval-harness: when each is useful
Choose an evaluation tool for prompts or models.
RAG evaluation: checking retrieval before blaming the model
Diagnose poor RAG answers.
Synthetic eval datasets: useful shortcut or false confidence?
Decide whether to generate test cases with LLMs.
Human evaluation for LLMs: rubrics that editors and SMEs can actually use
Design a review process for generated outputs.
LLM-as-a-judge: when automated grading helps and when it lies
Use another model to grade model outputs.
Coding benchmarks explained: HumanEval, MBPP, SWE-bench and real developer work
Interpret coding model claims.
Function-calling benchmarks: why tool-use scores do not guarantee agents work
Understand tool-use and agent benchmark claims.
Long-context benchmarks: needle tests, document QA and real recall
Evaluate long-context model claims.
Contamination and leakage: why benchmark scores can be too good
Understand benchmark trust issues.
Benchmark leaderboards for busy buyers: Chatbot Arena, LiveBench and what to ignore
Use public leaderboards without overtrusting them.
Creating a model scorecard for your own workload
Compare models for a specific business use case.
API model pricing: input, output, cache and batch costs
Understand how AI API bills are calculated.
Prompt caching explained: when repeated context becomes cheaper
Reduce repeated prompt costs.
Batch APIs for LLMs: cheaper, slower and often underused
Process non-urgent AI jobs cheaply.
Hosted API vs self-hosted open model: the real cost comparison
Decide whether local/self-hosted inference saves money.
GPU rental for LLM inference: what an operator needs to know
Understand GPU hosting options for open models.
Fine-tuning economics: when training a custom model pays back
Compare fine-tuning with prompting/RAG from a cost perspective.
RAG costs: vector database, embeddings, reranking and generation
Estimate total cost of a document QA system.
AI feature unit economics: cost per user, task and successful answer
Model AI feature profitability.
Output tokens are expensive: designing shorter AI answers without hurting usefulness
Cut costs from verbose model outputs.
Model routing: using cheap models first without breaking quality
Save money by routing tasks across models.
Caching AI answers: when it is safe, risky or pointless
Reduce repeated AI calls.
The hidden cost of retries, fallbacks and validation loops
Understand why real costs exceed estimates.
LLM observability cost: logs, traces and evaluation storage
Budget for monitoring AI systems.
A simple LLM cost calculator editors can maintain
Estimate AI costs from token volumes.
Open weights vs hosted APIs: practical trade-offs
Choose between open models and managed APIs.
Cloud AI platforms vs direct model APIs: Bedrock, Vertex and Azure trade-offs
Choose between cloud marketplace AI and direct provider integration.
Prompt injection explained for business users
Understand why retrieved text or user input can hijack an AI app.
Jailbreaks vs product safety: what operators can realistically control
Understand jailbreak risk without panic.
Tool-use safety: stopping agents from taking dangerous actions
Design guardrails for AI agents with tools.
PII handling for LLM apps: minimisation before redaction
Handle personal data safely in AI features.
AI output monitoring: what to log, sample and review
Monitor production LLM quality and incidents.
Citation quality in AI answers: source-grounded does not mean source-faithful
Improve citations in AI answers.
AI incident response: what to do when a model gives harmful or wrong advice
Prepare for AI-related production incidents.
25 briefs
Run
Workflows with inputs, outputs, checks, failure modes and a stopping point.
Chat history is not memory: how LLM apps remember users
Understand AI memory features and their privacy implications.
AI agents vs workflows: a plain-English difference for teams
Decide whether a task needs an “agent” or deterministic automation.
How LLM benchmarks work, and what they miss
Understand benchmark claims in model launches.
Golden datasets for LLM products: how small regression sets prevent regressions
Build a practical evaluation set for an AI feature.
Eval CI for AI apps: testing prompts before every release
Put AI regression tests into a software pipeline.
The model release treadmill: how to avoid rebuilding every month
Reduce disruption from frequent model releases/deprecations.
Hallucination testing: how to build a small regression set
Reduce factual errors in LLM outputs.
Refusals and over-refusals: testing whether safety blocks useful work
Diagnose unwanted model refusals.
Eval gaming: when models optimise for the test rather than the task
Understand why high benchmark scores may not translate.
Red teaming an LLM feature: a practical first-week checklist
Test an AI feature before launch.
Fine-tuning vs prompting vs RAG: decision checklist
Choose an adaptation strategy for an AI product.
Function calling and tool use: where agents actually fail
Build reliable tool-using AI workflows.
MCP explained: tools, resources, prompts and the current hype gap
Understand Model Context Protocol and when it helps.
Vector databases: when semantic search is enough and when it is not
Decide whether to add a vector database.
Chunking documents for RAG: size, overlap and metadata choices
Improve retrieval quality in document AI.
Rerankers explained: the quiet quality layer in RAG systems
Understand reranking after vector search.
Building a minimum viable RAG system without overengineering
Plan a simple document-QA prototype.
AI coding agents: what to measure before trusting them
Evaluate coding agents for real engineering work.
Schema-first AI extraction: making LLMs useful for messy documents
Extract structured data from unstructured text.
LLM observability basics: traces, prompts, evals and feedback loops
Monitor an LLM app in production.
Fallback design: what happens when the AI call fails?
Design resilient AI features.
Human-in-the-loop AI: approval queues that do not become bottlenecks
Add human review to AI workflows.
Prompt versioning: treating prompts like production code
Manage changes to prompts in teams.
Building an internal AI policy bot: safe pattern or risky shortcut?
Use AI over internal policies and procedures.
The evidence-led AI website manifesto: how theLLMs will review claims
Understand the site’s editorial trust model.
21 briefs
Diff
Dated context pieces: what changed, who should care, and what remains unproved.
What model cards tell you — and what they do not
Read model cards critically.
OpenAI, Anthropic, Google and Mistral APIs: what comparison pages should measure
Compare major hosted providers without hype.
Meta Llama and open model licensing: what builders must check
Understand whether an open model can be used commercially.
Small language models: when smaller is better
Decide whether a compact model fits a task.
Reasoning models: what “thinking” modes change for cost and latency
Understand reasoning-model trade-offs.
Mixture-of-experts models: why active parameters matter
Interpret MoE claims in model launches.
Local LLM runtimes: Ollama, llama.cpp, vLLM and TGI in plain English
Choose a runtime for running open models.
Quantisation explained: why model files have Q4, Q5 and GGUF labels
Understand local model file choices.
Provider data retention policies: what API users should compare
Assess privacy and data-use differences between providers.
Model gateways and routers: OpenRouter, LiteLLM and build-vs-buy questions
Add provider flexibility without rewriting apps.
Changelog watching for AI teams: deprecations, pricing and model aliases
Keep AI apps stable amid provider changes.
Data leakage in LLM apps: logs, prompts, files and vendor retention
Identify places sensitive data can escape.
Guardrails compared: policy prompts, classifiers, validators and permissions
Choose safety controls for an AI app.
Enterprise AI procurement: questions before buying a platform
Evaluate AI vendors and platforms.
Copyright and training data: what AI product teams can responsibly say
Understand the controversy around training data and outputs.
AI energy use: useful facts without moral panic
Understand AI energy and infrastructure claims.
Hardware supply and inference economics: why chips shape AI products
Understand why GPUs and accelerators affect AI availability and cost.
AI vendor lock-in: model APIs, embeddings, vector stores and eval data
Reduce switching risk in AI systems.
AI SLAs and status pages: what reliability evidence vendors publish
Evaluate provider reliability claims.
Responsible AI policies that builders can actually operationalise
Turn abstract AI principles into processes.
AI adoption in small businesses: where LLMs help first
Identify practical AI use cases for small firms.