Embeddings explained for business search and RAG
Understand embeddings before buying a vector database.
Cache
Concepts, ideas, and knowledge snippets worth keeping loaded
This is the playful replacement for "guides": not a dusty manual shelf, more like working memory for AI decisions. Tokens, context windows, evals, model trade-offs, RAG, agents, pricing, benchmarks, and all the tiny concepts that stop projects turning into expensive fog.
28published Cache pages
48Cache briefs
25how-to briefs
21news/context briefs
Published now
Live Cache in this prototype
Multimodal models explained: text, images, audio and video in practical products
Assess whether multimodal AI fits a workflow.
Function-calling benchmarks: why tool-use scores do not guarantee agents work
Use tool-use benchmarks without mistaking them for production readiness.
Benchmark leaderboards for busy buyers: Chatbot Arena, LiveBench and what to ignore
Read LLM leaderboards without treating them as procurement truth.
Creating a model scorecard for your own workload
Turn model choice into a repeatable, evidence-led decision.
RAG costs: vector database, embeddings, reranking and generation
Estimate the full cost of RAG before the chatbot becomes popular.
AI feature unit economics: cost per user, task and successful answer
Measure AI cost against outcomes, not just token totals.
Caching AI answers: when it is safe, risky or pointless
Use answer caching without serving stale or private responses.
A simple LLM cost calculator editors can maintain
Build a lightweight model-cost calculator without pretending prices are timeless.
lm-eval-harness explained for non-researchers
A plain-English guide to what lm-eval-harness does, why teams use it, and why a benchmark runner is not the same thing as proof of real-world usefulness.
API model pricing: input, output, cache and batch costs
How LLM API bills usually work, where cache and batch discounts help, and why the cheapest headline rate is not always the cheapest workload.
Context windows explained: why bigger is not always better
A plain-English guide to context windows, long-context trade-offs, and when retrieval or chunking beats stuffing everything into one prompt.
Fine-tuning vs prompting vs RAG: decision checklist
How to decide whether your AI product needs better prompting, retrieval, or fine-tuning, with a practical checklist, current docs snapshot, and clear caveats.
Function calling and tool use: where agents actually fail
A practical guide to why tool-using LLM workflows fail in production, what function calling does and does not guarantee, and what to guard before real tools are touched.
RAG evaluation: checking retrieval before blaming the model
A practical guide to testing retrieval-augmented generation, spotting whether the retriever or the generator failed, and measuring the right thing first.
Rate limits explained: requests, tokens, tiers and hidden launch risks
A practical guide to API quotas, request caps and token limits, and how to plan fallbacks before an AI feature goes live.
Local quantized LLM vs frontier model: what changed in the same writing task
A reader-facing comparison of a small local Qwen2.5 7B quantized model and a frontier model writing the same practical LLM ethics article.
Open weights vs hosted APIs: practical trade-offs
How to choose between self-hosted/open-weight models and managed API models when control, privacy, cost and switching risk all matter.
Structured outputs and JSON mode: reliability limits
A practical guide to what JSON mode and structured outputs really guarantee, where schema validation still fails, and what to check after the model responds.
What is a token, and why does it affect AI cost?
A plain-English guide to LLM tokens, why token counts change across tokenizers, and how those counts turn into API cost.
Temperature, top-p and deterministic outputs: what the settings actually do
A plain-English guide to LLM generation parameters: what temperature and top-p control, how deterministic vs creative outputs work, and a practical tuning matrix for common tasks.
Prompt length, output length and why AI bills surprise teams
How prompt and output token counts affect LLM costs, common billing surprises from verbose system prompts and long conversation histories, and practical ways to keep costs predictable.
System prompts, developer prompts and user prompts: who controls what?
A plain-English guide to the prompt hierarchy in LLM apps: what system, developer and user roles mean, how instructions overlap, and how to handle conflicts.
Model parameters and sizes: why 7B, 70B and MoE labels can mislead
A plain-English explanation of model parameter counts, what 7B, 70B and MoE labels actually tell you, and why bigger numbers do not always mean better performance.
Prompt caching explained: when repeated context becomes cheaper
How prompt caching works with LLM APIs, which workloads benefit from cached prefixes, and the provider-specific constraints you need to know before relying on it.
Output tokens are expensive: designing shorter AI answers without hurting usefulness
Why output tokens cost 3-4x more than input tokens, how verbose model defaults multiply costs, and practical strategies for getting shorter, cheaper AI answers that still work.
Batch APIs for LLMs: cheaper, slower and often underused
Many teams pay for synchronous chat APIs when their workloads could use batch APIs at 50% lower cost. This guide explains when batch APIs work, what they cost, and what to watch out for.
Model routing: using cheap models first without breaking quality
How to route LLM requests across cheap and expensive models using classifier gates, fallback criteria and shadow testing — without degrading output quality or confusing your team.
Briefed pipeline
Cache queued for the 100-page programme
What is a token, and why does it affect AI cost?
Understand tokenisation and pricing basics before using an API.
Context windows explained: why bigger is not always better
Decide whether long-context models solve a product problem.
Temperature, top-p and deterministic outputs: what the settings actually do
Configure generation settings for consistency or creativity.
Embeddings explained for business search and RAG
Understand embeddings before buying a vector database.
Latency in LLM apps: first token, total time and user experience
Diagnose why an AI feature feels slow.
Rate limits explained: requests, tokens, tiers and hidden launch risks
Plan capacity before launching an AI feature.
Prompt length, output length and why AI bills surprise teams
Explain unexpected token usage and cost spikes.
System prompts, developer prompts and user prompts: who controls what?
Understand prompt hierarchy and instruction conflicts.
JSON mode and structured outputs: what reliability does and does not mean
Get valid machine-readable outputs from LLMs.
Multimodal models explained: text, images, audio and video in practical products
Assess whether multimodal AI fits a workflow.
Inference vs training vs fine-tuning: three terms operators confuse
Understand what kind of AI work a project actually needs.
Model parameters and sizes: why 7B, 70B and MoE labels can mislead
Interpret model-size claims and open-model labels.
lm-eval-harness explained for non-researchers
Learn what lm-eval-harness can and cannot test.
HELM-style evaluation: why transparency matters as much as scores
Understand holistic model evaluation.
Promptfoo vs lm-eval-harness: when each is useful
Choose an evaluation tool for prompts or models.
RAG evaluation: checking retrieval before blaming the model
Diagnose poor RAG answers.
Synthetic eval datasets: useful shortcut or false confidence?
Decide whether to generate test cases with LLMs.
Human evaluation for LLMs: rubrics that editors and SMEs can actually use
Design a review process for generated outputs.
LLM-as-a-judge: when automated grading helps and when it lies
Use another model to grade model outputs.
Coding benchmarks explained: HumanEval, MBPP, SWE-bench and real developer work
Interpret coding model claims.
Function-calling benchmarks: why tool-use scores do not guarantee agents work
Understand tool-use and agent benchmark claims.
Long-context benchmarks: needle tests, document QA and real recall
Evaluate long-context model claims.
Contamination and leakage: why benchmark scores can be too good
Understand benchmark trust issues.
Benchmark leaderboards for busy buyers: Chatbot Arena, LiveBench and what to ignore
Use public leaderboards without overtrusting them.
Creating a model scorecard for your own workload
Compare models for a specific business use case.
API model pricing: input, output, cache and batch costs
Understand how AI API bills are calculated.
Prompt caching explained: when repeated context becomes cheaper
Reduce repeated prompt costs.
Batch APIs for LLMs: cheaper, slower and often underused
Process non-urgent AI jobs cheaply.
Hosted API vs self-hosted open model: the real cost comparison
Decide whether local/self-hosted inference saves money.
GPU rental for LLM inference: what an operator needs to know
Understand GPU hosting options for open models.
Fine-tuning economics: when training a custom model pays back
Compare fine-tuning with prompting/RAG from a cost perspective.
RAG costs: vector database, embeddings, reranking and generation
Estimate total cost of a document QA system.
AI feature unit economics: cost per user, task and successful answer
Model AI feature profitability.
Output tokens are expensive: designing shorter AI answers without hurting usefulness
Cut costs from verbose model outputs.
Model routing: using cheap models first without breaking quality
Save money by routing tasks across models.
Caching AI answers: when it is safe, risky or pointless
Reduce repeated AI calls.
The hidden cost of retries, fallbacks and validation loops
Understand why real costs exceed estimates.
LLM observability cost: logs, traces and evaluation storage
Budget for monitoring AI systems.
A simple LLM cost calculator editors can maintain
Estimate AI costs from token volumes.
Open weights vs hosted APIs: practical trade-offs
Choose between open models and managed APIs.
Cloud AI platforms vs direct model APIs: Bedrock, Vertex and Azure trade-offs
Choose between cloud marketplace AI and direct provider integration.
Prompt injection explained for business users
Understand why retrieved text or user input can hijack an AI app.
Jailbreaks vs product safety: what operators can realistically control
Understand jailbreak risk without panic.
Tool-use safety: stopping agents from taking dangerous actions
Design guardrails for AI agents with tools.
PII handling for LLM apps: minimisation before redaction
Handle personal data safely in AI features.
AI output monitoring: what to log, sample and review
Monitor production LLM quality and incidents.
Citation quality in AI answers: source-grounded does not mean source-faithful
Improve citations in AI answers.
AI incident response: what to do when a model gives harmful or wrong advice
Prepare for AI-related production incidents.
Search by idea
Find articles by describing what you need
Try "how much do tokens cost?", "run a model on my own hardware", or "stop prompt injection attacks". Search runs in your browser against our article index.
Describe what you are trying to understand.