theLLMs

Last checked: 2026-06-27

Scope: Global. Sources checked as of 2026-06-27.

AI draft model: qwen3.6:35b

AI review model: qwen3.6:35b

Key terms:

>_ large language model (LLM) >_ context window >_ top-p (nucleus sampling) >_ top-k sampling >_ JSON mode >_ reasoning model
Hero image for Best Embedding Models for RAG: Dimensions, Context Window & Accuracy Comparison

TL;DR

Embedding models are the hidden first hop in every retrieval-augmented generation pipeline they set the ceiling for answer quality. Choosing the right model means balancing three non-negotiable trade-offs: retrieval accuracy (measured on the MTEB Retrieval sub-scores), output dimensionality (which drives storage and latency costs linearly), and per-million-token pricing (where indexing and query-side cost diverge sharply).

Among the leading options:

  • OpenAI text-embedding-3-small $0.02/M tokens, 8,192 context window, strongest starting point for English RAG with lowest friction
  • Cohere embed-v3 variable dimensions, 512-token window, matches or beats OpenAI on MTEB retrieval scores in many tasks
  • Jina v3 multilingual (89+ languages), open-weight commercial license, strong MTEB retrieval performance
  • BGE (BAAI General Embedding) near-commercial accuracy at zero marginal cost when self-hosted, best-in-class open-source retrieval scores

Why Embeddings Are the Foundation of RAG

Embedding models are the hidden first hop in every retrieval-augmented generation pipeline and they set the ceiling for answer quality. An embedding model converts raw text into fixed-length numeric vectors; the semantic relationships encoded in those vectors determine whether a vector database can match a user’s query to the right document chunk or return something tangentially related instead source: GitHub Embedding Benchmark / MTEB.

The most common mistake teams make is picking an embedding model based on brand recognition rather than evaluating it against three concrete constraints: retrieval accuracy (how well does this model actually retrieve relevant passages on standard benchmarks), embedding dimensions (what storage and query-latency footprint does the chosen dimensionality impose on your vector database?), and cost per million tokens (which diverges sharply between bulk indexing workloads and high-frequency query workloads). These three axes are not independent higher accuracy often comes with larger dimensions and higher inference cost, while smaller dimensions save money linearly in storage but can degrade recall by measurable margins.

Understanding this trade-off triangle matters because the MTEB (Massive Text Embedding Benchmark) leaderboard source: GitHub Embedding Benchmark / MTEB provides a standardized, apples-to-apples comparison of retrieval accuracy across models, yet nearly every production RAG application ultimately makes its budget and storage decision through the dimensions and pricing lenses. The rest of this article quantifies each of those three dimensions for the leading embedding models on the market today.

Key Comparison Metrics Explained

Retrieval accuracy remains the most important single metric, but it is not a monolith. MTEB scores aggregate performance over dozens of tasks semantic textual similarity (STS), reranking, and retrieval but not all benchmarks weight these sub-tasks equally. A model that crushes STS may underperform on retrieval if its vector space does not preserve coarse-grained topic structure as tightly source: GitHub Embedding Benchmark / MTEB. When comparing models, look at the Retrieval-specific sub-score within MTEB rather than the aggregate average.

Dimensionality is the primary cost lever in a RAG system. Storage in vector databases (Pinecone, Weaviate, Chroma) scales linearly with embedding dimension: reducing from 1536 dimensions to 256 dims cuts index storage by approximately 83%. Query latency also declines roughly proportionally because both the per-vector data transfer and the cosine-similarity computation cost drop linearly with dim count. The critical question is how much recall you sacrifice at each step MTEB retrieval scores typically degrade gradually rather than cliff-diving as dimensions shrink, meaning a well-chosen smaller dimension (e.g., 512 or 768) can recover most of the accuracy at a fraction of the storage cost.

Context window defines the maximum input-token span the model accepts directly constraining how large your RAG chunks can be before truncation occurs. Models supporting 8,192 tokens (e.g., OpenAI text-embedding-3-small, Jina v3) allow chunk sizes that cover entire paragraphs or short documents without splitting, reducing fragmentation in retrieval. Models with shorter windows (Cohere embed-v3 at 512 tokens) force smaller chunks, which means more chunks needed per document and higher indexing latency but Cohere compensates by having the embeddings themselves encode task instructions natively source: Cohere Docs.

Cost structure breaks into indexing (bulk embedding of a corpus, write-heavy, often batch-priced at discount) versus query (per-user request volume, read-heavy, typically full list price). OpenAI’s text-embedding-3-large lists at $0.13/M input tokens and $0.65/M output tokens source: OpenAI Pricing; its small variant at $0.02/M input tokens is 6.5x cheaper with only ~3% lower MTEB retrieval performance, making it the default pick for cost-conscious applications. Contextually aware indexing (e.g., Cohere’s indexed pricing) can yield competitive bulk rates for embedding-heavy pipelines source: Cohere Pricing.

Top Embedding Models Scorecard

OpenAI text-embedding-3-small Output dimensions: 1536 or 512 (via Matryoshka truncation), Context window: 8,192 tokens, Cost: $0.02/M input tokens / free (output) source: OpenAI Docs. The lowest-friction option for English RAG pipelines. Strong retrieval scores across MTEB sub-tasks; the 8192-context window covers most natural chunk sizes without truncation. Free tier available with rate limits source: OpenAI Pricing.

Cohere embed-v3 Output dimensions: variable via command prefix (default 1024 for English retrieval, 384 available), Context window: 512 tokens, Cost: competitive with per-million-token indexing pricing source: Cohere Docs. Optimized explicitly for English-language retrieval. Native instruction-encoding lets you swap between classification and embedding modes without changing the API call. The tight 512-token context window requires shorter chunks, which trades indexing latency for precision in dense passage retrieval source: Cohere Pricing.

Jina jina-embeddings-v3 Output dimensions: 1024 or 1600 (with multilingual LoRA variants), Context window: 8,192 tokens, Cost: free via Jina-hosted tier; open-weight under commercial license source: Jina AI Models. Strong MTEB retrieval scores with competitive performance against commercial-only models. Multilingual support across 89+ languages makes it the strongest multilingual candidate among models compared here. Apache 2.0-compatible open-weight distribution source: Jina AI Models.

BGE (BAAI General Embedding) models Output dimensions: multiple variants spanning 768 to 4096, Context window: varies by variant (BGE-M3 supports 8192), Cost: open-source and free via Apache 2.0 license source: BGE on HuggingFace. Best-in-class retrieval scores among open-source models on MTEB. The “large” variant (64+ MTEB aggregate score) matches commercial mid-tier models at zero marginal cost when self-hosted via vLLM or Ollama source: CodeSOTA Benchmarks. Multiple dimension options allow precise storage/accuracy tuning without model swaps.

Decision Framework for Choosing Embedding Models

Need maximum accuracy? Prioritize models with the highest MTEB Retrieval sub-scores regardless of dimension or cost. At time of writing, Jina v3 (max-dim variant) and BGE-large-en-v1.5 lead open-source retrieval scores, while Cohere embed-v3 competes strongly on English-only benchmarks source: GitHub Embedding Benchmark / MTEB. Use 768+ dimensions in full precision and reserve dimensionality reduction for final-stage tuning, not initial selection.

Budget-constrained at scale? Self-host open-weight models as the primary savings lever. BGE-large via vLLM or Ollama delivers near-commercial accuracy at zero marginal inference cost, with 768-dimensional embeddings cutting Pinecone/Weaviate vector storage by roughly 50% compared to 1536-dim options source: Pinecone Docs. Jina v3’s open-weight commercial license offers similar flexibility source: Jina AI Models.

Vector database constraints matter. Pinecone standard indices cap at 3072 dimensions; Atlas (newer tier) supports higher. Weaviate and Chroma impose no hard numerical cap, but their storage and compute cost scales linearly with dimension every extra dimension adds proportional disk IOPS during index builds and ANN searches source: Weaviate Schema Docs, source: Chroma Docs. If your data center already bills vector storage by capacity, moving from 1536-dim to 256-dim embeddings is worth the MTEB recall trade-off investigation.

Multilingual needs tip the scale toward Jina v3 or Cohere embed-v3. Both explicitly support multilingual encoding Jina v3 targets 89+ languages, Cohere embed-v3 covers multiple major language families. OpenAI text-embedding-3 is English-only, and BGE (at least the base large-en variant) focuses on English source: Jina AI Models, source: Cohere Docs. For cross-language retrieval, a multilingual embedding model is non-negotiable.

Cost Analysis for RAG Workloads

Indexing cost (corpus-wide embeddings): Bulk indexing dominates initial and periodic re-index costs. Embedding one million documents averaging 6,000 tokens each with OpenAI text-embedding-3-large at $0.13/M input tokens yields approximately $78 in indexing fees for that corpus batch. Cohere’s indexed pricing structure offers competitive bulk rates for comparable throughput source: Cohere Pricing. OpenAI text-embedding-3-small at $0.02/M drops this to roughly $15.60 per million documents a 5x savings for only ~3% lower MTEB retrieval performance.

Query cost (per-user ingestion): In production RAG the query-side volume dwarfs indexing: a mid-tier app with 10,000 daily active users sending an average of 3 queries each generates approximately 109.5M tokens/year in query embeddings. At OpenAI text-embedding-3-small’s $0.02/M rate that is roughly $2.19/year in embedding costs trivial for most budgets. But at larger scale (100k daily active users), the same calculation escalates to approximately $21.90/year, which starts mattering in margin analysis. Jina-hosted and BGE self-hosting offer effectively free query-side embedding, making their infra cost profile attractive at 10k+ DAU source: Jina AI Models.

Dimension size drives vector database storage linearly: Reducing from 1536 to 256 dimensions cuts any Pinecone or Weaviate index storage by approximately 83% while the MTEB recall degradation is typically in the 2-5 percentage point range a defensible trade-off for cost-sensitive production systems. At larger scales, this linear reduction compounds: storing 1 billion vectors at 768 dims costs roughly half what 1536 dims requires in Pinecone source: Pinecone Docs.

Total Cost of Ownership formula: A practical approximation for annual embedding-related cost is:

AnnualEmbeddingCost = (index_tokens x indexing_price_per_M)
  + (query_volume x pricing_per_M)
  + (vector_count x dim_size x $0.005/year/MiB storage_cost)

The first term covers corpus indexing, the second covers query-side inference, and the third captures vector database storage (Pinecone’s published rate is approximately $0.30/GB/month for indexed vectors, roughly $3.60/year/GB). Dimension changes impact both the second and third terms simultaneously smaller dimensions reduce both per-vector inference cost and monthly storage bills.

Final Recommendations by Use Case

General-purpose English RAG with lowest friction: Pick OpenAI text-embedding-3-small. It provides strong baseline MTEB retrieval scores, an 8192-token context window that handles most natural chunk sizes, and a free tier for prototyping source: OpenAI Docs. The $0.02/M input price makes it the cheapest commercial option without significant accuracy penalty relative to text-embedding-3-large source: OpenAI Pricing.

Highest accuracy regardless of cost: Use Cohere embed-v3 (English) or Jina v3 max-dim variant, depending on which holds superior MTEB retrieval scores at time of writing. Both models target maximum retrieval quality over storage economy choose based on your output-dimension requirements and whether multilingual coverage matters source: Cohere Docs, source: Jina AI Models.

Budget-conscious production RAG: Self-host BGE-large-en-v1.5 via vLLM or Ollama at 768 dimensions. MTEB retrieval performance tracks commercial mid-tier models closely, and the marginal inference cost is zero (limited only by your GPU hours) source: CodeSOTA Benchmarks. The combination of open-source licensing and dimensionally-optimized storage makes this the top pick for teams that already operate GPU infrastructure.

Multilingual RAG: Choose Jina-embeddings-v3’s multilingual LoRA variant (89+ languages) or Cohere embed-v3’s multi-lingual mode. Neither OpenAI text-embedding-3 nor BGE base models provide comparable cross-language coverage source: Jina AI Models, source: Cohere Docs. If your user base spans multiple languages, a monolingual English model even one with higher MTEB scores on English subsets will systematically miss non-English passages regardless of how well it ranks the ones it can see.

Conclusion: Choosing the Right Embedding Model for Your RAG Pipeline

The decision of which embedding model to deploy sits at the intersection of three non-negotiable constraints: retrieval accuracy (measured by MTEB Retrieval sub-scores), output dimensionality (which drives storage and latency linearly), and cost structure (where indexing and query-side pricing diverge sharply). This article quantified that trade-off triangle for four leading models across six dimensions of comparison accuracy benchmarks, vector storage impact, context window constraints, indexing versus query pricing, decision frameworks for distinct use cases, and total cost of ownership.

Across the MTEB Retrieval sub-tasks source: GitHub Embedding Benchmark / MTEB, Jina v3 (max-dim variant) and BGE-large-en-v1.5 establish the open-source accuracy baseline, while Cohere embed-v3 holds strong on English-specific retrieval source: Cohere Docs. For teams prioritizing budget over every last percentage point of recall, dimensionality reduction offers the most leveraged savings: moving from 1536 to 256 dimensions cuts vector storage by roughly 83% with only a 2-5 percentage point hit on MTEB recall source: Pinecone Docs. At that scale, the TCO formula presented here maps annual embedding costs across three terms corpus indexing fees, query-side inference rates, and vector database storage and reveals how dimension choices compound across all three cost buckets simultaneously.

The practical takeaway is straightforward: match the model to your primary bottleneck. Accuracy-first teams should default to Cohere embed-v3 or Jina v3 at full precision source: Cohere Docs, source: Jina AI Models. Budget-constrained production pipelines benefit most from self-hosting BGE-large-en via vLLM or Ollama at 768 dimensions, where marginal inference cost drops toward zero source: CodeSOTA Benchmarks. Multilingual RAG demands Jina v3’s multilingual LoRA variant or Cohere’s multi-language mode neither OpenAI text-embedding-3 nor BGE base variants deliver comparable cross-language coverage source: Jina AI Models, source: Cohere Docs. And for teams seeking the lowest-friction commercial entry point, OpenAI text-embedding-3-small at $0.02/M with its 8192-token window remains a strong default source: OpenAI Pricing, source: OpenAI Docs.

As the HuggingFace Embedding Models Leaderboard continues to evolve and new models push MTEB retrieval scores higher source: GitHub Embedding Benchmark / MTEB, the architecture patterns dimension truncation techniques, Matryoshka representations, and cost-tracking formulas described here remain the stable framework for evaluating whatever comes next. The specific model rankings shift quarterly; the trade-off triangle does not.

Methodology

  • Data checked: 2026-06-27
  • Sources consulted: OpenAI platform docs, OpenAI pricing page, Cohere documentation, Cohere pricing page, Jina AI HuggingFace model page, jina.ai models page, BGE on HuggingFace, CodeSOTA Benchmarks, Pinecone indexing docs, Weaviate schema docs, Chroma docs, GitHub Embedding Benchmark (MTEB)
  • Assumptions: MTEB Retrieval sub-scores as reported at time of writing remain representative of current model capabilities; vector database pricing (Pinecone ~$0.30/GB/month) reflects publicly published rates at time of writing; all cost calculations use standard per-million-token pricing with no volume discounts applied.
  • Limitations: This guide focuses on English and multilingual general-purpose embedding models for dense retrieval specialized models (e.g., legal-domain embeddings, code-specific embeddings) are out of scope. Pricing and benchmark data may change over time; always verify against current source documentation before production deployment.
  • Jurisdiction: Global.

Source list

Trust Stack

  • Last checked: 2026-06-27
  • Corrections: Contact us to report errors

Change log

  • 2026-06-27: first published