Best Embedding Model for RAG: How to Choose in 2026

The honest answer to “what is the best embedding model for RAG” is that the leaderboard winner and the right model for your corpus are usually different. MTEB’s own authors found no single embedding method dominates across tasks, and BEIR showed dense retrievers losing to BM25 out of domain. Below: the 2026 shortlist, the metric to rank on, and a script that settles it on your own chunks.

The model is one stage of the RAG pipeline, and it only matters if retrieval is the right tool at all: RAG vs fine-tuning covers when to embed a corpus versus training the model.

The short answer: a 2026 shortlist

Model	Weights	Dimensions	Max input	Notes
Voyage voyage-4-large	API	1024 default, 256/512/2048	32K tokens	$0.12/M tokens, first 200M free
Google gemini-embedding-001	API	128 to 3072, 768/1536/3072 recommended	2,048 tokens	task_type for query vs document
Cohere embed-v4.0	API	256/512/1024/1536	128K tokens	int8/binary output, PDF and image input
OpenAI text-embedding-3-large	API	3072, shortenable	8,192 tokens	64.6 MTEB, vendor-reported
Qwen3-Embedding-8B	Apache 2.0	32 to 4096	32K tokens	70.58 MTEB multilingual, self-reported June 2025
BAAI bge-m3	MIT	1024	8,192 tokens	dense, sparse and multi-vector in one model
multilingual-e5-large-instruct	MIT	1024	512 tokens	560M params, best public model in the MMTEB paper

Every row is from the vendor’s own docs or model card (linked in Sources), so read each score as a vendor benchmark. The independent view, the MMTEB paper (ICLR 2025, 500+ tasks, 250+ languages), found the best public model at time of writing was the 560-million-parameter multilingual-e5-large-instruct.

Two tiers. Data can leave your network: Voyage voyage-4-large and Gemini lead the vendors’ retrieval numbers; Cohere’s 128K context and native int8/binary output are the operational edge. Data stays in your VPC: Qwen3-Embedding (8B, 4B and 0.6B, all Apache 2.0) is the strongest open-weight family; BGE-M3 is the pick when lexical matching must come from the same model.

The metric that matters: recall on your own golden set

The original MTEB paper averaged eight task types across 58 datasets; a RAG pipeline exercises one of them. Rank on the retrieval column, which reports nDCG@10, and treat the average as noise.

nDCG@10 (normalised discounted cumulative gain) scores the top 10 hits by graded relevance, discounted by log of rank and normalised against the ideal ordering, so rank 1 beats rank 9. With a cross-encoder reranker downstream, recall@k is the better operational metric: a reranker reorders the candidate set but cannot recover a chunk that never made the cut.

Public retrieval sets are Wikipedia and web questions; your corpus is Confluence exports, error codes and jargon, the out-of-domain gap BEIR measured across 18 datasets, where BM25 held up as a strong baseline and dense models generalised poorly. Build 50 to 200 question-to-chunk pairs from real search logs, per the retrieval debugging checklist, rank candidates on that set, and keep it as the regression suite for later changes.

Wiring it up: a bake-off in one file

# bakeoff.py: recall@10 and MRR on your own golden set
import json
import numpy as np
from sentence_transformers import SentenceTransformer

CANDIDATES = [
    "Qwen/Qwen3-Embedding-0.6B",
    "BAAI/bge-m3",
    "intfloat/multilingual-e5-large-instruct",
]
K = 10

# golden.jsonl rows: {"query", "chunk_id"}; chunks.jsonl rows: {"id", "text"}
golden = [json.loads(l) for l in open("golden.jsonl")]
chunks = [json.loads(l) for l in open("chunks.jsonl")]
chunk_ids = [c["id"] for c in chunks]

for name in CANDIDATES:
    model = SentenceTransformer(name)
    docs = model.encode_document([c["text"] for c in chunks], normalize_embeddings=True)
    qs = model.encode_query([g["query"] for g in golden], normalize_embeddings=True)
    scores = qs @ docs.T  # cosine, because both sides are unit-normalised
    ranks = np.argsort(-scores, axis=1)
    hits, rr = 0, 0.0
    for i, g in enumerate(golden):
        top = [chunk_ids[j] for j in ranks[i, :K]]
        if g["chunk_id"] in top:
            hits += 1
            rr += 1.0 / (top.index(g["chunk_id"]) + 1)
    n = len(golden)
    print(f"{name:42s} recall@{K}={hits / n:.3f}  MRR@{K}={rr / n:.3f}  dims={docs.shape[1]}")

encode_query and encode_document apply each model’s own prompts: Qwen3’s model card reports a 1 to 5 percent gain from query instructions, and multilingual-e5-large-instruct degrades without its Instruct: ... Query: ... format. normalize_embeddings=True makes dot product equal cosine, the metric mismatch behind many broken first deployments.

What you’ll see

Good is a tight cluster, candidates within a couple of recall points, and the decision turns on hosting, dimension cost and context length. Bad is a 10 to 20 point spread with the leaderboard leader mid-pack: the corpus is off-distribution (code, non-English, dense jargon), and the specialised options (voyage-code-4, voyage-law-2, voyage-finance-2) deserve a run, as does BGE-M3’s sparse head for exact-token queries.

Dimensions and the Matryoshka lever

Storage scales linearly with dimensions: one million chunks at 3072 float32 dimensions is 12.3 GB of raw vectors before any HNSW graph, 4.1 GB at 1024; the chunk and index sizing calculator adds index overhead. Storage and index type are a database decision as much as a model one; once the embedding width is set, how to choose a vector database covers matching it to a store.

Matryoshka Representation Learning (Kusupati et al., 2022) front-loads information into the leading dimensions, so a stored vector can be truncated after the fact. The paper reports up to 14x smaller embeddings at equal accuracy, on ImageNet rather than text. OpenAI’s docs claim a 256-dimension slice of text-embedding-3-large beats ada-002 at 1536, and Google’s docs put gemini-embedding-001 at 67.99 on MTEB at 768 dimensions against 68.16 at 2048. Both are vendor numbers and both point one way: cut to 768 or 1024, then verify recall on the golden set. Cohere’s int8 and binary output types stack on top.

Caveats

• Switching is a full re-index. Vectors from different models, and per Google’s docs even different versions of one model, are incompatible. Price the re-embedding job before committing to an API whose deprecations you do not control.
• Context length is not chunk size. A 32K-token window does not mean 32K-token chunks, since one vector averages every topic in it; sizes that work are in the chunking guide.
• Retrieved text is untrusted input. Whatever embeds a document also embeds any instructions hidden in it; see indirect prompt injection in RAG pipelines.
• Query drift. Recall measured at launch decays as the question mix shifts; track it like any production model, and SentryML’s ML monitoring metrics taxonomy is the checklist.

Related across the network

• LangChain Chunk Size for RAG: Pick It With a Recall Sweep — langchainguide.com
• LangChain RAG Pipeline: Setup to First Answer — langchainguide.com
• LlamaIndex Chunk Size Best Practices for RAG — llamaindexhub.com
• LlamaIndex Retrieval Troubleshooting: Fix Bad Answers — llamaindexhub.com
• Embedding Store Reliability: What to Monitor Beyond Recall@k — mlmonitoring.report