Best Embedding Model for RAG: How to Choose in 2026

The best embedding model for RAG in October 2026 is Voyage’s voyage-4-large if your text can go to a hosted API ($0.12 per million tokens, 32K-token input), and Qwen3-Embedding-8B if it must stay on your own GPUs (Apache 2.0, 32K tokens). Treat both as starting points: confirm the pick on 50 to 200 of your own queries before indexing.

The leaderboard winner and the right model for your corpus are often different. MTEB’s own authors found no single embedding method dominates across tasks, and BEIR showed dense retrievers losing to BM25 out of domain. Below: the 2026 shortlist, picks by use case, what changed since 2025, current prices, the metric to rank on, and a script that settles it on your own chunks.

The model is one stage of the RAG pipeline, and it only matters if retrieval is the right tool at all: RAG vs fine-tuning covers when to embed a corpus versus training the model.

The short answer: a 2026 shortlist

Every row is from the vendor’s own docs or model card (linked in Sources), so read each score as a vendor benchmark, and do not compare scores across rows: MTEB multilingual, MTEB v2 and RTEB are different task sets. The independent view, the MMTEB paper (ICLR 2025, 500+ tasks, 250+ languages), found the best public model at time of writing was the 560-million-parameter multilingual-e5-large-instruct.

Two tiers. Data can leave your network: voyage-4-large topped Voyage’s own 29-dataset RTEB comparison, 3.87 percent ahead of gemini-embedding-001 and 8.20 percent ahead of Cohere Embed v4 on nDCG@10. Gemini Embedding 2 and Cohere Embed 5 answer with multimodal input, and Cohere’s 128K context and native int8/binary output are the operational edge. Data stays in your VPC: Qwen3-Embedding (8B, 4B and 0.6B, all Apache 2.0) is the safest open-weight default, Nemotron-3-Embed-8B is the newest retrieval leader to bake off against it, and BGE-M3 is the pick when lexical matching must come from the same model.

Best embedding model by use case

Best embedding models 2025 vs 2026: what changed

A “best embedding models 2025” list is now a year stale. Each of 2025’s leading picks has a successor or a challenger:

If you indexed with a 2025 pick, nothing forces a move yet: gemini-embedding-001’s listed shutdown is no earlier than May 14, 2028. The exception is Google’s older text-embedding-004, whose earliest shutdown date, January 14, 2026, has already passed. Otherwise rerun your golden set against the 2026 models and switch only for a recall gain that pays for a full re-index.

Embedding API prices, checked October 2026

Figures are from the Voyage, Gemini, Cohere and OpenAI pricing pages. Voyage’s batch discount does not draw on the free tokens.

A worked example: one million 500-token chunks is 500M tokens. That costs $60 on voyage-4-large ($36 after the free 200M), $65 on text-embedding-3-large, $100 on gemini-embedding-2 ($50 through batch) and $10 on text-embedding-3-small. The one-off embedding bill is small next to vector storage and the full re-index every model change forces. Self-hosting swaps per-token fees for GPU hours; Cohere’s dedicated Model Vault instances for Embed 5 start at $3.00 an hour.

The metric that matters: recall on your own golden set

The original MTEB paper averaged eight task types across 58 datasets; a RAG pipeline exercises one of them. Rank on the retrieval column, which reports nDCG@10, and treat the average as noise.

nDCG@10 (normalised discounted cumulative gain) scores the top 10 hits by graded relevance, discounted by log of rank and normalised against the ideal ordering, so rank 1 beats rank 9. With a cross-encoder reranker downstream, recall@k is the better operational metric: a reranker reorders the candidate set but cannot recover a chunk that never made the cut.

Public retrieval sets are Wikipedia and web questions; your corpus is Confluence exports, error codes and jargon, the out-of-domain gap BEIR measured across 18 datasets, where BM25 held up as a strong baseline and dense models generalised poorly. Build 50 to 200 question-to-chunk pairs from real search logs, per the retrieval debugging checklist, rank candidates on that set, and keep it as the regression suite for later changes.

Wiring it up: a bake-off in one file

# bakeoff.py: recall@10 and MRR on your own golden set
import json
import numpy as np
from sentence_transformers import SentenceTransformer

CANDIDATES = [
    "Qwen/Qwen3-Embedding-0.6B",
    "BAAI/bge-m3",
    "intfloat/multilingual-e5-large-instruct",
]
# Query instructions for models that ship no sentence-transformers prompts;
# the rest fall back to their own built-in "query" prompt, if any.
PROMPTS = {
    "intfloat/multilingual-e5-large-instruct":
        "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: ",
}
K = 10

# golden.jsonl rows: {"query", "chunk_id"}; chunks.jsonl rows: {"id", "text"}
golden = [json.loads(l) for l in open("golden.jsonl")]
chunks = [json.loads(l) for l in open("chunks.jsonl")]
chunk_ids = [c["id"] for c in chunks]
queries = [g["query"] for g in golden]

for name in CANDIDATES:
    model = SentenceTransformer(name)
    docs = model.encode_document([c["text"] for c in chunks], normalize_embeddings=True)
    qs = model.encode_query(queries, prompt=PROMPTS.get(name), normalize_embeddings=True)
    scores = qs @ docs.T  # cosine, because both sides are unit-normalised
    ranks = np.argsort(-scores, axis=1)
    hits, rr = 0, 0.0
    for i, g in enumerate(golden):
        top = [chunk_ids[j] for j in ranks[i, :K]]
        if g["chunk_id"] in top:
            hits += 1
            rr += 1.0 / (top.index(g["chunk_id"]) + 1)
    n = len(golden)
    print(f"{name:42s} recall@{K}={hits / n:.3f}  MRR@{K}={rr / n:.3f}  dims={docs.shape[1]}")

encode_query and encode_document apply a model’s built-in prompts only when it ships them. Qwen3 ships its query prompt, and its model card reports a 1 to 5 percent gain from query instructions. By contrast, multilingual-e5-large-instruct ships none, so its Instruct: ... Query: ... format has to be passed explicitly through PROMPTS; without it the model degrades and the bake-off under-ranks it. normalize_embeddings=True makes dot product equal cosine, the metric mismatch behind many broken first deployments.

The 2026 open models drop into CANDIDATES the same way. harrier-oss-v1 takes the same Instruct: ... Query: ... query format, so give it a PROMPTS entry. Nemotron-3-Embed-8B ships saved prompts that encode_query and encode_document apply, but its card requires Transformers 5.2.0 and sentence-transformers 5.4.1 or later; Qwen3-Embedding needs transformers 4.51.0 or later.

What you’ll see

Good is a tight cluster, candidates within a couple of recall points, and the decision turns on hosting, dimension cost and context length. Bad is a 10 to 20 point spread with the leaderboard leader mid-pack: the corpus is off-distribution (code, non-English, dense jargon), and the specialised options (voyage-code-4, voyage-law-2, voyage-finance-2) deserve a run, as does BGE-M3’s sparse head for exact-token queries.

Dimensions and the Matryoshka lever

Storage scales linearly with dimensions: one million chunks at 3072 float32 dimensions is 12.3 GB of raw vectors before any HNSW graph, 4.1 GB at 1024; the chunk and index sizing calculator adds index overhead. Storage and index type are a database decision as much as a model one; once the embedding width is set, how to choose a vector database covers matching it to a store.

pgvector is the tightest ceiling: its HNSW and IVFFlat indexes take vector columns up to 2,000 dimensions and halfvec up to 4,000, per the pgvector README. Full-width 3072-dimension OpenAI or Gemini vectors and 4096-dimension Qwen3-8B or Nemotron vectors have to be truncated or stored at half precision before they can be indexed there. The Qdrant vs Milvus vs Pinecone comparison covers index types and RAM sizing for the dedicated stores.

Matryoshka Representation Learning (Kusupati et al., 2022) front-loads information into the leading dimensions, so a stored vector can be truncated after the fact. The paper reports up to 14x smaller embeddings at equal accuracy, on ImageNet rather than text. OpenAI’s docs claim a 256-dimension slice of text-embedding-3-large beats ada-002 at 1536, and Google’s docs put the older gemini-embedding-001 at 67.99 on MTEB at 768 dimensions against 68.16 at 2048. Both are vendor numbers and both point one way: cut to 768 or 1024, then verify recall on the golden set. Cohere’s int8 and binary output types stack on top: its Embed 5 launch post estimates 100 million chunks shrinking from about 819 GB at 2048-dimension float32 to 3.2 GB at 256-dimension binary, and recommends 1024-dimension int8 as the balance point.

Caveats

• Switching is a full re-index. Vectors from different models, and per Google’s docs even different versions of one model, are incompatible: gemini-embedding-001 and gemini-embedding-2 vectors cannot share an index, so moving off the deprecated model means re-embedding everything. Price the re-embedding job before committing to an API whose deprecations you do not control. The exceptions are families built on one shared space: all four Voyage 4 models, including the open voyage-4-nano, and Cohere’s Embed 5 Pro and Fast, where Cohere puts the cross-model recall cost at 1.6 to 2.7 percent.
• Context length is not chunk size. A 32K-token window does not mean 32K-token chunks, since one vector averages every topic in it; sizes that work are in the chunking guide, and how to chunk text for RAG caps each chunk with the embedding model’s own tokenizer.
• Retrieved text is untrusted input. Whatever embeds a document also embeds any instructions hidden in it; see indirect prompt injection in RAG pipelines.
• Query drift. Recall measured at launch decays as the question mix shifts; track it like any production model, and SentryML’s ML monitoring metrics taxonomy is the checklist.

Related across the network

• LangChain Chunk Size for RAG: Pick It With a Recall Sweep — langchainguide.com
• LangChain RAG Pipeline: Setup to First Answer — langchainguide.com
• LlamaIndex Chunk Size Best Practices for RAG — llamaindexhub.com
• LlamaIndex Retrieval Troubleshooting: Fix Bad Answers — llamaindexhub.com
• Embedding Store Reliability: What to Monitor Beyond Recall@k — mlmonitoring.report

FAQ

what was the best embedding model in 2025

Qwen3-Embedding-8B was the standout open-weight model of 2025, ranked first on MTEB multilingual at 70.58 on June 5, 2025. On the hosted side, gemini-embedding-001 went GA in July 2025 and Cohere’s Embed 4 shipped in April with a 128K-token window. All three now have 2026 successors or challengers, so retest before trusting a 2025 ranking.

is openai text-embedding-3-large still good for rag

It still works, but it is no longer a default pick. OpenAI lists it at 64.6 on MTEB with an 8,192-token limit, and it costs $0.13 per million tokens, more than voyage-4-large at $0.12. Voyage’s January 2026 comparison puts voyage-4-large 14.05 percent ahead of it on RTEB, a vendor-reported figure worth checking on your own queries.

which embedding model is best for multilingual rag

harrier-oss-v1 is the highest-scoring self-hosted option: Microsoft’s 27B model scored 74.3 on multilingual MTEB v2 and the 0.6B scored 69.0, both MIT-licensed with 32K context and 94 languages tagged on the 0.6B card. Hosted, Cohere Embed 5 and Gemini Embedding 2 each cover 100+ languages. BGE-M3 covers 100+ and adds sparse vectors.

can i run a good embedding model on cpu

Yes, if you pick a model near 300M parameters. EmbeddingGemma-300m (Gemma licence) takes 2,048 tokens, outputs 768 dimensions truncatable to 128, scores 69.67 on MTEB English v2 and ships int4 and int8 QAT checkpoints; its activations do not support float16. voyage-4-nano, also about 0.3B and Apache 2.0, produces vectors compatible with Voyage’s hosted 4-series.

is the mteb leaderboard reliable for choosing an embedding model

Only its retrieval numbers are, and only for building a shortlist. The overall MTEB average mixes classification, clustering and other tasks RAG never runs. RTEB, added to the leaderboard on October 1, 2025, scores retrieval alone by nDCG@10 and holds some datasets private to expose models overfit to public test sets. Final selection belongs to your own golden set.