Release  ·  BreadBowl-Embed v1.3  ·  November 19, 2026

One model. As good as two.

v1.3 retrieves and reranks from one stored representation. On the full MS MARCO corpus it ranks as well as a retriever followed by a cross-encoder of similar size. No second model. No backbone pass over candidate text. The rerank step takes 2.9 ms, not 150.

Ming Xu Founder, BreadBowl AI 5 min read
  • 0.8B encoder
  • 16 slots
  • 256-d routing + 256-d value
  • 4 kB per passage
  • 43 ms per query
  • Apache-2.0

Retriever + cross-encoderTwo models. One backbone pass per candidate.

  1. Retrievermodel 1
  2. Vector index
  3. 72 candidate texts
  4. Cross-encoderQwen3-Reranker-0.6Bmodel 2

150 msto rerank

nDCG@1042.4 · 72.8 · 69.4 (TREC DL20 significantly below v1.3)

BreadBowl-Embed v1.3One model. Reranking is attention over stored value vectors.

  1. Slot Encoderthe only model
  2. Routing index
  3. 72 stored slot setsrouting + value
  4. Attention over stored valuesno backbone pass

2.9 msto rerank

nDCG@1041.7 · 74.6 · 73.1

nDCG@10: MS MARCO · TREC DL19 · TREC DL20 significantly below v1.3 one candidate

Both rows rerank the same candidates: v1.3’s routing top 100, 72 per query on average, from the full MS MARCO corpus, on one RTX 5090. The cross-encoder is Qwen3-Reranker-0.6B, a model of similar size. Its time is candidates divided by measured batch throughput, which favors it. The animation runs at one tenth of real speed.

01 — The result

Level with a cross-encoder its own size. At a fiftieth of the rerank time.

The usual way to reach this accuracy is two models. A retriever finds candidates. A cross-encoder then runs a full backbone pass over every query–candidate pair. v1.3 does the second step from vectors it already stored.

We ran both on the full MS MARCO passage corpus, 8.8 million passages, and gave the cross-encoder v1.3’s own candidates. Qwen3-Reranker-0.6B is a cross-encoder of similar size. v1.3 is level with it on MS MARCO and TREC DL19, and 3.7 nDCG@10 ahead on TREC DL20.

Complete two-model pipelines give the same picture. Put that cross-encoder behind Qwen3-Embedding-0.6B or Jina v5 text-small: level on MS MARCO and DL19, and behind v1.3 on DL20 by 5.5 and 6.4 points. Across fifteen comparisons with released systems of similar size, v1.3 wins six, is level in nine and loses none.

Ranking quality against rerank timeMeasured
Show the table
SystemRerank msMS MARCODL19DL20
BreadBowl-Embed v1.3 (one model)2.941.774.673.1
v1.3 candidates + Qwen3-Reranker-0.6B15042.472.869.4 significantly below v1.3
v1.3 candidates + Qwen3-Reranker-4B66044.6 significantly above v1.375.375.3
Qwen3-Embedding-0.6B alone—38.0 significantly below v1.366.8 significantly below v1.367.6 significantly below v1.3
Qwen3-Embedding-0.6B + Qwen3-Reranker-0.6B20041.972.067.6 significantly below v1.3
Qwen3-Embedding-0.6B + Qwen3-Reranker-4B87044.4 significantly above v1.374.574.6
Jina v5 text-small alone—42.170.672.0
Jina v5 text-small + Qwen3-Reranker-0.6B20042.072.666.8 significantly below v1.3
Jina v5 text-small + Qwen3-Reranker-4B88044.5 significantly above v1.373.974.4
nDCG@10 × 100 on the full MS MARCO passage corpus. Rerank time is per query on one RTX 5090; cross-encoder times are candidates divided by measured batch throughput, which favors them. Hollow markers do not differ significantly from v1.3 (95% paired bootstrap intervals). ▲ and ▼ are significantly above and below it. Level does not mean equivalent: TREC DL19 and DL20 have 43 and 54 queries. Differences quoted in the text are computed before rounding.

02 — The mechanism

The reranker is six lines of linear algebra.

A cross-encoder reranks by reading. Every layer of its backbone runs over the query and the candidate text, once per candidate. v1.3 stores what it needs when it encodes the passage: 16 routing vectors and 16 value vectors.

The routing vectors are what the index searches. At rerank time the same similarities become attention weights over the candidate’s stored values, and the query’s value vectors are compared with what that attention reads out. We call this polymorphic routing: the vectors that find a passage are the keys that read it.

One stored representation, three stepsIllustration

Diagram. A query or passage goes through the backbone, Qwen3.5 0.8B, and becomes 16 slots, stored once per passage. Each slot has a routing vector, which finds candidates, and a value vector, which reranks. Score equals 0.75 times routing plus 0.25 times value.
Queries and passages share one encoder. Passages are encoded once, offline. At query time only the query runs through the backbone.
The whole rerank step, per candidate

# Rq, Rd: 16 × 256 routing vectors, unit norm. Vq, Vd: 16 × 256 value vectors.

  1. C = Rq @ Rd.T

    CThe 16 × 16 routing similarities. The index already searched with these vectors.

  2. s_r = (tau_r * (torch.logsumexp(C / tau_r, -1) - math.log(16))).mean()

    s_rRouting score: a smooth MaxSim over the passage’s slots, averaged over the query’s.

  3. A = torch.softmax(C / tau_v, dim=-1)

    AThe same similarities, as attention weights over the passage’s 16 slots.

  4. U = F.normalize(A @ Vd, dim=-1)

    UOne readout per query slot: a weighted mix of the stored value vectors.

  5. s_v = (F.normalize(Vq, dim=-1) * U).sum(-1).mean()

    s_vValue score: does each readout agree with the query’s own value vector?

  6. s = 0.75 * s_r + 0.25 * s_v

    sThe final score.

tau_r = 0.02 · tau_v = 0.10 · no learned weights at comparison time

That is about 1.4 × 105 multiply–adds per candidate, whatever the length of the text. For the same 72 candidates, the cross-encoder runs 72 backbone passes. v1.3 runs none.

A whole query takes 43 ms on one RTX 5090, over 8.8 million passages. 34 ms encode the query, 5.7 ms retrieve, 2.9 ms rerank. Of those 2.9 ms, 0.6 are arithmetic; the rest reads the candidates’ value vectors from a warm file cache. From a cold cache that read takes about 11 ms.

One query, end to end43 ms

  • Encode the query 34 ms
  • Retrieve 5.7 ms
  • Rerank 2.9 ms
Median stage times for one query, approximate index (IVF-PQ, 5.7 GB), one RTX 5090. Against exact search, the index costs at most 0.54 points of nDCG@10.

The price is storage. Each passage keeps 32 vectors of 256 dimensions. At 4-bit precision that is 4,160 bytes, about the size of one 1,024-dimensional float32 embedding, for at most 0.12 points of nDCG@10.

The launch post walks through the slots with real model outputs.

03 — What changed since v1.2

The ranking gradient now reaches the index.

v1.2 learned to rerank with the backbone and the routing head frozen. Two stop-gradients kept the distillation loss out of the routing score and the attention weights, so distillation trained the value vectors. The pooling is shared, so the routing vectors still moved, and replay kept retrieval in place.

v1.3 drops the stop-gradients and trains everything, the backbone included. The ranking losses now reach the routing vectors two ways: through the routing score, and through the attention over the values. Retrieval replay, every eighth update, is still what holds candidate discovery in place.

The teacher grew too, from Qwen3-Reranker-0.6B to Qwen3-Reranker-4B. The whole stage is 1,152 updates.

Where the ranking gradient flowsIllustration
TrainedFrozenTeacherRanking gradient reaches
v1.2RCA pooling, value headbackbone, routing headQwen3-Reranker-0.6Bvalue vectors (value head, RCA pooling)
v1.3everything, backbone includednothingQwen3-Reranker-4Bvalue vectors, routing vectors (through sr and through A), backbone
From the paper’s training section. v1.2 is the model after stage 2, released as v1.2-preview; v1.3 adds stage 3: a 128-update warm-up, then 1,024 joint updates.

In matched comparisons on the development panel, unfreezing the backbone gave +0.98 nDCG@10 after 64 updates, and letting the distillation and top-order terms train the routing vectors gave +0.41 after 128. Both are significant.

v1.2 → v1.3, nDCG@10Measured
Show as a table
Test setv1.2v1.3Gain
MS MARCO34.2741.71+7.4
TREC DL1968.9974.63+5.6
TREC DL2059.5773.14+13.6
BEIR, mean of 15 tasks51.5455.13+3.6
Same pipeline settings for both checkpoints; full MS MARCO corpus for the first three rows. Scores are given to two decimals, as in the paper’s appendix. Four of the 15 BEIR tasks have training sets in our training pool. On the other eleven the mean rises from 48.4 to 50.9.

On DL20, the student passes its first teacher.

Qwen3-Reranker-0.6B taught v1.2. On the same candidates v1.3 now ranks level with it on MS MARCO and DL19 and 3.7 points ahead on DL20, without reading any candidate text at query time.

04 — Look inside

A score you can take apart.

The value score is attention over 16 stored slots, so every rerank decision has a 16 × 16 map behind it. Below is the released v1.3 checkpoint on one passage and five questions. The stored vectors are identical in every view. Only the query changes.

Real model output for the query "How many hearts does an octopus have?". Three rows, one per passage slot, each showing the passage with words shaded by attention. The darkest words are "hearts" and the full stop after it, and, in the third row, "Octopuses".
A real run of v1.3. Each row is one of the three passage slots the query reads most. The shading shows which words that slot pooled.

This shows where the model looked. It is not proof of why a passage won.

05 — The control

Do the value vectors earn their bytes?

The obvious objection: drop the values, keep the 16 routing vectors, and you have a fixed-budget late-interaction model at half the storage. So we trained it. Same starting checkpoint, same data in the same order, same teacher labels, seed and learning rates, with every ranking loss applied to the routing score alone.

After the same 1,024 updates the full model is significantly ahead on TREC DL19, by 1.7 points, and on the mean of 14 BEIR tasks, by 0.7. MS MARCO and DL20 are level. The control is ahead on none of the four.

v1.3 minus a routing-only model, nDCG@10Measured · same training
MS MARCO95% interval −0.24 to +0.38+0.07level
TREC DL1995% interval +0.04 to +3.37+1.66ahead
TREC DL2095% interval −0.50 to +2.58+1.01level
BEIR, 14 tasks95% interval +0.45 to +0.91+0.68ahead
Dots: v1.3 minus the control. Lines: 95% paired bootstrap intervals. Filled: the interval excludes zero. The BEIR row is the mean of the 14 tasks without MS MARCO. By the reading we fixed before the runs, the BEIR lead (under 1 point) is a small lead, and on MS MARCO a gain is not shown. The control starts from a checkpoint trained with both scores.

That is a small lead, and we report it as one. It costs the second half of the store and 2.9 ms per query.

What we do not claim
  • A larger cross-encoder is still ahead. Qwen3-Reranker-4B, five times our size, beats v1.3 by 2.9 points on MS MARCO, at about 230 times the rerank time. On DL19 and DL20 the gap is not significant.
  • Not the top of every chart. Jina v5 text-small alone is level with v1.3 on all three sets. The Jina v5 paper reports BEIR means of 55.5 for Qwen3-Embedding-0.6B and 56.7 for Jina v5 text-small, against our 55.1, under settings not matched to ours.
  • Late interaction not run. We did not run ColBERTv2 or ConstBERT. Their published MRR@10 on MS MARCO is higher than ours.
  • Storage. About 4 kB per passage. A retriever with a cross-encoder stores less. So do the published ColBERTv2 index and ConstBERT’s with 32 vectors.
  • Recall. At the default search width, recall@100 is below both single-vector retrievers and, on DL19 and DL20, below v1.2. A four times wider search reverses the latter.
  • Argument retrieval. On ArguAna the value score falls short of the cross-encoders.
  • Scope. English only. One training seed, one backbone, and both cross-encoders from one model family. Latency measured on a GPU, one query at a time.

06 — Run it

Run it.

Open weights, Apache-2.0. Python 3.12 or newer.

pip install "git+https://github.com/BreadBowlAI/breadbowl-embed.git"
from breadbowl_embed import BreadBowl

model = BreadBowl.from_pretrained("dotproductx/breadbowl-embed-v1.3")
docs = ["Octopuses have three hearts.", "The Pacific is Earth's largest ocean."]

# encode once
index = model.index(model.encode_documents(docs), ids=["octopus", "ocean"])
query = model.encode_queries(["How many hearts does an octopus have?"])
print(index.search(query, top_k=1)[0][0].id)  # octopus

Spec sheet

Backbone
Qwen3.5-0.8B-Base, shared by queries and passages
Representation
16 slots · 256-d routing + 256-d value per slot
Input limits
128 tokens (query) · 384 tokens (passage)
Storage
4,160 bytes per passage at 4-bit · 32,768 at float32
Query latency
43 ms end to end · one RTX 5090 · 8.8M passages · approximate index
Rerank
2.9 ms per query · about 1.4 × 105 multiply–adds per candidate
Language
English
License
Apache-2.0

A retrieval index can carry its own reranker.

Reranking used to mean a second model and a second read of every candidate. In v1.3 it is attention over vectors the index already holds, and it ranks level with a cross-encoder of its own size.

How far this goes against larger cross-encoders is the open question. The weights, the code and the paper are public.

More from the blogOne vector is too few. One per token is too many.The launch post: how the slots work, with interactive figures.