Release · BreadBowl-Embed v1.3 · November 19, 2026
One model. As good as two.
v1.3 retrieves and reranks from one stored representation. On the full MS MARCO corpus it ranks as well as a retriever followed by a cross-encoder of similar size. No second model. No backbone pass over candidate text. The rerank step takes 2.9 ms, not 150.
- 0.8B encoder
- 16 slots
- 256-d routing + 256-d value
- 4 kB per passage
- 43 ms per query
- Apache-2.0
Retriever + cross-encoderTwo models. One backbone pass per candidate.
- Retrievermodel 1
- Vector index
- 72 candidate texts
- Cross-encoderQwen3-Reranker-0.6Bmodel 2
150 msto rerank
nDCG@1042.4 · 72.8 · 69.4 (TREC DL20 significantly below v1.3)
BreadBowl-Embed v1.3One model. Reranking is attention over stored value vectors.
- Slot Encoderthe only model
- Routing index
- 72 stored slot setsrouting + value
- Attention over stored valuesno backbone pass
2.9 msto rerank
nDCG@1041.7 · 74.6 · 73.1
nDCG@10: MS MARCO · TREC DL19 · TREC DL20 significantly below v1.3 one candidate
01 — The result
Level with a cross-encoder its own size. At a fiftieth of the rerank time.
The usual way to reach this accuracy is two models. A retriever finds candidates. A cross-encoder then runs a full backbone pass over every query–candidate pair. v1.3 does the second step from vectors it already stored.
We ran both on the full MS MARCO passage corpus, 8.8 million passages, and gave the cross-encoder v1.3’s own candidates. Qwen3-Reranker-0.6B is a cross-encoder of similar size. v1.3 is level with it on MS MARCO and TREC DL19, and 3.7 nDCG@10 ahead on TREC DL20.
Complete two-model pipelines give the same picture. Put that cross-encoder behind Qwen3-Embedding-0.6B or Jina v5 text-small: level on MS MARCO and DL19, and behind v1.3 on DL20 by 5.5 and 6.4 points. Across fifteen comparisons with released systems of similar size, v1.3 wins six, is level in nine and loses none.
Show the table
| System | Rerank ms | MS MARCO | DL19 | DL20 |
|---|---|---|---|---|
| BreadBowl-Embed v1.3 (one model) | 2.9 | 41.7 | 74.6 | 73.1 |
| v1.3 candidates + Qwen3-Reranker-0.6B | 150 | 42.4 | 72.8 | 69.4 significantly below v1.3 |
| v1.3 candidates + Qwen3-Reranker-4B | 660 | 44.6 significantly above v1.3 | 75.3 | 75.3 |
| Qwen3-Embedding-0.6B alone | — | 38.0 significantly below v1.3 | 66.8 significantly below v1.3 | 67.6 significantly below v1.3 |
| Qwen3-Embedding-0.6B + Qwen3-Reranker-0.6B | 200 | 41.9 | 72.0 | 67.6 significantly below v1.3 |
| Qwen3-Embedding-0.6B + Qwen3-Reranker-4B | 870 | 44.4 significantly above v1.3 | 74.5 | 74.6 |
| Jina v5 text-small alone | — | 42.1 | 70.6 | 72.0 |
| Jina v5 text-small + Qwen3-Reranker-0.6B | 200 | 42.0 | 72.6 | 66.8 significantly below v1.3 |
| Jina v5 text-small + Qwen3-Reranker-4B | 880 | 44.5 significantly above v1.3 | 73.9 | 74.4 |
02 — The mechanism
The reranker is six lines of linear algebra.
A cross-encoder reranks by reading. Every layer of its backbone runs over the query and the candidate text, once per candidate. v1.3 stores what it needs when it encodes the passage: 16 routing vectors and 16 value vectors.
The routing vectors are what the index searches. At rerank time the same similarities become attention weights over the candidate’s stored values, and the query’s value vectors are compared with what that attention reads out. We call this polymorphic routing: the vectors that find a passage are the keys that read it.
# Rq, Rd: 16 × 256 routing vectors, unit norm. Vq, Vd: 16 × 256 value vectors.
C = Rq @ Rd.TCThe 16 × 16 routing similarities. The index already searched with these vectors.
s_r = (tau_r * (torch.logsumexp(C / tau_r, -1) - math.log(16))).mean()s_rRouting score: a smooth MaxSim over the passage’s slots, averaged over the query’s.
A = torch.softmax(C / tau_v, dim=-1)AThe same similarities, as attention weights over the passage’s 16 slots.
U = F.normalize(A @ Vd, dim=-1)UOne readout per query slot: a weighted mix of the stored value vectors.
s_v = (F.normalize(Vq, dim=-1) * U).sum(-1).mean()s_vValue score: does each readout agree with the query’s own value vector?
s = 0.75 * s_r + 0.25 * s_vsThe final score.
tau_r = 0.02 · tau_v = 0.10 · no learned weights at comparison time
That is about 1.4 × 105 multiply–adds per candidate, whatever the length of the text. For the same 72 candidates, the cross-encoder runs 72 backbone passes. v1.3 runs none.
A whole query takes 43 ms on one RTX 5090, over 8.8 million passages. 34 ms encode the query, 5.7 ms retrieve, 2.9 ms rerank. Of those 2.9 ms, 0.6 are arithmetic; the rest reads the candidates’ value vectors from a warm file cache. From a cold cache that read takes about 11 ms.
One query, end to end43 ms
- Encode the query 34 ms
- Retrieve 5.7 ms
- Rerank 2.9 ms
The price is storage. Each passage keeps 32 vectors of 256 dimensions. At 4-bit precision that is 4,160 bytes, about the size of one 1,024-dimensional float32 embedding, for at most 0.12 points of nDCG@10.
The launch post walks through the slots with real model outputs.
03 — What changed since v1.2
The ranking gradient now reaches the index.
v1.2 learned to rerank with the backbone and the routing head frozen. Two stop-gradients kept the distillation loss out of the routing score and the attention weights, so distillation trained the value vectors. The pooling is shared, so the routing vectors still moved, and replay kept retrieval in place.
v1.3 drops the stop-gradients and trains everything, the backbone included. The ranking losses now reach the routing vectors two ways: through the routing score, and through the attention over the values. Retrieval replay, every eighth update, is still what holds candidate discovery in place.
The teacher grew too, from Qwen3-Reranker-0.6B to Qwen3-Reranker-4B. The whole stage is 1,152 updates.
| Trained | Frozen | Teacher | Ranking gradient reaches | |
|---|---|---|---|---|
| v1.2 | RCA pooling, value head | backbone, routing head | Qwen3-Reranker-0.6B | value vectors (value head, RCA pooling) |
| v1.3 | everything, backbone included | nothing | Qwen3-Reranker-4B | value vectors, routing vectors (through sr and through A), backbone |
In matched comparisons on the development panel, unfreezing the backbone gave +0.98 nDCG@10 after 64 updates, and letting the distillation and top-order terms train the routing vectors gave +0.41 after 128. Both are significant.
Show as a table
| Test set | v1.2 | v1.3 | Gain |
|---|---|---|---|
| MS MARCO | 34.27 | 41.71 | +7.4 |
| TREC DL19 | 68.99 | 74.63 | +5.6 |
| TREC DL20 | 59.57 | 73.14 | +13.6 |
| BEIR, mean of 15 tasks | 51.54 | 55.13 | +3.6 |
On DL20, the student passes its first teacher.
Qwen3-Reranker-0.6B taught v1.2. On the same candidates v1.3 now ranks level with it on MS MARCO and DL19 and 3.7 points ahead on DL20, without reading any candidate text at query time.
04 — Look inside
A score you can take apart.
The value score is attention over 16 stored slots, so every rerank decision has a 16 × 16 map behind it. Below is the released v1.3 checkpoint on one passage and five questions. The stored vectors are identical in every view. Only the query changes.
This shows where the model looked. It is not proof of why a passage won.
05 — The control
Do the value vectors earn their bytes?
The obvious objection: drop the values, keep the 16 routing vectors, and you have a fixed-budget late-interaction model at half the storage. So we trained it. Same starting checkpoint, same data in the same order, same teacher labels, seed and learning rates, with every ranking loss applied to the routing score alone.
After the same 1,024 updates the full model is significantly ahead on TREC DL19, by 1.7 points, and on the mean of 14 BEIR tasks, by 0.7. MS MARCO and DL20 are level. The control is ahead on none of the four.
That is a small lead, and we report it as one. It costs the second half of the store and 2.9 ms per query.
- A larger cross-encoder is still ahead. Qwen3-Reranker-4B, five times our size, beats v1.3 by 2.9 points on MS MARCO, at about 230 times the rerank time. On DL19 and DL20 the gap is not significant.
- Not the top of every chart. Jina v5 text-small alone is level with v1.3 on all three sets. The Jina v5 paper reports BEIR means of 55.5 for Qwen3-Embedding-0.6B and 56.7 for Jina v5 text-small, against our 55.1, under settings not matched to ours.
- Late interaction not run. We did not run ColBERTv2 or ConstBERT. Their published MRR@10 on MS MARCO is higher than ours.
- Storage. About 4 kB per passage. A retriever with a cross-encoder stores less. So do the published ColBERTv2 index and ConstBERT’s with 32 vectors.
- Recall. At the default search width, recall@100 is below both single-vector retrievers and, on DL19 and DL20, below v1.2. A four times wider search reverses the latter.
- Argument retrieval. On ArguAna the value score falls short of the cross-encoders.
- Scope. English only. One training seed, one backbone, and both cross-encoders from one model family. Latency measured on a GPU, one query at a time.
06 — Run it
Run it.
Open weights, Apache-2.0. Python 3.12 or newer.
pip install "git+https://github.com/BreadBowlAI/breadbowl-embed.git"
from breadbowl_embed import BreadBowl
model = BreadBowl.from_pretrained("dotproductx/breadbowl-embed-v1.3")
docs = ["Octopuses have three hearts.", "The Pacific is Earth's largest ocean."]
# encode once
index = model.index(model.encode_documents(docs), ids=["octopus", "ocean"])
query = model.encode_queries(["How many hearts does an octopus have?"])
print(index.search(query, top_k=1)[0][0].id) # octopus
Spec sheet
- Backbone
- Qwen3.5-0.8B-Base, shared by queries and passages
- Representation
- 16 slots · 256-d routing + 256-d value per slot
- Input limits
- 128 tokens (query) · 384 tokens (passage)
- Storage
- 4,160 bytes per passage at 4-bit · 32,768 at float32
- Query latency
- 43 ms end to end · one RTX 5090 · 8.8M passages · approximate index
- Rerank
- 2.9 ms per query · about 1.4 × 105 multiply–adds per candidate
- Language
- English
- License
- Apache-2.0
A retrieval index can carry its own reranker.
Reranking used to mean a second model and a second read of every candidate. In v1.3 it is attention over vectors the index already holds, and it ranks level with a cross-encoder of its own size.
How far this goes against larger cross-encoders is the open question. The weights, the code and the paper are public.
More from the blogOne vector is too few. One per token is too many.The launch post: how the slots work, with interactive figures.