Fixing SPLADE with BM25: Don't let it ignore a word that's right there

Looking for TL;DR? Check key takeaways

SPLADE learns a weight for every word instead of computing one. It also expands each document with words it never had. Expansions are contextual and signficantly raise search quality.

I was looking at the weights and then I found a flaw in SPLADE. While aiming for expansions, it sometimes weights an existing token/word as 0 degrading search. BM25 never does that, so I used BM25 as a floor: score every word both ways and keep the higher one. NDCG@10 goes from 0.6253 to 0.6454 across 13 NanoBEIR datasets, and the share of queries with 0 relevant results drops from 12.2% to 10.3%.

SPLADE can ignore words that are right there

Here's an example:

  query 106                        the document it should find (id 76695)

  "What approaches are there       "...this is my academic understanding of
   for pricing a small              business pricing... from my research, it
   business?"                       looks like many small business use Seller
                                    Discretionary Earnings (SDE)..."

  SPLADE encodes the two sides separately, then multiplies them term by term:

  ────────────────────────────────────────────────────
   term       in doc    w_query      w_doc     product
  ────────────────────────────────────────────────────
   pricing        x1      2.164  ×   2.189  =    4.738
   business       x7      1.454  ×   2.236  =    3.252
   small          x1      2.199  ×   0.000  =    0.000   ← contributes nothing
  ────────────────────────────────────────────────────

  pricing and business still match, so the document scores 12.99.
  That puts it 12th of 4598. Tenth place scored 13.41, so it misses
  the top 10 by 0.42 and NDCG@10 counts it as nothing.

  restore small at its BM25 value (+0.9) and the document scores
  14.34, which is 8th. Now it counts.

small is in the document, and the query side weights it 2.199, the most important term as per query. The document side gives it zero. SPLADE encodes the two sides separately and multiplies them, so a zero on either side means that word contributes nothing at all.

The document still scores well on the other two words. It just scores 0.42 too low to make the top 10.

That's not a rare thing. On the queries where SPLADE loses to BM25, the document side zeroes a content word that's in the right document 12.0% of the time. On the queries it wins, 4.5%. The query side is clean either way at about 3%, so only one side is at fault.

Some more words it dropped while the query valued them highly: macro (query weight 2.629), arte (2.445), strata (2.137). BM25 would have scored all of them.

The rule

For every term a document contains, score it both ways and keep the higher one.

\mathrm{bm25}(t,d) = \mathrm{idf}(t)\cdot\frac{f_{t,d}\,(k_1+1)}{f_{t,d} + k_1\left(1 - b + b\,\frac{|d|}{\mathrm{avgdl}}\right)}

w_t = \max\big(\,w^{\text{splade}}_t\;,\; C\cdot \mathrm{bm25}(t,d)\,\big) \qquad C = 0.31,\; k_1 = 1.2,\; b = 0.75

SPLADE's added words are safe. BM25 doesn't touch expansions but SPLADE zeroed comes back at its BM25 value. It lifts words SPLADE underrated and puts back words it deleted.

Why is this safe?

If it's an exansion, we don't touch it.
If it's an existing term, we lift underrated words and brings back deleted ones.

Observations:

+0.0201 NDCG@10, 0.6253 to 0.6454, bootstrap CI [+0.0125, +0.0281], winning on 12 of 13 datasets. Best are NanoSciFact +0.0540 and NanoFiQA2018 +0.0433. And there's a small loss for NanoMSMARCO at −0.0091.
The one loss is the one in-domain dataset. SPLADE v3 is trained on MS MARCO passages and nothing else, from a checkpoint that was also MS MARCO only. NanoMSMARCO is the only dataset here it has actually seen, and it's the only one where the floor gets in the way. The other 12 are zero-shot and it wins on all 12. So SPLADE's learned weights for literal terms are well calibrated on the distribution it trained on, and BM25's IDF is the better literal channel everywhere else.
The tail moves too, which is what we should care about the most since relevance tail is important. Queries with 0 relevant results drop from 12.2% to 10.3%, and p25 goes 0.351 to 0.360.
It helps with recall too Recall@10 gains +0.0160 and recall@100 by +0.0057. This means the rule pulls relevant documents from rank 30 or 60 into the top 10.
The same floor on the query vector buys tail only. max(w_q, 0.25·idf) moves p25 by +0.019 and the mean by nothing. Take it if the weak-query quartile is what you're optimising.

Taking the rule apart

variant	mean	vs SPLADE	p50	p25	zero	recall@10	W/L
SPLADE v3, no floor	0.6253	-	0.631	0.351	12.2%	0.6472	-
lift only, never re-add a pruned word	0.6379	+0.0126	0.649	0.356	11.4%	0.6548	12/1
revive only, never touch a kept weight	0.6325	+0.0071	0.631	0.346	10.8%	0.6587	8/5
never revive a stopword or punctuation	0.6443	+0.0192	0.672	0.360	10.3%	0.6619	12/1
the full rule	0.6454	+0.0201	0.672	0.360	10.3%	0.6632	12/1

Observations:

Lifting helps 1.8x more than revival, and the two roughly add up. SPLADE makes both mistakes, underrating words it keeps and deleting words it shouldn't, and each is worth fixing separately.
Skipping stopwords and punctuation is free. SPLADE deletes 70.9% of the query-reachable stopwords it sees, on purpose, and the rule puts them all back at weights around 0.20 against 1.23 for content words. Skipping them costs 0.0011 on the mean, gives identical p50, p25 and zero-rate, and drops 13% of the added postings. Take it.
Keeping the higher score beats adding the two. Adding double-counts every word the two already agree on, and the max only steps in where they disagree. Note the sum has the better p25 at 0.387, so if the weak-query tail is all you care about it is worth a look.

How much of BM25 do you actually need? (85% of it comes from idf alone)

Strip out the word counts and document lengths entirely, so the floor is just $0.31\cdot\mathrm{idf}(t)$ , and you still get 0.6425 against the full rule's 0.6454. That's 85% of the gain from one constant, with no average document length to maintain and no per-document normalisation. Worth taking if you don't want the bookkeeping.

form inside the floor	mean	vs SPLADE	p25	zero	W/L
`idf · tf(k1+1)/(tf + k1·L)`, full BM25	0.6454	+0.0201	0.360	10.3%	12/1
`idf` alone, no word counts at all	0.6425	+0.0171	0.372	10.5%	10/3
`idf · sqrt(tf)`	0.6406	+0.0153	0.380	10.6%	12/1
`idf · tf`, raw tf-idf, no saturation	0.6280	+0.0027	0.339	11.9%	8/5

The one thing not to do is drop the saturation and keep the raw word count. Plain $\mathrm{idf}\cdot f_{t,d}$ falls to 0.6280, and pushing its constant up makes it far worse, −0.0546 at C=0.2. Word counts either need BM25's saturation or they need leaving out.

Which fits SPLADE, whose own weight grows only 1.19x from one occurrence to two and drops past ten. Note the tail column too: full BM25 wins the mean but has the worst p25 of the working forms, and sqrt(tf) gives up 0.005 on the mean for +0.020 on p25.

What the rule does to each term type, and 13% of the index you can drop

Per type, over query-reachable terms present in the document:

type	n	rule fires	re-added	SPLADE p50	C·bm25 p50	final p50	mass
content	265,943	53.1%	20.3%	1.027	0.978	1.232	77.0%
subword	36,143	57.4%	28.8%	0.928	0.941	1.170	9.9%
stopword	149,418	80.5%	70.9%	0.000	0.204	0.257	12.4%
punctuation	12,538	90.2%	85.6%	0.000	0.040	0.046	0.7%

Subword pieces are unremarkable. They match content words at every percentile on all three quantities. That's why restricting the rule to content-plus-subword never beat flooring everything. The overrides are ordinary morphology: ##ise 0.018 → 1.235, ##ers 0.451 → 1.214, ##isation 0.531 → 1.109.

Note that this holds only because the rule acts on terms present in the document. Count only terms present in SPLADE's vector and subwords look like the highest-weighted type of all, p50 1.408, because that hides the 28.8% pruned to zero.

Stopwords are the interesting row. SPLADE deletes most query-reachable stopwords and the rule puts every one back. They end up 35% of the terms it touches and 12.4% of the final weight mass. Dropping stopwords and punctuation costs +0.0190 instead of +0.0201, both at 12 wins and 1 loss. That 13% of added mass is free.

The rule also separates content words from stopwords by 4.8x in final weight, 1.232 against 0.257. SPLADE's own literal weighting separates them by 2.3x, 1.206 against 0.518. So the rule is more discriminating about term type than SPLADE is, while knowing nothing about term type. IDF does that work implicitly.

Where it breaks, and by how much

Out of 649 queries, 131 get better on NDCG@10 and 78 get worse. But counting queries treats a rank 1-to-2 swap the same as rescuing a document from rank 40, so here is the size of the moves instead:

move size	NDCG@10 better	worse	recall@10 better	worse
tiny, under 0.05	46	42	13	5
small, 0.05 to 0.15	51	31	10	3
medium, 0.15 to 0.35	19	3	14	5
large, over 0.35	15	2	10	0
total NDCG moved	+17.84	−4.82	+11.91	−1.50

The losses are almost all tiny. Where the rule moves a query a long way, it is 15 wins to 2 on NDCG@10 and 10 wins to none on recall@10, and it gains 3.7x more NDCG mass than it loses (7.9x on recall@10). The median winner moves +0.069 and the median loser −0.041.

So the failures are mostly the rank 1-to-2 kind. They still tell you what the rule gets wrong, and they come in two shapes that sit at opposite ends of the same dial.

It revives words the question didn't care about. On how many paintings of sunflowers did van gogh paint, SPLADE had decided many doesn't help find documents. It's right, "how many" is just how you phrase a counting question. The floor gives many a weight anyway, and a different van Gogh sunflowers page that was 0.06 behind jumps 2.3 points past the correct answer. Both pages are about the same paintings. Only one says how many there are. The same thing happens with next, job, get and should, all sitting around idf 4 where BM25 thinks they're moderately informative.

And it trusts rare words too much. On Should election day be a national holiday?, the floor does exactly what it's designed for: holiday has idf 6.52, so it gets a large weight, and two documents that merely mention holidays climb past a document about why voting is inconvenient. The rare word worked perfectly and still gave the wrong answer, because containing the rare word isn't the same as answering the question.

No cutoff avoids both. Ignore low-idf words and you keep the second failure, trust high-idf words and you get it worse. I tried gating on idf, restricting by term type, and capping the floor's value, and all three cost more than they saved.

One reassuring detail. Every relevant document that fell out of the top 10 had its own score go up, not down: Global_warming went 38.76 to 41.72 and still fell out, Mauna_Loa_Observatory 45.36 to 45.97. The floor never hurt those documents, it helped their competitors more.

The cheap version

The setup people actually deploy is inference-free. Encode documents once with the doc-only model, send raw query terms at weight 1.0, no query-time model call. splade-v3-doc plus a BM25 floor:

system	mean	vs base	p50	p25	zero	W/L
`splade-v3-doc`, inference-free queries	0.5953	-	0.631	0.303	14.3%	-
+ floor at C=0.20	0.6074	+0.0120	0.631	0.316	12.5%	10/3
+ floor at C=0.31	0.5917	−0.0036	0.631	0.315	14.5%	7/6

That holds up, CI [+0.0013, +0.0236], but it's about half what the full model gets, and the best C drops from 0.31 to 0.20. Inference-free queries give every word a weight of 1.0, so the query side does no sorting at all and the document side carries the whole ranking. The floor is overwriting a bigger share of the signal, so it goes too far sooner.

SPLADE deletes, IDF doesn't

The value is in the max, not in either input. Neither one alone gets you there:

for every word the document contains	NDCG@10	W/L
always trust SPLADE's weight	0.6251	-
always trust BM25's weight	0.6101	5/8
take the higher of the two	0.6454	12/1

BM25's weights on their own are worse than SPLADE's, by 0.015. Taking the higher of the two beats SPLADE by 0.020 and BM25 by 0.035.

That works because they fail in different places. SPLADE deletes words aggressively, since training penalises it for every word it keeps (the FLOPS regulariser). Deleting is usually right, because most of what it deletes is the and for. But nothing catches it when it's wrong, and then a real word contributes exactly nothing instead of a little. IDF can't make that particular mistake, since a rare word scores high whether or not the model noticed it. It makes different mistakes instead, and the <details> above has two of them.

So they're not competing much. The max fires on 53% of content words, and the other 47% SPLADE's own weight is already higher and stays. It's a floor under the learned weights, holding them up from below.

One note if you go looking for the exponent yourself. SPLADE's document weights fit idf^0.508 and its query weights fit idf^0.567, but the product w_q · w_d fits 0.9936. BM25 applies IDF once at exponent 1.0. SPLADE splits the same total into two square roots, so fitting either side alone says BM25 is twice too steep when it isn't.

Key Takeaways

Take the higher of SPLADE's learned weight and C · bm25(t,d) for every term a document contains. NDCG@10 goes 0.6253 to 0.6454 over 13 NanoBEIR datasets, winning on 12, and the share of queries with 0 relevant results drops from 12.2% to 10.3%.
The rule survives dropping tf, k1, b and document lengths. max(w_splade, 0.31·idf(t)) keeps 85% of the gain with one constant and no index statistics beyond document frequency.
It fixes two separate failures. SPLADE underweights terms it keeps (+0.0126) and deletes terms it should have kept (+0.0071), and the two are close to additive.
On the inference-free setup people actually deploy, the same floor is worth +0.0120 with the constant retuned to 0.20.

Citation

If you find this useful, please cite:

bibtex

@misc{kumar2026spladebm25,
  author       = {Kumar Shivendu},
  title        = {Fixing SPLADE with BM25: Don't let it ignore a word that's right there},
  year         = {2026},
  url          = {https://www.kshivendu.dev/blog/splade-bm25},
  note         = {Blog post}
}