remove stale content and decisions from plan

This commit is contained in:
Dylan Couzon
2026-08-17 13:21:35 -04:00
parent e9457f4b8a
commit 278395f54b
+49 -58
View File
@@ -1,10 +1,8 @@
# Retrieval Tuning Series: Article And Experiment Plan
Revision 27, 2026-08-14. Branch `hybrid-fusion-experiment-plan`. Written for a session with no access to the conversation that produced it. Everything here is current instruction; nothing describes how the plan got this way.
Revision 28, 2026-08-17. Branch `hybrid-fusion-experiment-plan`. Written for a session with no access to the conversation that produced it. Everything here is current instruction; nothing describes how the plan got this way.
**The five articles are done and Dylan's narration and review pass is complete.** E1 to E7 are run and their artifacts are on disk. The five are drafted, gated, cross-linked and checked against the v1.19.0 API surface. `verify_articles.py` passes 207/207, which is every published figure in all five. **Dylan finished his narration and review pass on 2026-08-14 and committed it**, through `e1e619cf0`, so these are reviewed articles rather than drafts and any further edit changes reviewed work. **All five hero image sets are built.** A sixth article, `hybrid-search-recall-candidate-list`, was retired on 2026-08-14; 4a carries what moved and what that orphaned.
**Two global passes are all that remain before the PR**, and they are the next section. Nothing else is queued. Dylan called the content done on 2026-08-14, so treat a finding from either pass as a defect to fix rather than an opening to revisit scope, structure, or voice.
**The five articles are done and reviewed.** E1 to E7 are run and their artifacts are on disk. The five are drafted, gated, cross-linked and checked against the v1.19.0 API surface, and `verify_articles.py` passes 207/207, which is every published figure in all five. Dylan finished his narration and review pass on 2026-08-14 and committed it through `e1e619cf0`, so these are reviewed articles and any further edit changes reviewed work. All five hero image sets are built. A sixth article, `hybrid-search-recall-candidate-list`, was retired on 2026-08-14; 4a carries what moved and what that orphaned. Treat any later finding as a defect to fix rather than an opening to revisit scope, structure, or voice.
**Run `verify_articles.py` through `experiments/fusion/.venv`.** The venv was recreated on 2026-08-12 with numpy, pandas, pyarrow, pytrec_eval and qdrant-client, and it is gitignored. `python3` alone has no pandas and the script will not import.
@@ -35,51 +33,44 @@ Some of these readers arrive carrying an Elasticsearch or Lucene configuration,
## 1. How To Run This Session
- **Run `cd experiments/fusion && .venv/bin/python verify_articles.py` first.** It checks all 207 published figures two ways: against the artifact that produced them, and that the article file still contains the value. If it passes, the articles and the measurements agree and you can trust both. If it fails, fix that before anything else.
- **Do not re-derive sections 6 or 7, and do not re-run E1 to E6.** They are done and their numbers are already in the articles. Re-verify section 6 only against Qdrant releases after 2026-08-11.
- **Do not re-derive sections 6 or 7, and do not re-run E1 to E6.** They are done and their numbers are already in the articles. Re-verify section 6 only against a Qdrant release later than v1.19.0. Checked 2026-08-17: v1.19.0 is still the latest release, the local checkout is on its tag at commit `74f3e85b9`, and section 6 holds there.
- **A release Qdrant has already shipped is not covered by that exemption.** The 1.19 memory placement rework was in the pinned build the whole time and still reached a draft as the deprecated `on_disk`. When a section 6 entry names an API, grep the deprecations in the checkout before quoting it: `grep -rn 'deprecated(since' lib/ src/ --include='*.rs'`.
- **Do not rebuild the measurement layer.** Gate A (`test_replay.py`, 9/9) and Gate B (`parity/*.json`, deviation 0.000e+00) both pass.
- The harness is at `experiments/fusion/`; `README.md` there has the run order for both `run.py` and `study.py`.
- Write experiment code only into `experiments/fusion/`.
- **Never commit or push without explicit approval in that turn.**
**What is outstanding. Two global passes, then the PR.** Both run across all five articles at once, because both are about the set rather than any one piece. Dylan's narration and review pass is done and committed, so neither pass rewrites voice; they verify. Treat a finding as a defect to fix, not an invitation to reopen scope, structure, or voice.
**What is outstanding: pass one below, the PR, and three calls that belong to Dylan and Neil.** Nothing else is queued.
1. **Pass one, continuity and narration.** Do the five read in a logical order, and does the set say each thing once? **Continuity:** every cross-link resolves, points at an article that delivers what the link promises, and the reading order in 4a still holds after the retirement. **Repetition:** a finding stated in two articles is a defect per section 13's one-home-each rule, and so is the same sentence shape recurring across section openers and closers. Give the fusion article's `Confirm Fusion Beats Either Prefetch` extra attention, since it arrived from the retired article and is the newest prose in the set. **Vocabulary:** one name per thing across all five, per section 11. **Emphasis:** promote points important enough to risk a reader skimming past into an aside or a warning callout; Dylan's sense is that the five underuse both right now.
2. **Pass two, technical accuracy and the gates.** `/humanizer` and `/andrey-review` on all five, then `/neil-review` on the diff, then a final `verify_articles.py`. Two checks belong to this pass specifically. **Code snippet parity across all five:** consistent collection name, params, imports and API usage between articles, and no snippet contradicting a row in section 4c. **The four unresolved technical items below**, each of which a reviewer could correctly challenge, so resolve them here rather than carrying them into the PR.
1. **Pass one, continuity and narration, with no evidence it has run.** Do the five read in a logical order, and does the set say each thing once? **Continuity:** every cross-link resolves, points at an article that delivers what the link promises, and the reading order in 4a still holds after the retirement. **Repetition:** a finding stated in two articles is a defect per section 13's one-home-each rule, and so is the same sentence shape recurring across section openers and closers. Give the fusion article's `Confirm Fusion Beats Either Prefetch` extra attention, since it arrived from the retired article and is the newest prose in the set. **Vocabulary:** one name per thing across all five, per section 11. **Emphasis:** promote points important enough to risk a reader skimming past into an aside or a warning callout; Dylan's sense is that the five underuse both right now.
2. **Pass two, technical accuracy and the gates, ran on 2026-08-14** in commit `99eb3230b`: `/humanizer` and `/andrey-review` on all five, `/neil-review` on the diff, then `verify_articles.py`. Two checks belonged to it. **Code snippet parity across all five:** consistent collection name, params, imports and API usage between articles, and no snippet contradicting a row in 4c. **The four technical items below**, closed 2026-08-17.
3. **The PR.** `experiments/` does not ship in it and the notebook moves separately, both per section 13.
**The four unresolved technical items pass two owns.** Each is recorded in full where it belongs; this list exists so none is missed.
**The four technical items pass two owned, all closed 2026-08-17**, one line each so none gets reopened.
- **The E4 reranker baseline is selection-inflated while the reranker is not.** Logged 2026-08-14 and the most consequential of the four, because it can move a published headline. Section 3a carries the mechanism and the numbers.
- **Three tier 2 knobs lost their only home to the retirement**: Matryoshka `mrl`, SPLADE and miniCOIL, and ColBERT as a retriever. Either the audit article's rebuild tier gains a paragraph, or 5b stops listing them as covered. 4a carries the decision.
- **The `score_threshold` row for the audit article's silent-settings table**, decided on 2026-08-14 and reported as in progress in a parallel session. Confirm it landed; the last check of `before-tuning-a-qdrant-collection.md` did not find it.
- **Section 5c still promises tier 3 pointers that none of the five delivers.** Either narrow 5c to what shipped, or add the pointers. The open scope question below has the argument.
- **The E4 reranker baseline.** The held-out comparison already selects both sides on the same half; 3a carries the check that settled it.
- **The three tier 2 knobs the retirement orphaned**, Matryoshka `mrl`, SPLADE and miniCOIL, and ColBERT as a retriever. The audit article's rebuild tier gained the paragraph at `before-tuning-a-qdrant-collection.md:131`, each with a rebuild condition. 4a carries the decision.
- **The `score_threshold` row** for the audit article's silent-settings table. It landed as a check rather than a value, at lines 100 and 105 to 106 of the same article.
- **5c's tier 3 pointers**, which none of the five delivered. 5c narrowed to `score_threshold`, the one that shipped.
**PR description must carry Dylan's philosophy for this series, added 2026-08-14.** The reader is an adult who already runs a Qdrant collection and has an AI coding agent on hand, so the series does not need to hand them an implementation to copy. It teaches what to check, what to measure, and what question to ask next, the way an experienced colleague would brief someone before they go write the code themselves, or hand the task to their agent. That is the reasoning behind favoring instructions and checks over full code in these articles, and it belongs in the PR body so a reviewer reads the series the way it was intended rather than as tutorials that happen to be light on code.
**The PR description carries these six points, all Dylan's and all added 2026-08-14.**
**Second point for the same PR paragraph, added 2026-08-14.** This series optimizes for what the reader takes away, not for exhaustive technical completeness. Every claim is measured, verified, and checked against the source, the same bar as any other Qdrant content. But where a technically complete treatment and a teachable one pulled in different directions, on scope, depth, or what to include versus link out to, this series chose teachable. The goal is a reader who can reason about their own collection afterward, not a reader who has seen every caveat.
- **No implementation to copy.** The reader is an adult who already runs a Qdrant collection and has an AI coding agent on hand. The series teaches what to check, what to measure, and what question to ask next, the way an experienced colleague briefs someone before they go write the code themselves or hand the task to their agent. That is why these articles favor instructions and checks over full code, and saying so in the PR body stops a reviewer reading them as tutorials that happen to be light on code.
- **Teachable over exhaustive.** Every claim is measured, verified, and checked against the source, the same bar as any other Qdrant content. Where a technically complete treatment and a teachable one pulled apart, on scope, depth, or what to link out to, this series chose teachable. The goal is a reader who can reason about their own collection afterward.
- **The small numbers, for Neil to decide on.** Every gain is measured against a fusion setting that was tuned first, so the swept gains are small by construction, which is why 3a's clearing counts and interval widths run tight. The baseline is a well-optimized hybrid search rather than a naive default, a second reason the numbers come in small. The corpora and their embeddings are still cached and the containers restart from their volumes, so a rerun on a more realistic dataset is cheap; it would not change the conclusions, but it would make the numbers look better. Leave that call to Neil.
- **Written for readers and for agents.** A reader working through their own collection may well be doing it through an agent, so the series is written to be as useful read by one as by a person.
- **Input from other teams is welcome.** These articles can be shared with new and existing customers, so they are material for pre-sales and post-sales too. Given the size of the diff, reviewers should feel free to propose direct inline suggestions rather than comments to respond to.
- **The experiments are not being released.** `experiments/` does not ship in this PR and Dylan does not plan to publish it separately. The series is about the learnings and the methodology, and he sees no point releasing the harness for numbers that are not the point.
**Third point for the same PR paragraph, added 2026-08-14: flag the baseline for Neil to decide on.** Every gain in the five articles is measured against a fusion setting that was already tuned first, so the swept gains are small by construction, this is why section 3a's clearing counts and interval widths run so tight. **Separately from that presentation choice, the E4 baseline has a correctness defect**, logged in 3a: it is selected on the queries it is reported on while the reranker beside it is not. That one is fixed before the PR rather than flagged in it. On top of that, the baseline itself is already a well-optimized hybrid search, not a naive default, which is a second reason the numbers come in small. The harness is still up and the corpora are still cached. Re-running on a more realistic dataset, if one can be sourced, would not change the series' conclusions, but it would make the numbers look better. Leave the call on whether that rerun is worth doing to Neil.
**Fourth point for the same PR paragraph, added 2026-08-14.** These articles are written for both human readers and AI agents. A reader working through their own collection may well be doing it through an agent, so the series is written to be as useful read by one as by a person.
**Fifth point for the same PR paragraph, added 2026-08-14: invite input from other teams.** These articles can be shared with new and existing customers, so they are useful material for pre-sales and post-sales too. Input from those teams on the PR is welcome. Given the size of the diff, reviewers should feel free to propose changes as direct inline suggestions rather than comments to respond to, since a full back-and-forth over five articles is a lot to read on both sides.
**Sixth point for the same PR paragraph, added 2026-08-14: the experiments are not being released.** `experiments/` already doesn't ship in this PR, per step 2 below, and Dylan does not plan to publish it separately either. The series is about the learnings and the methodology, not the numbers themselves, and he doesn't see the point of releasing the harness for numbers that aren't the point. He could, but isn't planning to.
**Done on 2026-08-13, so do not redo it.** The E7a float32 row was re-measured and the registered rule re-applied; section 8 carries both. The RAM article is now linked from the audit article's stage list, its symptom table and its scope paragraph, and the depth article's RAM section has shrunk to a pointer per section 13.
**Open question for Neil, added 2026-08-14: whether to redo all five hero images.** All five sets are built, per section 2, but the ones Dylan was able to make don't match the site's current hero style. Ask Neil whether they need a redo before the PR ships, rather than assuming either way.
**Two calls that are Dylan's and are not blockers.** Series positioning: whether to name the set publicly as a retrieval-quality series, leaving collection architecture and operations to a possible companion series, which is the structural review's Option 1 and still unmade. And Neil's length read: he flagged the set as possibly too comprehensive on 2026-08-11, when five articles ran 8,000 words; the five now run 8,739, down from six at 12,209 before the retirement and Dylan's narration pass. That length note is effectively answered. The independent-articles decision in section 13 stands unless Neil reopens it himself.
**Three calls that are not blockers.** **Hero images:** all five sets are built, per section 2, but they don't match the site's current hero style, so ask Neil whether they need a redo before the PR ships rather than assuming either way. **Series positioning:** whether to name the set publicly as a retrieval-quality series, leaving collection architecture and operations to a companion series, which is the structural review's Option 1 and still Dylan's to make. **Length:** Neil flagged the set as possibly too comprehensive on 2026-08-11 at 8,000 words; the five now run 8,581, down from six at 12,209 before the retirement and the narration pass, so that note is effectively answered. Two articles sit under the 1,500-word floor, per 4a. The independent-articles decision in section 13 stands unless Neil reopens it himself.
**Polish backlog**, none of it blocking: a shared pipeline diagram each article can point at; WANDS threaded as the consistent worked example wherever a corpus is named without a reason; a standardized short "How We Measured" block replacing each article's bespoke setup paragraph.
**One open scope question.** Section 5c specifies that every tier 3 knob gets a sentence and a docs link so a reader knows it exists. None of the five delivers that: distance metric, chunking, query rewriting, aliases, float16 and uint8, prefetch nesting, and `score_threshold` appear nowhere, and `is_tenant` is missing beside ACORN in the depth article. The 2026-08-13 review round declined the pointer block again, on the grounds that a list of seven links with no decision attached to any of them is the section a reader skips, in the longest article of the set. `score_threshold` is the one with a case, because silently emptying a fused result is what the audit's silent-settings table is for. Either it gains that one row, or 5c stops promising the rest. **Decided 2026-08-14: it gains the row, but the row is a check, not a value.** `score_threshold` itself isn't Tier 0, since the right value depends on the score distribution and the product's acceptance rule, a Tier 1 decision. The Tier 0 check is narrower and binary: don't carry a `score_threshold` into a fused query uncalibrated. Qdrant applies it to whatever score the returning stage produces, which for root-level RRF or DBSF is the fused score, not either leg's raw score, so a threshold copied from a single-vector setup can silently truncate the result list or empty it. Adding this row to the audit's table is in progress in a parallel session; do not re-litigate.
**The tier 3 pointer question, closed and not to be re-litigated.** Two review rounds declined a block of links for the tier 3 knobs, because a list with no decision attached to any entry is the section a reader skips. `score_threshold` was the one with a case, and it shipped on 2026-08-14 as a binary check rather than a value: don't carry a `score_threshold` into a fused query uncalibrated, because Qdrant compares it with whatever score the returning stage produces, which for root-level RRF or DBSF is the fused score rather than either leg's raw score, so a threshold copied from a single-vector setup can silently truncate or empty the result list. Choosing the value itself is a tier 1 decision that depends on the score distribution and the product's acceptance rule. `is_tenant` also landed, beside ACORN at `candidate-depth.md:170`. 5c narrowed to what shipped on 2026-08-17.
**Rejected by a review round, do not resurrect:** a workload worksheet in the hub, which belongs to the ops companion series; a running example that evolves across articles, since the WANDS threading above is the cheap version; and the reproducibility review's asks in section 13.
**Added 2026-08-14: the `turbo4` datatype, in the RAM article.** Missed on first draft because it shipped in v1.19.0, the same release the series pins to, and reads as a rename of TurboQuant quantization if you don't check the Rust. It is not: `Datatype::Turbo4` (`lib/segment/src/types.rs:2048`) is a storage datatype with no float32 copy at all, distinct from `TurboQuantization`, the quantization config the article's E7 measurements cover, which keeps one. Confirmed against the 1.19 release post (`qdrant-landing/content/blog/qdrant-1.19.x.md`), which states Qdrant cannot rescore against it since there is no full-precision copy to rescore against. It gets one paragraph, named and linked, no measurement, in "Where These Numbers Stop": E7 never ran it, and a knob we did not measure does not get a table per section 5's triage rule.
**`turbo4` is a storage datatype, not a rename of TurboQuant quantization.** `Datatype::Turbo4` (`lib/segment/src/types.rs:2048`) keeps no float32 copy at all, so Qdrant cannot rescore against it, confirmed against the 1.19 release post (`qdrant-landing/content/blog/qdrant-1.19.x.md`). `TurboQuantization`, which the article's E7 measurements cover, does keep one. It reads as a rename if you skip the Rust, which is how it missed the first draft. It shipped on 2026-08-14 as one paragraph in the RAM article's "Turbo4 Removes the Rescoring Option", named and linked with no measurement, because E7 never ran it and section 5's triage rule gives an unmeasured knob no table.
## 2. State Of The Artifacts
@@ -100,7 +91,7 @@ Gate A, `experiments/fusion/test_replay.py`: seven RRF fixtures ported from the
The measurement layer is sound. Do not rebuild it.
**The environment is warm.** All five collections are live in the `fusion-qdrant` container on the manifest's build (v1.19.0, commit `74f3e85b9`), and document embeddings are cached in `cache/*_dense.npy`. E6 also leaves `scifact_sq` and `dbpedia-entity_sq` behind, which are the int8 scalar quantized rebuilds. Nothing here needs re-embedding.
**The corpora are cached and the containers are stopped, checked 2026-08-17.** `fusion-qdrant` and `e7-qdrant` both exited on 2026-08-13 and need `docker start` before any live query; the five collections are in their volumes on the manifest's build (v1.19.0, commit `74f3e85b9`), and document embeddings are cached in `cache/*_dense.npy`. `verify_articles.py` reads artifacts and needs no server. E6 also leaves `scifact_sq` and `dbpedia-entity_sq` behind, which are the int8 scalar quantized rebuilds. Nothing here needs re-embedding.
## 3. What We Know
@@ -138,9 +129,9 @@ The Part column is the original ordinal from when the series was six pieces. 1 i
Two caveats ship with those rows. **The metric disagreement is partly structural**: WANDS has 358.9 relevant documents per query, so a query with that many relevant documents cannot pass 0.28 at Recall@100. The bound is per query, which is why the measured macro average reaches 0.3877. The lesson is that the metric has to match the relevance structure of the data. **The clearing counts are an upper bound**, since 31 arms are compared against the default at 95% with no multiplicity correction; the split-half check is the honest arbiter and carries the argument wherever a count currently does.
**Four E5 findings are now unpublished, logged 2026-08-14 so a future session knows they exist rather than re-measuring them.** The retirement of the second-prefetch article took its tables with it. "Which leg wins, per corpus" survives: it is the dense against sparse against fused table that now opens the fusion article. These four appear in no article: the reordering against new-candidates split, the union's relevant recall and ceiling at depth 200, the per-query helped-against-hurt admission counts, and leg agreement. Their artifacts are intact in `study/e5_second_prefetch.json` and their `verify_articles.py` checks were deleted along with the article, so nothing guards them now. They become publishable again only if a later experiment finds a downstream stage that turns the extra recall into a measured gain, which is the same condition 4a puts on reviving the article.
**Four E5 findings are unpublished and their artifacts are intact**, logged so a future session finds them rather than re-measuring: the reordering against new-candidates split, the union's relevant recall and ceiling at depth 200, the per-query helped-against-hurt admission counts, and leg agreement. They lived in the retired second-prefetch article, their `verify_articles.py` checks went with it, and `study/e5_second_prefetch.json` still holds them. "Which leg wins, per corpus" survives as the dense against sparse against fused table that opens the fusion article. The four become publishable again only if a later experiment finds a downstream stage that turns the extra recall into a measured gain, the same condition 4a puts on reviving the article.
**The E4 reranker baseline is selection-inflated and the reranker is not, found 2026-08-14 and unresolved.** `study.py:817` sets the baseline with `best_arm = per_query.mean().idxmax()`, the argmax over 31 fusion arms across the full query set, then scores that arm on those same queries. The reranker beside it goes through `_e4_held_out`, chosen on one half and reported on the other. The baseline keeps its selection luck and the challenger does not, which is the asymmetry gate 12 exists to prevent, and it runs in the direction that makes the reranker look worse. E1 sizes the effect: a swept winner keeps 67 to 95% of its gain on fresh queries, so with tuning gains of +0.0135 to +0.0471 the baselines are inflated by roughly 0.002 to 0.011. That is the same order as the reranker's deficits on SciFact, WANDS and CodeSearchNet.
**The E4 baseline gets the same treatment as the reranker, checked 2026-08-17.** `_e4_held_out` (`study.py:889`) picks the reranker configuration and the fusion arm on the same half and reports their difference on the other, and `study/e4_reranking.json` records `held_out.baseline` as `split_selected_fusion_arm` on all five corpora. The full-set argmax at `study.py:817` feeds only the `vs_best_fusion_arm` column, where the reranker is selected on the full query set too, as the best of 15 model-and-count configurations against the best of 31 fusion arms, and the article states those deltas use all 200 queries. Read `_e4_held_out` before reopening this.
**The baseline choice, not the measurement, is what makes the reranker article's headline.** Against the tuned arm at ten candidates the best reranker wins on one corpus of five. Against Qdrant's default RRF, which is what a reader runs before tuning anything, the same measurements give SciFact +0.0133, WANDS +0.0136, DBPedia +0.0459, CodeSearchNet +0.0020 and ArguAna -0.0200: three wins, one flat, one loss. Both readings are in `study/e4_reranking.json` as `vs_best_fusion_arm` and `vs_rrf_default`. **Do not quietly switch to the friendlier baseline**; the article's premise is that a reader tunes fusion first, which is the cheaper stage, and that premise is what earns the harder comparison. The two things to settle are whether the baseline gets the same held-out treatment as the challenger, which is a correctness question, and whether the article reports both columns, which is an editorial one.
@@ -199,9 +190,9 @@ Every measurement here comes from single-shard, unquantized collections of 5,183
- **Candidate depth is per shard.** `ShardQueryRequest` carries its own `limit` and every shard runs the full prefetch and returns its own top-`limit` (`lib/shard/src/query/mod.rs:36-51`). A reader on twelve shards setting `limit=200` is not fusing 200 candidates. Attach to prefetch `limit` in part 3.
- **Root-level fusion runs once at collection level; only a nested fusion inside a prefetch is per shard.** Already in section 6, currently buried as one audit row. It gates whether part 4's tuning applies as measured, so part 4 must say the word shard.
- **`indexing_threshold` is evaluated per segment, not per collection.** `is_optimization_required(&self, segment: &Segment)` compares one segment's `available_vectors_size_in_bytes` against the threshold, per named vector (`lib/shard/src/optimizers/indexing_optimizer.rs:53-90`, v1.19.0). The "roughly 6,700 vectors" figure the audit article quotes is therefore per segment, and a collection splits into `default_segment_number` of them, which defaults to the CPU count. A 50,000-vector collection across eight segments holds 6,250 per segment and can sit under the threshold on every one. Setting the value to 0 disables indexing outright (`config.rs:253-259`).
- **`indexing_threshold` is evaluated per segment, not per collection.** `is_optimization_required(&self, segment: &Segment)` compares one segment's `available_vectors_size_in_bytes` against the threshold, per named vector (`lib/shard/src/optimizers/indexing_optimizer.rs:53-90`, v1.19.0). The "roughly 6,700 vectors" figure the audit article quotes is therefore per segment, and a collection splits into `default_segment_number` of them, which defaults to one segment per two CPUs clamped to between two and eight (`lib/shard/src/optimizers/config.rs:232-239`), not to the CPU count, which is the 4c correction. A 50,000-vector collection across eight segments holds 6,250 per segment and can sit under the threshold on every one. Setting the value to 0 disables indexing outright (`config.rs:253-259`).
Verified against the local checkout, which currently sits at v1.18.2 while section 6 is pinned to v1.19.0. Re-verify these three at the pinned tag before drafting.
**Re-verified 2026-08-17 at the pinned tag**, since the local checkout now sits at v1.19.0, commit `74f3e85b9`, which is what section 6 pins to. All three hold: `ShardQueryRequest` still carries its own `limit`, `RescoreStages::collection_level` still puts root-level fusion at collection level, and `is_optimization_required` still takes one `&Segment`. The segment-count figure was wrong in this section and is corrected above.
## 4. The Series Shape
@@ -213,17 +204,17 @@ Word counts below come from `~/.claude/skills/neil-review/scripts/check_content.
| Slug | Title | Words | What it owns |
| --- | --- | --- | --- |
| `before-tuning-a-qdrant-collection` | What to Check Before Tuning a Qdrant Collection | 2,282 | The pipeline mental model, the tier 0 audit grouped as index state, correctness and performance, the symptom router, the cost-ordered ladder with per-query against collection scope, metric choice, labeled-set sizing, the held-out check, the rebuild floor |
| `candidate-depth` | Candidate Depth: How Much Retrieval Is Enough? | 1,428 | Prefetch `limit` as a sweep starting range, `hnsw_ef` as a check, the compressed "When RAM Is the Constraint" section: quantization (sole home of the E6 table), `memory` placement, the gated-knob closer |
| `when-your-collection-outgrows-ram` | When Your Collection Outgrows RAM | 2,279 | Memory placement per structure, the latency price of `rescore` across the RAM boundary, the recovery curve at the selected storage class, the block-read consistency check, the reader's own retention-and-latency check. Sole home of every E7 number |
| `how-to-tune-hybrid-search` | How to Tune Hybrid Search in Qdrant | 1,262 | Dense against sparse against fused, the fusion family, RRF `k`, weights, ties, the `k=60` to `k=61` port. Quantization robustness compressed to a pointer at part 3 |
| `when-a-reranker-is-worth-it` | When Is a Reranker Worth It? | 1,488 | Cross-encoder payoff, candidate count, model size, cost as capacity, the symptom-to-stage routing table, MMR, grouping, ColBERT as a reranker (sole home of its storage numbers) |
| `before-tuning-a-qdrant-collection` | What to Check Before Tuning a Qdrant Collection | 2,372 | The pipeline mental model, the tier 0 audit grouped as index state, correctness and performance, the symptom router, the cost-ordered ladder with per-query against collection scope, metric choice, labeled-set sizing, the held-out check, the rebuild floor |
| `candidate-depth` | Candidate Depth: How Much Retrieval Is Enough? | 1,431 | Prefetch `limit` as a sweep starting range, `hnsw_ef` as a check, the compressed "When RAM Is the Constraint" section: quantization (sole home of the E6 table), `memory` placement, the gated-knob closer |
| `when-your-collection-outgrows-ram` | When Your Collection Outgrows RAM | 1,989 | Memory placement per structure, the latency price of `rescore` across the RAM boundary, the recovery curve at the selected storage class, the block-read consistency check, the reader's own retention-and-latency check. Sole home of every E7 number |
| `how-to-tune-hybrid-search` | How to Tune Hybrid Search in Qdrant | 1,276 | Dense against sparse against fused, the fusion family, RRF `k`, weights, ties, the `k=60` to `k=61` port. Quantization robustness compressed to a pointer at part 3 |
| `when-a-reranker-is-worth-it` | When Is a Reranker Worth It? | 1,513 | Cross-encoder payoff, candidate count, model size, cost as capacity, the symptom-to-stage routing table, MMR, grouping, ColBERT as a reranker (sole home of its storage numbers) |
**Renamed 2026-08-13.** Title and slug changed for the hub article (`tuning-retrieval-what-to-check-first` / "Seven Qdrant Settings That Fail Silently" → `before-tuning-a-qdrant-collection` / "What to Check Before Tuning a Qdrant Collection"). Slugs only, title unchanged, for the other three: `what-a-second-retrieval-prefetch-buys` → `hybrid-search-recall-candidate-list`, `retrieval-candidate-depth-and-memory` → `candidate-depth`, `when-a-reranker-pays` → `when-a-reranker-is-worth-it`. Updated everywhere: article frontmatter and cross-links, `qdrant-landing/static/articles_data/` image directories, this table, and `experiments/fusion/verify_articles.py`'s slug map.
**Retired 2026-08-14.** `hybrid-search-recall-candidate-list` is deleted, with its preview images and its slug in `verify_articles.py`. Its finding duplicated `candidate-depth`: the best possible score rises, the current score barely moves. Its escape hatch, keep the second prefetch when a downstream stage can use the recall, is contradicted by `when-a-reranker-is-worth-it`, where extra candidates rescued none of the four losing rerankers. Two things moved rather than died: the dense against sparse against fused table now opens `how-to-tune-hybrid-search` as its premise, and the second prefetch's cost, 0.6 to 1.5 ms over dense alone, went into the audit article's cost-order table. The three inbound links, two in the audit article and the closer in `candidate-depth`, now point at the fusion article. Revisit only if a later experiment finds a reranker that wins on the extra candidates.
**What the retirement orphaned, and the one open decision it leaves.** The article was the only home for four tier 2 knobs in its "When the Retrieval Stack Has Hit Its Limit" section: Matryoshka `mrl`, SPLADE and miniCOIL, ColBERT as a retriever, and the pointer to `how-to-choose-an-embedding-model`. Checked across the five surviving articles on 2026-08-14: none of them appears anywhere, and ColBERT survives only in the reranker article, which was always its home per 5b. Four E5 findings also lost their only home; 3a logs them. **The decision to make before the PR:** either the audit article's rebuild tier gains one paragraph naming those four, since it already owns the cost-order ladder and the embedding model as the rebuild floor, or 5b stops listing them as covered. The same argument section 1 uses against the tier 3 pointer block applies here, so a list of four links with no decision attached is the wrong shape; a rebuild-tier paragraph that says when each one is worth the rebuild is the right one.
**What the retirement orphaned, and where it went.** The article was the only home for four tier 2 knobs in its "When the Retrieval Stack Has Hit Its Limit" section: Matryoshka `mrl`, SPLADE and miniCOIL, ColBERT as a retriever, and the pointer to `how-to-choose-an-embedding-model`. Checked across the five surviving articles on 2026-08-14: none of them appeared anywhere, and ColBERT survived only in the reranker article, which was always its home per 5b. Four E5 findings also lost their only home; 3a logs them. **Decided and applied 2026-08-14:** the audit article's rebuild tier gained one paragraph naming those four, since it already owns the cost-order ladder and the embedding model as the rebuild floor. It sits at `before-tuning-a-qdrant-collection.md:131` and gives each knob its rebuild condition rather than a bare link, because the same argument section 1 uses against the tier 3 pointer block applies here.
**Reading order by weight:** audit -214, candidate depth -213, fusion -211, reranker -210, RAM -209. Depth sits before fusion because the fusion article opens by telling the reader to set a prefetch depth. -212 is free since the retirement; leave the gap rather than renumbering four files.
@@ -233,7 +224,7 @@ The entry point is the audit article: its symptom table routes to the other four
**Why this exists at all**, and the line to keep if the set ever gets rewritten: `/course/essentials/` is 46 pages and 47,735 words that teach what each knob is. Its day 3 covers dense against sparse and computes an RRF table without ever mentioning `k`, weights, or DBSF. Nothing on the site teaches which knob to turn, in what order, and what it costs.
**Length calibration.** The five sit at 1,262 to 2,282 words and total 8,739 (recounted 2026-08-14 after Dylan's narration pass, which cut roughly 1,150 words, most of it from the fusion and reranker articles). Three now sit under the script's 1,500-word floor for an article: `how-to-tune-hybrid-search` at 1,262, `when-a-reranker-is-worth-it` at 1,488 and `candidate-depth` at 1,428. `/neil-review` will flag all three. The set is deliberately lean and Neil's own note was that it ran too comprehensive, so the likely answer is that the floor is the wrong bar here; decide it once, before the PR, rather than three times during it. For comparison: the ACORN article is 2,258, the longest article on the site is 4,924 (`modern-sparse-neural-retrieval`), and the median across 80 is 1,607. Length was not the problem the style pass fixed; density and flat narration were.
**Length calibration.** The five sit at 1,276 to 2,372 words and total 8,581, recounted 2026-08-17 with the script named above. The revision 27 figures were wrong, by 3 to 25 words on four articles and by 290 on `when-your-collection-outgrows-ram`, which has been between 1,989 and 1,994 words at every commit on this branch. Two sit under the script's 1,500-word floor for an article: `how-to-tune-hybrid-search` at 1,276 and `candidate-depth` at 1,431. `/neil-review` will flag both. The set is deliberately lean and Neil's own note was that it ran too comprehensive, so the likely answer is that the floor is the wrong bar here; decide it once, before the PR, rather than three times during it. For comparison: the ACORN article is 2,258, the longest article on the site is 4,924 (`modern-sparse-neural-retrieval`), and the median across 80 is 1,607. Length was not the problem the style pass fixed; density and flat narration were.
**Narration baseline.** The style register comes from `qdrant-landing/content/articles/filtered-vector-search-acorn.md` on the `acorn-article` branch. What it does that these now do: section headings assert the finding rather than naming the topic, mechanism is explained before the number lands, and the reader is told how to read the numbers ("read the ratios, not the absolute milliseconds").
@@ -264,7 +255,7 @@ Each of these shipped wrong at some point in drafting and was caught by a gate.
| The wrong version | The measured truth |
| --- | --- |
| `params=models.SearchParams(...)` in a snippet | `client.query_points` takes `search_params`. `models.Prefetch` takes `params`. The wrong one raises `AssertionError: Unknown arguments` for every reader |
| `indexing_threshold` defaults to 100,000 KB, about 66,000 vectors | 10,000 KB, about 6,700 vectors at 384 dimensions, measured per segment. Filed as documentation fix 10 |
| `indexing_threshold` defaults to 100,000 KB, about 66,000 vectors | 10,000 KB, about 6,700 vectors at 384 dimensions, measured per segment |
| A collection splits into as many segments as CPUs | One segment per two CPUs, clamped to 2 through 8 |
| ACORN engages below `max_selectivity` 0.4 by default | ACORN is disabled until you set its `enable` flag. The threshold only applies once enabled |
| Move payload to disk to save memory through `on_disk` and `on_disk_payload` | Both are deprecated since 1.19. Placement is `memory` on six structures, and payload already defaults to `cold` |
@@ -285,11 +276,11 @@ Each of these shipped wrong at some point in drafting and was caught by a gate.
| "With 50 queries, skip the sweep, the procedure will keep the default almost regardless of what is true" | E2 disagrees per corpus: a sweep at 50 confirmed the corpus's own best gain in 7% to 38% of draws where that gain was under 0.02, and in 93% on WANDS where it was 0.038 |
| "Take the setting closest to `k=2`, since the extremes are where the ties live" | `k=2` is the extreme with the worst measured tie rate, 12.5% against 2.8% at `k=61`. Ship what clears the split, and prefer equal weights because a weight pair is absolute |
| "A large gap means no amount of retrieval tuning will move it" | Depth raises the best achievable score by 0.10 to 0.28, so retrieval does move the gap. What barely moves is the score the reader sees |
| "Almost everyone reading this will find a large gap" | The gap was 0.14 to 0.51 on the five corpora here. Five corpora are not everyone |
| "Almost everyone reading this will find a large gap" | The gap ran 0.247 to 0.487 at candidate depth 200 on the five corpora here, which is what the reranker article publishes and `verify_articles.py:156-157` locks. Five corpora are not everyone. **The 0.14 to 0.51 this row carried through revision 27 matches no artifact**; do not restore it |
| "The gain will be largest where the two prefetches disagree enough to be informative and still overlap enough to vote" | Nothing measured predicts the size of the gain from leg agreement. CodeSearchNet has the lowest agreement at 0.418 and the largest reordering gain; DBPedia has 0.901 and no gain |
| "A cross-encoder earns its cost by making fine distinctions among many plausible documents" | Relevant documents per query is measured; the mechanism is inferred from five outcomes. Say which is which |
| "Somewhere above this scale the graph stops saturating" for `hnsw_ef` | Point count is one input. Vector distribution, filters and query difficulty move the same line, so no collection size tells a reader which side of it they are on. Describe the regime by the saturation check |
| "Depth is cheap in latency" | It cost 40% to 45% between `limit` 10 and 500 on one idle single-shard laptop. Cheap against a model call, not against a tight p95 or a shard fan-out |
| "Depth is cheap in latency" | It cost 37% to 43% between `limit` 10 and 500 on one idle single-shard laptop, which is the measured range in `e3_latency` and in the depth article; **the 40% to 45% this row carried through revision 27 is wrong**. Cheap against a model call, not against a tight p95 or a shard fan-out |
| TurboQuant `bits4` "reached float32 quality" | It matched float32 on labeled nDCG@10 and kept 0.039 less of the exact top 10. Name which quality measure matched |
| The roughly thirty-fold CPU to GPU reranker ratio read as a comparison | Our CPU figures and sbert.net's GPU figure come from different machines and different documents. It is the order of magnitude serving hardware moves, not a controlled result |
| Over-fetching to 50 pins tie-boundary membership | Only if the whole tied group fits inside the larger response. Compare the rank-10 score with the last fetched score and raise the limit while they match |
@@ -329,13 +320,13 @@ Forty-one knobs in four tiers plus a cut list. The tiering governed what the art
### 5b. Tier 2: Gated
The home column names the article that covers the knob. **The three reading `none` lost theirs to the 2026-08-14 retirement**, and 4a carries the decision that has to be made about them before the PR.
The home column names the article that covers the knob. **The three that lost theirs to the 2026-08-14 retirement moved to the audit article's rebuild tier on 2026-08-14**, one sentence each with a rebuild condition; 4a carries that decision.
| Knob | Home | The gate |
| --- | --- | --- |
| Matryoshka `mrl` and embedding dimension | none | Only if you are memory-bound. Trades quality for size, 6.18 MTEB points from 768 to 64 |
| SPLADE, miniCOIL | none | Only if core BM25 underperforms on your vocabulary. Model inference on every document and query |
| ColBERT as a retriever, with MUVERA | none | Only if you need the ceiling and have the storage: 286 GiB per 9M passages at 128 dimensions. **ColBERT as a reranker over already-fetched candidates carries no such cost and belongs in part 5's tier 1 discussion instead**, so do not let one gate turn both readers away |
| Matryoshka `mrl` and embedding dimension | audit | Only if you are memory-bound. Trades quality for size, 6.18 MTEB points from 768 to 64 |
| SPLADE, miniCOIL | audit | Only if core BM25 underperforms on your vocabulary. Model inference on every document and query |
| ColBERT as a retriever, with MUVERA | audit | Only if you need the ceiling and have the storage: 286 GiB per 9M passages at 128 dimensions. **ColBERT as a reranker over already-fetched candidates carries no such cost and belongs in part 5's tier 1 discussion instead**, so do not let one gate turn both readers away |
| `m`, `ef_construct` | depth | Only if you can rebuild. `m` costs memory permanently, `ef_construct` costs build time only. At scale these are one-shot design decisions, so say once that they bound everything the series tunes |
| `quantile` and quantization type selection | depth | Only after quantization is on, which for most collections above RAM it already is |
| Filterable HNSW, ACORN, and tenant-aware indexing (`is_tenant`) | depth | Only if you filter, which every multi-tenant collection does by definition. ACORN default `max_selectivity` 0.4, and 2 to 10x slower when it engages |
@@ -346,7 +337,9 @@ The home column names the article that covers the knob. **The three reading `non
### 5c. Tier 3: Named And Linked
Distance metric, which is match what the model was trained with. Chunking strategy, which decides what a document even is, is not a Qdrant setting, and has no measurement of ours, so it gets a pointer to the course. Query rewriting and expansion, usually the largest lever and also outside Qdrant, with no page anywhere. Collection aliases, because tier 1 says the embedding model needs a rebuild to change and for this reader the hard part is doing that rebuild without downtime; one sentence and a link to the model-migration docs. Vector datatype float16 and uint8, one line inside the memory discussion. `exact`, a diagnostic rather than a setting. Prefetch nesting. `m=0` on rescore-only vectors, a pure memory saving. `score_threshold`, which on a fused query truncates at the first point below the threshold and can silently empty a result set, shipped as a warning.
**Narrowed 2026-08-17 to the one pointer that shipped.** `score_threshold` is in the audit article's silent-settings table as a check: don't carry a threshold into a fused query uncalibrated, because Qdrant compares it with the fused score and can silently truncate or empty the result list.
The other eight stay listed here for a companion series and appear in none of the five, which is deliberate. Two review rounds declined a pointer block, on the grounds that eight links with no decision attached is the section a reader skips: distance metric, which matches what the model was trained with; chunking strategy, which decides what a document even is, is not a Qdrant setting, and has no measurement of ours; query rewriting and expansion, usually the largest lever and also outside Qdrant; collection aliases, for rebuilding the embedding model without downtime; the float16 and uint8 vector datatypes; `exact`, a diagnostic rather than a setting; prefetch nesting; and `m=0` on rescore-only vectors, a pure memory saving.
## 6. Verified Facts, Do Not Re-Derive
@@ -496,13 +489,11 @@ Four limits. Each bounds a claim rather than invalidating one; every number in 3
Five ArguAna queries were dropped because its own qrels reference five documents its corpus does not contain.
## 8. What Still Needs Measuring
## 8. The E7 Measurement Record
**E1 to E6 are done.** Their results are the rows in 3a, the code is in `experiments/fusion/study.py`, the artifacts are in `experiments/fusion/study/`, and the claim trail is in `grounding.md`. Do not re-run them. If one ever has to be re-run, 4d holds the two constraints that cost the most to rediscover: `e4_score` caches per corpus and model so it resumes rather than restarting, and `e3_latency` needs an idle machine.
E7 was the only measurement left. It is run, and the results are in this section.
**Not recommended:** a Matryoshka sweep. It needs a rebuild per dimension and section 6 has per-dimension numbers from a primary source.
Nothing is queued for measurement. E7 was the last one and it ran on 2026-08-12; this section is its record, kept because the measurement discipline in it is expensive to reacquire and because it is what defends the RAM article's numbers under review. A Matryoshka sweep stays unrun: it needs a rebuild per dimension, and section 6 already has per-dimension numbers from a primary source.
**E7. The RAM boundary and the price of recovery, two experiments behind one article. Run on 2026-08-12; the results are below.** Adversarially reviewed and overlap-reviewed on 2026-08-11, then reviewed three more times during execution; the constraints below are those reviews' output and are not optional. Where a constraint was changed, the change and its date are recorded with it.
@@ -510,7 +501,7 @@ E7 was the only measurement left. It is run, and the results are in this section
**One collection, two container memory limits. Dylan's simplification, 2026-08-11, replacing the two-ingestion design.** The regime boundary is relative, so the fits-in-RAM and exceeds-RAM regimes come from running the same collection under two Docker memory limits, not from ingesting a corpus at two sizes. That removes the corpus-size confound between regimes: same data, same graph, same labels, and RAM is the only variable.
**The corpus is DBPedia-entity's full 4.6M-document BEIR corpus**, of which the harness's current collection is the 100k sample: same queries, graded labels, license handling, and embedding model already in `experiments/fusion/`, which also keeps the experiment public alongside the others. At 384 dimensions the originals are about 7 GB of float32, so the boundary is real on a laptop. One overnight embedding pass is the only new ingestion cost. Fallback if that pass is unacceptable: the existing 100k collection under a proportionally small limit, disclosed in the article as a scaled-down demonstration. Confirm the BEIR distribution's license before downloading, since section 7 cleared only the sampled version.
**The corpus is DBPedia-entity's full 4.6M-document BEIR corpus**, of which the harness's other collection is the 100k sample: same queries, graded labels, license handling, and embedding model already in `experiments/fusion/`. At 384 dimensions the originals are about 7 GB of float32, so the boundary is real on a laptop.
**Dense only. No sparse prefetch, 2026-08-11.** The knob under test acts on dense vector storage and on the quantized copy of it, which section 8 already scoped the claims to. A BM25 prefetch in the same request adds about 1.5 GB to the working set, and that memory competes for the same page cache the experiment is trying to starve, so under the tight limit the evicted structure could be the sparse index rather than the originals. Fusion would also hide the effect on the metric, because a sparse prefetch rescues documents the dense side lost. E7 therefore measures the dense stage on its own, and the article hands the reader the fused check rather than claiming their fused latency is unaffected: the sparse stage does the same work either way, and it shares the page cache, so the reader picks a dense placement and recovery setting here and then reruns their own fused request to see the end-to-end number.
@@ -542,7 +533,7 @@ E7 was the only measurement left. It is run, and the results are in this section
- The four rules those six cells support, which are what the article teaches: pin whatever the initial search reads, test `rescore` at your own memory cap rather than a generous one, spend RAM caching originals only if it moves your tail, and treat `rescore` off as a control rather than a configuration to benchmark everywhere.
- **These are placement policies, not observed states, and the rule keys off what was observed.** Under the tight limit `cached` originals are evicted anyway, so a rule written on the configuration label would be wrong for the reader who sets that label and gets eviction. Every row carries its measured refault and block-read counts, and the article's rule reads off those rather than off the setting.
- **The `rescore` off rows are the negative control.** With no rescore there is no reread of an original, so moving originals between `cached` and `cold` should not move latency. If it does, the treatment is confounded and the cold rows do not carry a claim. The control ships as a sentence, not a table.
- - **Configure and measure in different containers.** Changing a placement starts an optimizer pass whose transient spike exceeded the 12 GiB limit and had the kernel kill the server twice. Configuration now happens on a roomier container, and the measured container is started fresh at the cell's own limit with its configuration already correct. That also keeps optimizer work out of the measured window, which would have contaminated the latency even when it did not kill anything.
- **Configure and measure in different containers.** Changing a placement starts an optimizer pass whose transient spike exceeded the 12 GiB limit and had the kernel kill the server twice. Configuration now happens on a roomier container, and the measured container is started fresh at the cell's own limit with its configuration already correct. That also keeps optimizer work out of the measured window, which would have contaminated the latency even when it did not kill anything.
- **Readiness is a successful query, not a status field.** This collection reports `grey` while its shard is `Active` and serving. `Grey` means "optimizations are possible but not triggered" (`lib/collection/src/operations/types.rs:66`), so it is a settled state after a configuration change, and waiting for the literal `green` waits for something only an update operation triggers. Two separate hangs came from asserting on how a value prints rather than what it is: `str(shard.state)` is `Active`, not `ReplicaState.ACTIVE`. Compare enum values.
- **Every cell starts from a controlled cache state, and this is not optional.** Page cache is charged to the cgroup that first faults a page in, and the charge outlives the container that made it, so recreating a container over a warm volume hands it gigabytes of resident pages it is never billed for. Measured on 2026-08-12: reopening the 4 GiB limit that way, the cgroup reported 119 MB of file cache while the process held 9.49 GB, `memory.events` recorded no pressure at all, and 400 queries ran in 2.9 s. From a controlled cache state the same limit reported 4.29 GB of 4.29 GB used, 37,888 limit events, 6,065,509 refaults, and 158.1 s. Dropping the cache while the server still has the files mapped does not work either, because `drop_caches` frees only clean unmapped pages. The sequence is stop the container, `sync`, drop the VM page cache, check it fell to its floor, then start a fresh container. `e7.py cold <limit>` does exactly that and refuses to continue if the cache did not drop.
- Latency protocol. Every cell begins from the controlled cache state above and receives the same fixed warm-up pass before its measured pass, so "warmed" means a stated number of passes rather than whatever the previous cell left behind. Order is randomized within a round, seeded by the round, so a round is a complete replicate and no cell always follows the same neighbour. Settings are never randomized inside one long-lived container, because under a tight limit the previous cell's pages decide which of this cell's pages survive. One dense request per query, on an idle machine per 4d. A cold-start arm is no longer separate, because every cell now starts cold by protocol. **Superseded, kept only to explain the shape of the artifacts:** cold originals with `rescore` off and on under the tight limit, one setting per container boot, no randomization, and the host file cache dropped with `purge` before each boot, because randomizing settings inside a live server warms the pages the next setting reads and a container restart on its own leaves both cache layers intact. Running cold start across the whole matrix would double the results surface to answer a question the article does not ask. With 400 queries the cold arm is 400 observations and no more, so it reports the first-pass distribution, p50 and p95 with bootstrap intervals, and paired per-query differences. No p99.
@@ -715,8 +706,8 @@ Two artifacts need reconciling. `floor/scifact.json` reports `sparse_caches_iden
| Who is the reader on the day they open this? | Someone already running two prefetches and a fusion. Dylan's call, 2026-08-11, after the drafts read as basics to that reader. It cost two retitles and one reframe, both on articles whose titles the earlier rounds argued over because they asked a question this reader had answered. The surviving half of that call is the hub going to `Seven Qdrant Settings That Fail Silently`, since "Before You Tune Search" reads pre-adoption; the other half was applied to the article retired on 2026-08-14. The rule that came out of it is in the brief and in section 11. **Narrowed 2026-08-14: this reader profile holds for the fusion article only, after the second-prefetch article was retired.** The other four don't assume a second leg; see the North Star after the brief's hybrid paragraph |
| Does the E7 deployment point stay at `bits1` oversampling 1? | **Yes, and since 2026-08-13 the held-out half confirms it.** The article recommends `bits1` with `rescore` on at oversampling 1, described as chosen by the pre-registered rule and confirmed on the queries that had no say in choosing it: 0.0011 better than float32 on nDCG@10 there, paired 95% interval -0.003 to +0.005, retention 0.006 below float32, for 7.121 GB of vectors compressed to 0.260 GB. Oversampling 2 is now just another point on the published curve and carries no special status, so the robustness framing it used to need is gone. **Every published E7b latency number was measured at oversampling 1**, so a recommendation of 2 would put a latency table under a configuration it never measured, which the benchmark-measures-the-recommendation gate forbids. Never write "equivalent to float32"; quote the interval instead |
| Does the depth article's RAM section shrink in this PR? | **Yes. Settled 2026-08-12**, reversing the earlier "not now", whose only reason was that the RAM article's URL did not exist. It does now, and both articles ship in one PR. What stays in depth: the rule to test quantization before reducing candidate depth, and the E6 table, whose sole home it is. What becomes a pointer: the "Once the collection outgrows RAM" paragraph, meaning placement, the `memory` parameter and the latency price of `rescore`. Shrinking re-runs the gate order and `verify_articles.py` over the locked figures, which is the cost and is accepted |
| What happened to the E7a int8 cells? | **Cut from the article, and the float32 row is under re-run.** Section 8 carries the mechanism and the fix. Do not resurrect the int8 rows as a storage ladder: that is the TurboQuant article's job, and the publication discipline in section 8 forbids a second one |
| The three review passes of 2026-08-12 (one Fable agent, two Codex) | **Taken:** every factual defect they found in the RAM article, listed in section 8, and seven corrections to the others, listed in 4c. **Rejected, do not resurrect:** trimming the reranker article's held-out paragraph to one sentence, because gate 12 requires each article to disclose that its own winner was selected and reported on different halves; and cutting the `bits4` rescoring row, which is what shows that rescoring buys retention the nDCG column does not reveal. **Open:** the tier 3 pointer question in section 1 |
| What happened to the E7a int8 cells? | **Cut from the article, and the float32 row was re-run on 2026-08-13.** Section 8 carries the mechanism and the fix, and `e7/results/e7a_float32.json` is the file anything comparing against float32 reads. Do not resurrect the int8 rows as a storage ladder: that is the TurboQuant article's job, and the publication discipline in section 8 forbids a second one |
| The three review passes of 2026-08-12 (one Fable agent, two Codex) | **Taken:** every factual defect they found in the RAM article, listed in section 8, and seven corrections to the others, listed in 4c. **Rejected, do not resurrect:** trimming the reranker article's held-out paragraph to one sentence, because gate 12 requires each article to disclose that its own winner was selected and reported on different halves; and cutting the `bits4` rescoring row, which is what shows that rescoring buys retention the nDCG column does not reveal. **Closed 2026-08-17:** the tier 3 pointer question, by narrowing 5c to `score_threshold` |
| The reader-quality review (Codex, 2026-08-13) | **Taken:** fourteen items, the prose ones now in 4c. The Recall@100 passage, `hnsw_ef` without a size threshold, per-shard `limit` in the fusion article, the conditional tie workaround, five companions not four, reranking listed once in the cost ladder, two over-length `short_description` fields, the `bits4` quality measure named, the CPU-to-GPU ratio decontrolled, "depth is cheap" made relative, and the rebuild-variance section given the production sources it did not test. **Rejected, do not resurrect:** the tier 3 pointer block, per section 1; an "Advanced Edge Cases" heading over the fusion article's weight-zero and query-less-prefetch sentences, since a heading over two sentences inside the weights section is scaffolding; and retitling the audit's rebuild-variance section to "A Clean Rebuild Did Not Move the Top 10 Here", because the claim needed the fix and the title already hedges. **Its plan-hygiene finding was correct** and this revision is the fix |
| The five-article engineering reproducibility review (Codex, 2026-08-11) | **Taken:** the reranker stop rule diagnoses model and input mismatch before it tells anyone to stop buying candidates, and the reranker article surfaces its held-out footing, which was the thinnest of the five. **Rejected, do not resurrect:** p95 and p99 under concurrency, multi-shard latency, GPU throughput, and cost per fixed query volume, all of which need hardware we do not have and would ship numbers section 12 forbids; the solo, team and enterprise labeling paths and annotation guidelines, which turn the hub into three articles against Neil's length note; a second embedding model or learned sparse retrieval in part 2, which is the re-embedding section 7 rules out; and splitting depth from memory as separate articles, which the RAM article now answers instead. **Deferred, worth revisiting after publication:** the reviewer's one strong idea, a downloadable harness taking a collection, queries, labels and a variant config and returning pass, reject or inconclusive. `experiments/fusion/` plus the notebook is most of it already, and section 14 moves that code to its own repo, so decide there whether it gets packaged for readers. Half the review's findings came from reading the drafts without the artifacts, so brief any future reviewer with this plan |