The evaluation was testing the wrong search
What we believed. Runs 2 to 5 measured the search the site uses.
What the measurement showed. Every chunk stores two kinds of vector. The evaluation searched with the older MiniLM vectors; the site searches with the arctic ones. Nothing failed, because both were there.