Mimiron AI

Live results from my latest test · 112 test questions

Did the search find the answer?

  1. Someone asks a question.
  2. A search pulls the 5 most useful-looking pieces from the Kubernetes manual.
  3. The AI answers using only those pieces.

That is RAG. The answer can only be as good as the pieces it gets, so below: did the search pull the right ones? Pick a question, then slide across the pieces to see what an AI grader said about each.

For engineers: proof this page is live · 4 calls · slowest 40 ms

These calls ran live for your visit

  1. Latest test run (run 10, 18 setups) · 30 ms
  2. Results for “run a script before container…” · 35 ms
  3. Results for “How to define a liveness comm…” · 39 ms
  4. Results for “restartPolicy Never vs OnFail…” · 40 ms

Times are from this site's server to the test-results service, not your page load.

Pick a test question
Pick how it searches

Across all 106 test questions, this search method found more than keyword search on 36, the same on 62, fewer on 8.

Meaning + keywords, re-sorted: 4 of 5 answer it; first answering passage at position 1.

Passage 1 of 5 · 5 graded

  1. 1 Container states

    AI grader · Answers it

    The passage directly answers the question by explaining that a preStop hook runs before the container enters the Terminated state, which is exactly what's needed to run a script before container shutdown for connection draining.

    Closest to the questionWhen you use kubectl to query a Pod with a container that is Terminated, you see a reason, an exit code, and the start and finish time for that container's period of execution.

  2. 2 Types of probe

    AI grader · Answers it

    The passage directly addresses connection draining during container shutdown by explaining how Pod termination works and referencing the Pod termination documentation that covers lifecycle hooks for graceful shutdown.

    Closest to the questionReadiness probes run on the container during its whole lifecycle.

  3. 3 Define postStart and preStop handlers

    AI grader · Answers it

    The passage directly answers the question by explaining how to use preStop handlers to run a script before container shutdown, with a concrete example showing graceful connection draining (nginx -s quit with a wait loop).

    Closest to the questionThe preStop command shuts down nginx gracefully.

  4. 4 Untitled section

    AI grader · Answers it

    The passage directly answers the question by explaining that Kubernetes supports preStop events that execute immediately before a Container is terminated, which is exactly what's needed to run a script before container shutdown for connection draining.

    Closest to the questionKubernetes sends the postStart event immediately after a Container is started, and it sends the preStop event immediately before the Container is terminated.

  5. 5 Container probes

    AI grader · Off topic

    The passage discusses liveness and readiness probes but does not address running scripts before container shutdown or connection draining, which requires information about lifecycle hooks like preStop.

    Closest to the questionIf a container fails its liveness probe more times than the configured tolerance, the kubelet restarts that container.

Grades are by an AI. A person re-graded 20 of them on 2026-10-16: 13 matched exactly, the other 7 were one step off, and where they differed the AI was the generous one. What we got wrong

Why look things up at all

The same AI answered this question twice.

From memory

Graceful Shutdown with Connection Draining

Use a preStop hook to run a script before the container terminates:

``yaml apiVersion: v1 kind: Pod metadata: name: example spec: containers

  • name: app

image: myapp:latest lifecycle: preStop: exec: command: ["/bin/sh", "-c", "/scripts/drain.sh"] terminationGracePeriodSeconds: 30 ```

drain.sh example: ```bash #!/bin/bash # Signal app to stop accepting new connections kill -TERM 1

Wait for existing connections to close sleep 15 ```

Key points:

  • preStop runs before SIGTERM is sent to the container
  • terminationGracePeriodSeconds (default 30s) is the total time allowed before force kill
  • Script should complete before the grace period expires
  • Combine with app-level graceful shutdown (e.g., Spring Boot's server.shutdown=graceful)

For HTTP services, also consider:

  • Setting readinessProbe to fail immediately when shutdown starts
  • This removes the pod from load balancers before draining

After looking it up

Based on the passages provided:

To run a script before container shutdown so connections can drain, you should use a preStop hook.

According to the passages, Kubernetes sends the preStop event immediately before a container is terminated. If a container has a preStop hook configured, this hook runs before the container enters the Terminated state.

You can attach a preStop handler to container lifecycle events. This allows you to execute a script that can gracefully drain connections before the container is shut down.

Additionally, when a pod is evicted or deleted, the endpoint's ready condition is set to false, which prevents load balancers from sending new traffic to the pod. Combined with a preStop hook, this enables proper connection draining during shutdown.

Sources

  • task-lifecycle-handler.md — section 1
  • pod-lifecycle.md — section 4

Sources: pod-probes-concept.md — Types of probe; pod-lifecycle.md — Container states; task-lifecycle-handler.md — ; pod-disruptions.md — Pod disruption budgets; pod-lifecycle.md — Container probes.

From an earlier test: looked up 2026-10-13, from memory 2026-10-14, not live.

How one grade becomes a score

Question run a script before container shutdown so connections drain

  1. #1Container states2
  2. #2Types of probe2
  3. #3Define postStart and preStop handlers2
  4. #42
  5. #5Container probes0

Judge grade: 2 answers it · 1 related · 0 off topic

Hit@51.00an answering passage is in the top 5 → 1
Precision@50.804 answering ÷ 5 retrieved
Recall@50.674 found ÷ 6 that exist
F1@50.732 × 0.80 × 0.67 ÷ (0.80 + 0.67)
MRR1.001 ÷ position 1 of the first answering passage
nDCG@50.87DCG 5.12 ÷ ideal 5.90 (grade ÷ log₂(position+1), summed; ideal = 5 answering passages first)
One question as the evaluation run scored it, through the winning setup. The scoreboard is each of these, averaged over every answerable question.

Averaged over the run, this setup's recall@5 is 0.70 and precision@5 is 0.44.

Each line is one chunking method with one search mode, graded on 106 answerable questions. Its lower dot is without the reranker, its upper dot with it: 9 lines, 18 setups. Every line points up: the reranker helped every time.

0.750.800.850.900.560.600.640.68

Winner Recursive · Hybrid · Reranker

Semantic · Hybrid: a statistical tie ↓

BM25 alone: misses paraphrased questions

Recall@5: finds more of the evidence →

↑ MRR: right passage ranked first

without reranker with reranker hybrid vector BM25

Winner · run 10

Recursive chunking + Hybrid retrieval + Cross-encoder reranker

0.70Recall@5 (joint highest)
0.89MRR
101/106Hit rate@5
All 18 configurations →

From an earlier test (run 10), not live

Semantic chunking led on my first 58 questions. On 54 new ones, it lost.

Choosing it after the first test would have been overfitting to a small question set. On all 112 the two tie: paired bootstrap, 95% interval −0.08 to +0.03 on recall@5. On a tie the site uses the simpler one, recursive, which needs no embedding pass per sentence to index. Source: mimiron-ai docs/api/12-eval-results.md.

0.710.670.680.72
First 58 questions54 new questions
Semantic · Vector · Reranker Recursive · Hybrid · RerankerRecall@5
Test resultsAll 18 setups, every metric, and how each passage was graded.PlaygroundAsk your own question and change how it searches.How it is builtEvery stage, every quality check, and the full scoreboard.What went wrongNine mistakes this project made, and how each one was caught.