Ask it about your own numbers
The exact (Clopper-Pearson) interval for one run; McNemar's mid-p and Tango's interval for two, paired case by case. PASS only when the whole interval clears the bar, and a difference only when its interval leaves zero.
What it said about real systems
Greek, graded right
A log line, scrubbed before it becomes a test
Greek tax numbers, AMKA and IBAN are taken only when their check digits hold. Names are the weak part: a capitalised pair is taken as a name, so a name typed in lower case gets through and a product with a capital letter does not.
Run it on your own cases
git clone https://github.com/Broikos-Nikos/evalbar && cd evalbar npm ci npx evalbar run your-cases.jsonl --base-url http://localhost:8080/v1 --model your-model --bar 0.9
A case is one JSON line: {"input": "...", "expected": "...", "tags": ["el"]}. Any OpenAI-compatible endpoint, cloud or local, or a program of your own. Exit codes are 0 for PASS, 1 for FAIL and 3 for CANNOT TELL, so a CI step can stop on the first two. Node 24, no runtime dependencies.