Coding assistant + evals
30 synthetic notes with the codes a careful coder would assign. A run sends every note through the assistant with the current prompt, one at a time, and scores each answer by set overlap: precision, recall, and F1 on the codes, with a pass only for an exact match. Each run is compared with the last finished run of a different prompt version, so you can see what an edit bought or broke.
Prompt v1
Fake adapter · offline, deterministicTry it
Run the evals on v1, then add this line to the prompt, save, and run again. Several golden notes mention a finding only to deny it or rule it out, and v1 codes them anyway.
Do not code conditions the note says are denied, ruled out, or negative.
Offline, the Fake adapter is a keyword matcher that honours exactly this instruction, so the score moves; it still codes family history, which is the kind of miss a real model has to be prompted out of.
Runs
No runs yet. Run the evals to score prompt v1.