László Fazakas

Earlier this year I built an instrument to test whether language models rebuild the layer of assumptions a legal standard takes for granted. It gave me a clean result: models from different countries came out differently, and the difference held up statistically.

Then I built the control the result deserved, and the result went away. With the word order shuffled and the token counts matched, the gap fell to +0.09, with a confidence interval running from −0.08 to +0.25. I had been measuring how fluently the sentences read.

Something did survive, and I hadn't gone looking for it. There's a default that shifts with the language you ask in, and it's the same whichever country the model came from. So it's in the training data, not the lab.

The part that stayed with me is what the instrument couldn't do. It gave me a number, and the number was fine. It just wasn't about the thing I thought I was measuring, and nothing in it could have told me so, because the question I was asking was built into how I asked it.

I keep running into that shape. A proof is sound inside a type system somebody chose. A coverage score can't see what was left out, if the list of what was owed came from the thing being scored. Models agree most where they share a blind spot. A rule that a person must supervise the machine assumes the person still can.

Two different situations, one reading on the dial, and the instrument you would use to tell them apart is the one that can't. I think that is one problem rather than four. Working out where that holds, and where it doesn't, is what I do.