Correct arithmetic is still a fabrication
The first thing I built for my market-intelligence pipeline was the exam it has to pass every week. On its first run, the exam caught the model summing two real numbers into one that no source ever printed.
An AI tool that reports facts gets one chance with its readers, so the evaluation was built before the tool earned any use: numeric pass lines against three weeks of hand-built reports, checked mechanically. On its first run it caught the model adding 42MW and 30MW into a '72MW' that appeared in no article — correct arithmetic, still a fabrication — and computed figures are now banned outright.
Every week I put together a summary of market activity in one sector — EMEA data centres — by hand: the deals, developments and financings that moved, in a six-column table with sources. It is exactly the kind of job you would hand to an AI pipeline, and I have now built one. But the first thing I built wasn't the pipeline. It was the exam the pipeline has to pass.
The reason is how trust works for a tool like this. A reader who catches one invented number in a report doesn't recalibrate their confidence; they stop using the report. There is no partial credit, and no second first impression. So before the tool was allowed to produce anything that mattered, I took three weeks of my own hand-built reports and turned them into a gold standard, with pass lines written as numbers: zero fabricated companies, zero fabricated figures, zero broken links, and at least nine in ten of the hand-found items reproduced. One command runs the comparison and fails loudly if any line is missed.
On the first run, the exam caught the tool inventing a number.
Two announcements in the same story: one development of 42MW, another of 30MW. The generated report described a 72MW total. The arithmetic is correct. The number is fabricated: it appears in no article, no press release, nowhere. A person checking the report against its sources would search for "72MW" and find nothing.
That is the error class worth designing against, because it survives ordinary review. A wrong number looks wrong; someone eventually notices. A plausibly derived number looks like diligence. The lesson I took from it: fabricated doesn't mean false. It means unsourced, and a reader has no way to tell a sourced figure from a derived one, because both sit in the same column wearing the same font. The fix was blunt. The extraction rules now ban computed figures outright: numbers appear exactly as printed, "42MW + 30MW" and never a total, even when the addition is trivial.
The ban only works because "sourced" is checkable by machine. Every fact in the report must carry a verbatim quote from its article, and software compares the quote to the source character by character — it does not ask the model whether it is confident. A paraphrased quote gets one chance to be made exact; if it still fails, the claim is demoted to a separate watchlist section, never mixed into the main table. A figure the article doesn't state renders as "not disclosed". The checks are deliberately dumb, and that is their value: a dumb check cannot be argued with or worn down by a fluent explanation.
What the exam actually governs is circulation, not construction. Every output the tool produces carries a DRAFT watermark. For four weeks it runs silently alongside my manual process while I compare the two, and the thing being measured is boring on purpose: how many minutes of rework does reviewing the tool's output cost me, and is that number shrinking?
The last piece was written down before the first run: the conditions for retiring the tool. If the exam can't be passed honestly, or the checking burden isn't falling by week four, the tool is switched off and the manual process keeps the job — recorded in advance as a legitimate result, not a failure to be explained away. An evaluation added after you already trust a tool mostly confirms the trust. Built first, it is what the trust stands on. The first useful thing mine did was catch the tool doing correct arithmetic — and that was the moment I started to believe the design.