Teaching an agent what it doesn't know

Here's a hypothetical, made up to show the moment. An agent drafts release notes for a new sync feature. The draft is clear, well organized, and accurate about what changed. Near the end it says, "Sync is now twice as fast for large workspaces." It reads like the kind of thing release notes say. Nobody on the team measured that. Nobody knows where it came from. It's one sentence in an otherwise good draft, and it's the sentence that would have been quoted back at you.

The agent wasn't lying, exactly. It was asked to write release notes, it knew release notes often mention speed, and nothing told it that speed was off limits. So it filled the gap with the most plausible thing.

This is an opinion piece about building agents that write outward-facing copy, and here's my claim. I think an agent like that will fill gaps with plausible claims unless it's told, in writing, what it doesn't know. Telling it what's true isn't enough. It needs a list of what it may say, a list of what it must leave unknown, and a list of what it may not say at all, with a person at the end who can send a draft back.

Why "be accurate" doesn't work

The obvious fix is to put "be accurate" or "don't make things up" in the prompt. I don't think that does much on its own, and the reason is worth spelling out.

From the agent's side, there are two kinds of missing information. There's what it doesn't know, and there's what it hasn't been told. A person writing release notes can tell those apart. They know the team never benchmarked sync, so they don't claim a number. An agent can't easily make that distinction. If nothing in front of it says sync speed is unmeasured, it has no particular reason to think it's unmeasured. "Twice as fast" doesn't feel like an invention. It feels like filling in a detail.

So "be accurate" asks the agent to stay inside a boundary it can't see. It'll try. It just doesn't know where the edge is.

The three lists

What I'd give the agent instead is three short written lists.

The first is allowed claims, each with a source. Not "sync is improved" floating on its own, but "sync now runs in the background, see the changelog entry for this release." The source matters for two reasons. It tells the agent this claim is safe to make, and it tells the reviewer where to check. A claim with no source attached isn't on the list.

The second is unknowns, written out word for word. "Sync speed: not measured. Do not state or imply a speed change." "Number of users: unknown. Do not mention." It feels strange to write down what you don't know, but this list is the one that stops "twice as fast." Without it, the agent treats the absence of a number as a gap to fill. With it, the absence is the instruction.

The third is banned phrasings, including the soft ones. The hard cases are easy to think of: no invented statistics, no named customers you haven't cleared. The soft cases are where drift happens. "Growing fast." "Teams love it." "Trusted by engineers." None of these contain a number, so they slip past a check for numbers, and every one of them is a claim. If you wouldn't be comfortable defending it, put it on the list.

These lists don't need to be long. A useful version for one release might be a dozen lines. The point is that they exist in writing and the agent reads them every time.

Why "unknown" has to be written down

I want to spend a bit more time on the second list, because it's the one people skip.

It's natural to think that if you've listed the allowed claims, everything else is implicitly off limits. For a person, that mostly works. For an agent, I think an empty space reads as permission. If the allowed list says nothing about speed, the agent doesn't conclude "speed is forbidden." It concludes "speed wasn't covered," and then it does what it does with anything uncovered, which is make a reasonable guess.

Writing the unknown down changes that. "Not measured, do not state" is a specific instruction about a specific topic, and the agent can follow it the same way it follows anything else. It also helps with softening. An agent told only to avoid a number might write "noticeably faster" instead. An agent told the speed is unknown and should be left out has nothing to soften.

The reviewer at the end

The lists make the draft better. They don't make it safe to ship without a look. Someone still has to read it.

What changes is what that person is checking. Without the lists, a reviewer has to notice anything that sounds off, which is hard when the draft reads well. With the lists, the review is concrete. Does every claim in the draft appear on the allowed list with its source? Does anything on the unknown list show up, even softened? Does any banned phrasing appear? That's a check a person can do quickly and reliably, and it gives them clear grounds to send a draft back.

The reviewer also maintains the lists. When a draft makes a claim nobody can place, the fix isn't just to cut the sentence. It's to add the topic to the unknown list so the next draft doesn't try it again.

We run our own outbound copy through a written list of allowed claims and unknowns, with a person reviewing before anything ships.

Honesty is a design input

Here's the reframe I'd leave you with. It's tempting to think of accuracy in agent-written copy as a property of the model, something that's either good enough or not, and that you wait to get better.

I think it's closer to a design input. The agent can only respect boundaries it can see. If you want it to leave something unknown, you have to say so, in writing, every time. If you want it to stay away from a phrasing, you have to name the phrasing. And if you want a backstop, you need a person at the end who checks the draft against what you wrote down.

None of that is about the model. It's about what you give it. You write the boundaries down, or you don't get them.

Written by the team building Gander, live review for markdown an agent is still writing.