The site’s been quiet for a few weeks. This one pulls it back with something I’ve been thinking about rather than something I shipped.

Here’s the setup. A council receives a constant stream of planning applications: a supporting-documents folder each, national and local policy sitting behind them, and a human officer whose job is to read all of it and write a decision. Approve with conditions, or refuse with reasons. The volume is relentless, the deadlines are real, and the work is genuinely hard, because a bad decision can be appealed and the council has to defend it.

That’s a shape a lot of public-sector work shares. A high-volume queue, real consequences on each item, and not enough people to give every case the attention it technically needs. So it’s a fair question to ask, out loud and without a client attached: could an AI system take a first pass at this? Not to decide anything, but to draft a recommendation an officer could check?

I spent some time building one to find out, on my own time, with no client or employer involved. This is a hypothetical, so I’ll keep the specifics of any real authority out of it and talk about the choices instead. The choices are the interesting part anyway.

What “good” would even mean here

Before writing any code, it’s worth being honest about what would make a thing like this trustworthy, because it isn’t accuracy on its own.

A council gets challenged on its refusals. So whatever the system produces has to be inspectable: an officer needs to see why it landed where it did, not just what it concluded. Every policy the system leans on has to be a real policy, quoted correctly, because a recommendation propped up by a misremembered or invented rule is worse than no recommendation at all. It has to be cheap and fast enough that running it is never the bottleneck. And it has to fail loudly rather than quietly, because a wrong answer that looks confident is the whole risk.

None of those are model-quality problems. They’re system-design problems. That framing did most of the work.

The shape of it

I’ll put the shape first, because a sketch says more than the paragraph does. I drew the whole thing out as a quick diagram to think it through; the text version below carries the same structure.

parse documents            (plain code)
  → triage the folder       (plain code: drop off-case docs, quarantine
                             anything that looks planted)
  → extract the facts       (MODEL #1, a cheap one)
  → retrieve policy         (plain code: keyword search over an index)
  → assess each issue       (MODEL #2, an agent that queries as it goes)
  → validate every citation (plain code: reject anything not in the corpus)
  → apply the decision rule (plain code)
  → recommendation

Two model stages. Everything else is ordinary, testable code. That split is the actual thesis of the whole thing, so the rest of this post is really just the reasons behind it.

The choices, and why

1. Put the model in exactly two places

The temptation with something like this is to hand the whole folder to a capable model and ask it to reason its way to an answer. It’ll produce something, and it’ll even sound right.

I didn’t want “sounds right.” I wanted to know, for any given output, which parts were a judgement call and which parts were mechanical. So the model only does the two things that genuinely need judgement: reading messy documents into a clean set of facts, and weighing each planning issue on its merits. The reading step gets a cheap model. The weighing step gets a stronger one, and runs as an agent, because it composes its own questions against the policy rather than working from a fixed checklist I decided in advance.

Everything between and around those two steps is code. That means most of the pipeline can be unit-tested, and the parts that can’t be tested are small and clearly labelled.

2. The decision itself is code, not the model’s opinion

This is the choice I’d defend hardest.

The model grades each issue: acceptable, acceptable-with-a-condition, or unacceptable. But the step that turns those grades into “approve” or “refuse” is a plain function, a few lines long: if any issue is unacceptable and can’t be fixed by attaching a condition, it’s a refusal. Otherwise it’s an approval with whatever conditions apply.

Refusals are the decisions that get appealed. I wanted that rule to be something an officer, or a lawyer, could read in ten seconds and check, rather than another thing the model decides invisibly on each run. Pulling the final call out of the model and into code costs you nothing in capability and buys you an enormous amount in defensibility.

3. Trust no citation the model gives you

Language models are fluent about sources in a way that’s genuinely dangerous here. They’ll cite a policy by number, attribute it to a page, and be wrong in a way that reads exactly like being right.

So every citation the assessment step produces gets checked, after the fact, against the actual policy corpus: does this chunk exist, does the page match, does the policy number really belong to it? Anything that fails is stripped before it reaches the output. When I measured this, roughly one in seven proposed citations didn’t survive the check, and a couple of those were references to policy text that simply didn’t exist. The model had invented them. Catching that mechanically, every time, is the difference between a tool an officer can lean on and one they can’t.

4. Structure beats cleverness in retrieval

The policy corpus is too large to hand to the model whole, so it has to be searched. I used a plain keyword index, no embeddings and no vector database, and the interesting result is that the ranking algorithm barely mattered. What mattered was what the index knew about each chunk.

An early version split the policy by page. On a recall test built from the policies the real cases actually cited, it scored three out of nine. The reason was almost funny: one policy had its number and title on one page and its actual substance on the next, so the paragraph that discussed the relevant design question carried no policy number at all and was unreachable by any sensible query. Attaching each policy’s number and title to every one of its chunks took that from three out of nine to seven, all seven ranked first. Same algorithm, same queries. The only change was giving the index a bit of structure it had been missing. That one change is what made citation possible rather than decorative.

5. Handle the adversarial stuff before the model sees it

This is where a real document folder gets hostile, and it’s worth building for even in a hypothetical.

Two things turned up in testing. One folder contained a document written to look like an internal officer email, addressed to whoever was reviewing the case, telling them to disregard precedent and just recommend approval regardless of the merits. A planted prompt injection, in other words, sitting in the evidence. Separately, a large batch of documents that actually belonged to a completely different, fictional application had been salted across several folders.

The right place to deal with both is before any model runs. A deterministic pass strips the documents that don’t belong to the case, and quarantines anything that reads like an instruction to the reader rather than evidence: not by deleting it, but by flagging it, because an officer should know a document like that was in the file. On the injected case, the eventual recommendation came out as a refusal, the opposite of what the planted note asked for. The limit is worth stating plainly, though. That filter is tuned to phrasing I’ve already seen, and a cleverer injection written to dodge it could get through. I’d want a second, meaning-based check before trusting this on documents I hadn’t read myself.

6. Test the cheap-model assumption instead of believing it

It would have been easy to reach for the most capable model available and never question it. Instead, on the cases that were still coming out wrong, I ran them through a frontier model at roughly five times the cost. Not one flipped. The expensive model reached almost the same reasoning, sentence for sentence in places, and just attached more conditions to the same wrong judgement.

That told me the problem wasn’t capability, it was instruction: I was describing the task badly, not using a model that was too weak. Which meant I could stay on the cheap models with actual evidence behind the choice, rather than a hunch. Getting the instructions right took nine rounds of getting them wrong, mostly in instructive ways, but that’s a post of its own.

How well did it work, and what it cost

On the handful of past cases where the real outcome was known, the recommendations matched every one. On the cases where no answer was available, it produced a believable mix of approvals and refusals rather than rubber-stamping everything, which is the failure mode you’d most worry about. Each case cost pennies to run and finished in a minute or two.

I want to be careful with that “matched every one,” though. It’s a tiny sample, and matching the final decision is a weaker claim than reproducing the officer’s actual reasoning, which it did less well. Some of that gap is unfixable from where I was sitting: the real reasons were distilled from reports I never had, so there was context behind the officer’s wording that no amount of tuning could recover. Five known cases is enough to find a systematic bias. It is nowhere near enough to promise the fix generalises.

What I’d want before trusting it

The honest version of the ending is a list of things that aren’t done. A meaning-based check on top of the pattern-matched injection filter. A way to read the image-only documents the parser currently can’t (some of them, floor plans and a flood assessment, are exactly the kind of thing that changes a decision). Real geospatial data instead of a fixture lookup. And far more than a handful of cases, weighted toward the hard ones, before I’d believe any accuracy number out loud.

But the shape holds up. The thing I keep coming back to is that almost none of what made this trustworthy was the model. It was deciding where the model was allowed to be, and writing plain, checkable code for everything else. The council’s problem is volume, and a model does genuinely help with volume. But the reason you could put one anywhere near a real decision is everything around it that isn’t a model at all.

That’s the same lesson as last time, from a different angle: the interesting work is usually the structure, not the prompt.