Decision Operations, part two: when the AI drafts the decision

Page

20261013

Last December I wrote about how we built ethical AI at the Danish Business Authority, and how that work turned out to be something bigger: a discipline I called Decision Operations. The core of it was simple to state and hard to build. Say what you are trying to achieve, measure whether you are achieving it, keep a record that lets you reconstruct what happened years later, and give people real agency over what the machine does.

That work was done in a world where the model scored a case and a person decided. The model produced a number. The decision was human, the record was built around the decision, and the hard problems were drift, bias loops and evaluation.

That world is changing. Today the model does not only score. It drafts. It proposes the requirement, the architecture change, the code, the next step. The person no longer starts from a blank page; they start from a plausible proposal. This piece is about what Decision Operations has to become when that happens.

The new problem: plausible is not correct

A language model drafts well and judges badly. It produces structure that reads right, with no guarantee that it fits. A person reviewing free text cannot reliably tell a sound proposal from a well-worded wrong one, especially at the volume models now produce.

Teams using AI today get speed and lose the trail. Ask them what was decided last month, on which inputs, against which alternatives, and which model suggested it, and the answer is usually in a chat history nobody will read again.

For a regulated organisation this is not a productivity footnote. It is the question a supervisor will ask: when an AI-assisted decision was made, who made it, what did they know, and what else could they have chosen?

Put a vocabulary between the model and the record

The pattern I have been building around is one sentence: put a controlled vocabulary between the model and the record.

  • The model proposes freely.
  • A rigid vocabulary refuses what does not fit its shape. For architecture that can be the ArchiMate ontology and its validation rules. For terms, a SKOS concept scheme. For evidence, a schema.
  • A person accepts what remains. That acceptance is an explicit, cheap, visible step, and it is the only check on sense.
  • What is accepted is kept as a decision, with its provenance: what was proposed, by which model, what was refused and why, what the inputs were, who accepted it, and what the alternatives were.

Rigidity sounds like bureaucracy. Here it is the opposite. A typed, validated, versioned artefact with provenance can be handed to the next tool or the next agent without a person re-reading it. Rigidity is what makes a model’s output safe to build on.

Three properties that matter

Decisions, not edits. The unit of work is an accepted change with its reasoning and alternatives, never a silent modification. History is never rewritten. A different choice is a branch, and the difference between the choice and its branch is a counterfactual you can compute.

Evidence that keeps itself. Recording, versioning, linking and exporting the decision should cost the person nothing. In the DBA platform the RecordKeeper sat under every event so that nobody had to remember to document anything. The same principle holds now: the bookkeeping belongs in the plumbing, not in people’s discipline.

The person stays the gate. A vocabulary catches shape, not sense. A well-formed wrong requirement passes every rule. The human accept is the only safeguard against that, so it has to stay cheap and informed: show what the change would do, name what was dropped, name the model that drafted it.

Revisit, reflect, redecide

The part of Decision Operations I find most underrated is not recording decisions. It is returning to them.

In the original work, the Control Tower logged every time a domain specialist turned down a model’s effect, and that pattern of adjustments became a signal of drift. The decision trail was not an archive; it was feedback.

When decisions are stored as versions rather than notes, that feedback becomes concrete. The rejected alternative is not a sentence in a document. It can be recomputed as a version of its own, and compared with what was chosen. The question shifts from “what did we decide” to “what would have happened if we had decided otherwise, and did we know that at the time”. That is how an organisation learns from its decisions, and it is also the question a careful auditor would like to be able to ask.

A word on robustness

It is worth being exact about one thing. A decision trail makes evidence cheap. It does not make the work correct.

In my own practice, building tooling with AI assistance, the defects that mattered were found by review from a context that had not written the work: a separate reviewer, a separate verifier, reading the change cold. The author, human or AI, could not see them. So there are two disciplines here, and both are needed. The record keeps the evidence. Independent review keeps the quality. Neither substitutes for the other.

What has not changed

Reading back over last December’s piece, the foundations still hold:

  • Goals first. You cannot decide well, or check a decision, without a stated purpose.
  • Synchronised state. Documentation that is not tied to the version in production is fiction. Architecture and decisions belong in version control, next to the thing they describe.
  • Human agency over machine output, with every adjustment logged.
  • Records designed to be queried years later, causally linked rather than flat logs.

What has changed is where the decision starts. It used to start with a person reading a score. Increasingly it starts with a person reading a proposal. Decision Operations has to move upstream with it, to the moment of drafting, and treat the acceptance of a machine’s proposal as a decision in its own right.

Where this leaves us

The organisations that will get value from AI-assisted work are not the ones that generate the most. They are the ones that can show, for any decision that mattered, what was proposed, what was refused, who accepted it, and what the alternatives were, without anyone having had to stop and write it down.

That is still Decision Operations. The model just moved closer to the decision.


Previous article: How to Build Ethical AI: We Did It by Design (And Discovered Something Bigger)