AI Systems
Someone still has to answer for it
Generation gets cheaper every year. Being the party who answers when a regulator asks who approved the data model does not, and that gap is widening.
An agent can write a working application. It cannot sign a business associate agreement, choose where data lives, defend an access log to a reviewer, or be the party that answers when a regulator asks who approved the data model.
That gap is not closing. It is widening, and it is worth being precise about why.
The asymmetry
Every year, models get better at generating code. The cost of producing a working system falls, and it will keep falling.
Nothing in that trend makes anyone better at being answerable. Accountability is not a capability that scales with compute — it is a relationship between a decision and a person who made it. As the code gets cheaper, the person who understood it before it shipped becomes the scarce part, not the abundant one.
Which means the durable position in this market is not being fast. Speed is being commoditised in public, on a curve everyone can see. The durable position is being the party who can still explain the system a year later, when the person who prompted it has moved on and the question is why a particular boundary was drawn where it was.
What accountability actually decomposes into
Said plainly it is a platitude — every firm claims to take responsibility. It only means something when it names what is owned.
The data model. Someone decided what an entity is, what must always be true about it, and what happens to a record when the thing it describes stops existing. That decision is inherited by everything built afterwards, which is why it is the most expensive one to get wrong and the one least visible in a demo.
The authorisation boundary. Someone can state, without opening an editor, which role can perform which action against which record. Not because they memorised it, but because it was designed as a matrix rather than accumulated as checks.
The failure modes. Someone decided what is retried, what is idempotent, what may be lost, and what must page a human. Systems do not fail in ways nobody chose; they fail in ways nobody chose deliberately.
If each of those has a name attached, the claim is real. If they emerged from whatever the tooling produced, there is nothing to be accountable for — only something to be surprised by.
Why it cannot be delegated downward
The parts that need a person are the parts with a counterparty.
A model cannot be the second party to an agreement. It cannot decide that patient data stays in one jurisdiction because a contract says so. It cannot look at an access log with a reviewer and explain why a particular query pattern is expected. It cannot hold the lawful basis for processing, because that is a business decision before it is a technical one.
These are not gaps in current models that a better model closes. They are structural: they require somebody who can be asked, and who bears something if the answer is wrong.
Why it cannot be faked by scale
The honest version of this promise has a size limit.
A firm of forty cannot truthfully say a senior person reads every line touching money or regulated data. The arithmetic does not work, so what is claimed as a review process is actually a sampling process, described optimistically.
A small, senior team can make that claim because there are few of them and each carries more. That is not a marketing position, it is a constraint — and it is the reason the promise is worth anything. Any claim of this kind should be read against the headcount making it.
Where the line sits
Accountability here means engineering ownership, and it is worth being exact about the edge, because a vendor who is vague about it is telling you something.
What a build can carry: the data model, the authorisation boundary, the failure modes, the audit trail that demonstrates what happened, the evidence a reviewer will ask for, and keeping all of it current as the system changes.
What stays with the organisation running the system: the attestation itself, organisational policy, staff training and access approvals, the lawful basis decision, and incident response ownership once the process exists.
No vendor holds an attestation on your behalf. Any vendor implying otherwise is describing something that does not exist, and that single claim is a reliable way to sort the market.
The practical version
In a system where an incorrect result costs money, a licence or a patient, the useful question to ask a supplier is not how fast they build or which tools they use.
It is: when this breaks in eighteen months, who reads it, and are they still here?
That question is what our accountable architect engagement exists to answer — ongoing ownership of the security boundary, the data model and the failure modes, for a team now shipping faster than it can review.
More
Related reading.
AI Systems
Nobody has read your AI-generated codebase
It runs, it demos, and no human has traced a request through it end to end. Here is what that costs, and the four things worth checking first.
AI Systems
Engineer the boundary, generate the implementation
The useful split is not humans versus agents. It is that the decisions live upstream and the code lives downstream, and only one of those can be handed over.
Recognise the problem?
If this describes where you are, the first conversation is usually short and tells you whether we are useful.