What Did You Know, and When Did You Know It?

Every serious inquiry into a decision that went wrong converges on the same two questions. Not was the decision correct — that gets settled by the outcome, and the outcome is already known by the time anyone is asking. The questions are: what did you know, and when did you know it.

A congressional hearing asks it. A deposition asks it. An FDA inspector asks it. A post-incident review asks it, and so does a board that has just been surprised. The phrasing changes. The structure does not.

Notice what kind of question it is. It is not a question about truth. It is a question about a knowledge state at a moment — a reconstruction of what a particular set of people, or a particular system, believed on a particular Tuesday. And most of the AI systems now being placed in consequential decision paths cannot reconstruct that, because they were never built to.

Two clocks, not one

There are two independent times attached to any fact. There is when the fact was true in the world. And there is when your system came to believe it. These are different clocks, and they run at different speeds.

A credential was compromised on the third. You learned it on the eleventh. A supplier's certification lapsed in March. The registry updated in June. A patient's lab value was abnormal on admission. It was flagged four days later. In every case the world moved first and knowledge followed, and the gap between them is where accountability lives.

Database people have understood this for forty years. It is called bitemporal modeling, and it exists precisely because "when was it true" and "when did we know" are questions that must be answerable separately. The discipline is old, well-specified, and largely absent from the AI systems being deployed today.

Most of these systems keep one clock, or none. They hold a current view of the world. New evidence overwrites old evidence. Retracted evidence is deleted. The state of belief last Tuesday is not stored anywhere, because nothing in the design ever needed it — right up until someone asked.

The hindsight trap

Here is why this becomes a serious problem rather than an academic one.

With today's evidence in hand, every past decision looks obvious. The signal that mattered is right there, clearly visible, sitting among the noise that you now know was noise. Anyone reviewing the decision after the fact is reasoning from a knowledge state that did not exist when the decision was made.

Investigators know this about themselves. The protection they look for is a contemporaneous record — evidence that what you knew then was genuinely different from what is known now.

An AI system that cannot produce that record hands the reviewer nothing but the outcome. And an outcome, standing alone, is indistinguishable from luck or negligence. You cannot tell a well-reasoned decision that went badly from a careless one, and neither can anyone else, which means you will be judged on the result. That is a very bad position to occupy, and it is entirely avoidable.

Three ways this breaks in practice

Retraction. A source you relied on is later determined to be wrong. The reasonable-sounding response is to remove the bad data. Do that, and every decision that rested on it becomes unexplainable — the reasoning now points at evidence that is no longer there. The record has been quietly falsified in the name of hygiene.

Late arrival. A fact that was true on Tuesday reaches your system on Friday. A system with one clock files it under Friday. Now the Tuesday decision, reviewed later, appears to have ignored something that was sitting right in front of it. It wasn't. But nothing in the record says so.

Revision. An upstream provider updates a score, a rating, a classification. You acted on version three. Version seven is what is in the system now. Which one appears in your audit trail, and can you prove it?

Each of these is mundane. Each of them, in aggregate, is why organizations discover during an investigation that their AI decisions cannot be reconstructed at all.

What defensibility actually requires

Four properties. They are not exotic, but they have to be designed in, and they are close to impossible to retrofit onto a system that has been overwriting state for two years.

• Evidence is append-only. Nothing is overwritten and nothing is deleted. A retraction is a new assertion that supersedes an old one, and both remain in the record.
• Every belief carries both timestamps. When it became true, and when it became known. Separately. Always.
• The system is queryable at a past coordinate. You can ask what it believed as of a given moment and get the answer it would have given then, not the answer it would give now.
• Every conclusion carries its justification. Not a score. The specific evidence it rested on, at the versions that were in force at the time.

Together these turn "what did you know, and when did you know it" from a crisis into a query.

This is an architecture decision, not a compliance feature

The failure I see most often is treating this as something to be added later, once the model is working and the pipeline is stable and there is time. There is never time, and by then the shape of the system has hardened around an assumption of a single mutable present.

Nobody is going to ask you which model you used. They are going to ask the two questions. It is worth deciding, before you deploy, whether your system can answer them — because the moment someone asks is a very expensive time to find out.