Phil Pierce

Assessment

How defensible are your
automated decisions?

Fifteen questions about one system — pick the one that makes the decisions you would least like to explain. Answer for that system as it runs today, not as it is meant to. Four minutes. Nothing is recorded unless you ask for the result.

0 of 15
Question 1 · L1

Does every automated decision produce a durable record, rather than a log line on a rotation schedule?DDR-1 §5.1

Question 2 · L1

For any given past decision, can you retrieve the specific evidence items it relied on?DDR-1 §5.4

Question 3 · L1

Is recorded evidence immutable — never updated in place when the source changes?DDR-1 §5.4

Question 4 · L1

When a decision is corrected, does the original record remain retrievable?DDR-1 §5.10

Question 5 · L2

Does each record carry the time the conclusion was issued, distinct from when it was written to storage?DDR-1 §5.1

Question 6 · L2

Does evidence carry the time your system recorded it?DDR-1 §5.3

Question 7 · L2

Does evidence separately carry the time it was true in the world?DDR-1 §5.3

Question 8 · L2

Is each evidence item versioned or content-addressed, so you know which version the decision used?DDR-1 §5.4

Question 9 · L2

Can you query the system as of a past moment and get what it knew then, rather than what it knows now?DDR-1 §5.3

Question 10 · L2

Is the deciding model or rule recorded with a version identifier that changes when its behaviour changes?DDR-1 §5.5

Question 11 · L3

Does the record contain a rationale a reviewer could dispute, rather than only a score and a disposition?DDR-1 §5.5

Question 12 · L3

Does it record the conclusions that were considered and rejected, and why?DDR-1 §5.6

Question 13 · L3

Does it record what would have to be true for the conclusion to change?DDR-1 §5.7

Question 14 · L3

If a confidence value is recorded, has it been validated against measured outcomes for that population?DDR-1 §5.8

Question 15 · L3

Is the accountable actor recorded, along with the delegation under which it decided?DDR-1 §5.9

Scored against DDR-1, an open specification. Yes counts in full, Partially counts half, No and Don't know count zero. A level is met only when every requirement in it is met.

Contact

If you are working on this problem, I would like to hear about it.

These are the questions I spend my time on. If you are building or buying a system that makes consequential calls and cannot explain them, write to me.

phil@philpierce.com