PostMortem AI
Engineering

How postmortem drafting is grounded, mechanically

Written to be checkable against the running code, not just asserted. Short version: /docs.

The problem with a single unsupervised generation call

Ask a language model to write a postmortem from a vague description of an incident and it will happily invent specifics: a plausible root cause, a plausible timeline, a plausible impact number. None of it is checked against anything real. It reads well. It just isn't necessarily true.

Layer one: cited generation

When you draft a postmortem, the model is given your incident's recorded evidence — every entry you logged or that arrived via webhook — each one numbered. The system prompt instructs it to cite the entry number behind every claim it makes, and explicitly to leave a section unsupported rather than infer beyond what the evidence says.

This alone isn't the guarantee. A model can still cite incorrectly, or claim a citation supports something it doesn't.

Layer two: independent verification

After the model responds, code — not another model call, plain deterministic code — checks every citation against the real evidence list before anything is stored: is the cited entry number real (not missing, not out of range)? Is it the right kind of entry for what's being claimed?

If a claim's citation fails either check, it's replaced with a fixed marker — "Not established by the recorded evidence." — for the four required sections, or dropped outright for optional lists like contributing factors. This step can only remove or replace the model's text. It never adds anything.

The publish gate is a database constraint, not a UI convention

Even a perfectly grounded draft doesn't become a permanent record on its own. Publishing requires a named human's approval, enforced by a CHECK constraint on the table itself — a postmortem cannot be marked published in the database unless an approver and a timestamp are both set. If every layer of application code were bypassed entirely, the database still refuses.

What this doesn't claim

We don't claim the drafts are always excellent, or that grounding makes them un-improvable. A draft grounded in incomplete evidence will be an incomplete postmortem — the system says so honestly, rather than filling the gap with something invented. The mechanism's job is narrower than "good writing": every claim that survives either traces to evidence you actually recorded, or is visibly flagged that it doesn't.

Back to PostMortem AI