Incidents
Blameless postmortems: a template and guide for engineering leads
After an incident, the team either learns something about its system or learns to hide mistakes. A blameless postmortem is how an engineering lead makes sure it is the first. This guide covers when to hold one, how to run it, and a template you can copy.
What "blameless" means
Google's Site Reliability Engineering book describes a blameless postmortem as one that focuses on the contributing causes of an incident without indicting any individual or team. The reasoning is practical: people who fear blame hide information, and hidden information is what lets the same failure happen again.
Blameless does not mean nobody is accountable. It means the investigation asks why the system allowed a reasonable person to make that mistake, and what would stop the next person making it. Accountability shows up as owning the follow-up actions.
When to hold a postmortem
Agree on the criteria before incidents happen, so nobody has to argue for one afterward. The Google SRE book lists common triggers:
- user-visible downtime or degradation beyond a set threshold;
- data loss of any kind;
- on-call engineer intervention, such as a release rollback or rerouting traffic;
- a resolution time above a set threshold;
- a monitoring failure, which usually means the incident was discovered manually.
It also notes that any stakeholder may request a postmortem. Consider adding team-specific triggers: a security event, a near miss that could have been serious, or an incident caused by an AI agent's action.
Running the process
- Assign an owner within a day of resolution. Often someone involved, but not necessarily the person who made the triggering change.
- Build the timeline from chat logs, alerts, deployment records and dashboards while memory is fresh.
- Draft the document using the template below, before the meeting.
- Hold the review meeting. Open by stating the ground rule: the aim is to improve the system, not to evaluate people.
- Agree action items, each with an owner and a due date, tracked in the normal work tracker.
- Share the result beyond the team. Other teams often have the same weakness.
- Follow up. Review open postmortem actions regularly. Incomplete actions are one of the most common ways incidents repeat.
Postmortem template
Title: short description and date.
Status: draft, in review, complete.
Owner and participants.
Summary: two or three sentences on what happened and the impact.
Impact: who was affected, how, and for how long; data affected, if any.
Detection: how the team found out, and how long it took.
Response: what was done to mitigate and resolve.
Timeline: timestamps, in one time zone, from first signal to resolution.
Contributing factors: technical, process and organizational conditions that allowed the
incident.
What went well: practices worth keeping.
Where we got lucky: things that could have made it worse.
Action items: action, type (prevent, detect, mitigate), owner, due date, tracker link.
Lessons: what other teams should know.
Finding contributing factors
Incidents rarely have a single root cause. Asking "why" repeatedly is a useful start, but stop when you reach a condition the team can change: a missing test, an alert that did not fire, a confusing runbook, a deployment that could not be rolled back quickly. Look at several layers:
| Layer | Example questions |
|---|---|
| Technical | What failed? Why did safeguards not catch it? |
| Detection | Why did it take as long as it did to notice? |
| Process | Did review, testing or release steps work as designed? |
| Knowledge | Did responders have the information and access they needed? |
| Organizational | Did deadlines, staffing or ownership gaps contribute? |
Incidents involving AI-generated changes or agents
As more changes are produced with AI tools, some incidents will trace back to them. Keep the review blameless and systemic:
- Ask how the change passed review, not who used which tool. See reviewing AI-generated code.
- If an agent took an action, ask what permissions allowed it and what checkpoint was missing. See AI agents in the software lifecycle.
- Check whether generated tests covered the failing behavior, or only appeared to.
- Record the tool involved as a contributing factor where relevant, so patterns become visible across incidents.
Good action items
- Specific: "Add an alert on queue depth above the agreed limit," not "improve monitoring."
- Owned: one person, even if several help.
- Sized: small enough to finish; large fixes become projects with their own plan.
- Balanced: a mix of prevention, faster detection and faster recovery.
Postmortem actions compete with feature work. Treat them as part of the team's reserved capacity; see prioritizing technical debt.
Keeping a postmortem library
Individual postmortems teach one team once. A searchable library teaches the organization repeatedly:
- Store every postmortem in one place with consistent titles and tags: service, type of failure, contributing factors.
- Review the library periodically for repeated patterns, such as the same kind of deployment failure across teams.
- Use past incidents in onboarding and in practice exercises for on-call engineers.
- Track the share of action items completed, and raise it in planning when it falls behind.
The lead's role
- Model the tone: thank people for surfacing problems, including their own mistakes.
- Stop blaming language in the meeting early and calmly.
- Make sure action items get time in planning.
- Track incident trends alongside delivery measures such as change fail rate and failed deployment recovery time; see DORA, SPACE and DX Core 4.
Common questions
How soon after an incident should the review happen?
Soon enough that memories and logs are fresh, often within a week, and after the people involved have had time to rest.
Should the person who caused the incident attend?
Yes, if they are willing. They usually have the most useful context. The blameless ground rule protects them.
What if someone acted carelessly?
The postmortem still focuses on the system. Individual performance concerns are handled separately, privately, through the normal management process.
Should postmortems be shared with customers?
Many companies publish a customer-facing summary for significant incidents. It is a different document, reviewed for accuracy and confidentiality, and written for that audience.
Last reviewed 2026-09-17