What a reliability assessment actually looks at

Every engagement starts the same way: two weeks inside your systems, and a scored report at the end. Teams often expect an infrastructure audit. It is not one. Infrastructure is the easiest part of a platform to inspect and the least likely thing to be the reason you are paging at 3 a.m.

Here is the actual shape of it.

We start with the last ten incidents#

Not the architecture diagram. The diagram describes what someone intended; the incident history describes what the system does.

For each one we want four things: what the customer experienced, when someone first knew, what the fix was, and whether the same failure could happen again tomorrow. That last question is the one that scores. A team with twelve incidents and twelve permanent fixes is in better shape than a team with three incidents and three restarts.

The pattern we find most often is not a fragile system. It is a system whose failures are all detected by customers rather than by monitoring.

Then: what does "up" mean here#

Ask five people in the same company what counts as an outage and you will usually get five answers, none of them written down. Without that definition there is nothing to measure, so every reliability conversation becomes a matter of opinion and the loudest person wins.

An SLO is not a compliance artefact. It is the sentence that ends the argument. Checkout completes in under 800 ms for 99.9% of requests, measured at the edge, over 28 days. Now a change that costs 40 ms has a price, and a quarter spent on reliability work has a finish line.

Most platforms we assess have no SLO. A few have thirty, which is the same as none.

How long from commit to production#

Lead time is the single most predictive number we collect, and it is predictive of things that sound unrelated to delivery speed.

Long lead times force batching. Batching makes every release large, and large releases are the ones that break. Breakage during a large release is hard to attribute, so rollbacks become rare and forward-fixes become the norm, which extends the outage. The team responds by adding more pre-release process, which lengthens lead time further.

We have never seen a platform with a nine-day lead time and a healthy incident profile. The two are the same problem seen from different ends.

Can you deploy on a Friday#

This is a proxy question and everyone knows it, which is why the answer is honest.

What we are really asking is whether a rollback is a routine, automated, sub-minute operation, or a decision that requires a meeting. If it is the first, Friday is an ordinary day. If it is the second, the team has correctly concluded that Friday is dangerous, and the ban on Friday deploys is a rational response to a real risk rather than a cultural failing.

Fix the rollback and the ban dissolves on its own.

What happens when the person who knows leaves#

Every platform has at least one component that exactly one engineer understands. That is normal. What matters is whether the organisation knows which components those are.

We map them explicitly: for each critical service, who can debug it under pressure at 3 a.m. If the answer is one name, that is a finding with a number attached, not a note in an appendix.

What does it cost, and what is that per unit#

Cloud spend is an engineering metric. A team that cannot say what a thousand requests cost has no way to tell an efficiency regression from growth, and will find out about both from finance, a quarter late.

The unit varies. Cost per thousand requests, per active tenant, per gigabyte ingested, per thousand inferences. Which one it is matters less than having one and watching it.

Can you restore#

Backups that have never been restored are not backups. They are a line item.

We ask for the last restore drill. If there has not been one, we run it during the assessment, against real data, timed. That number goes in the report, and it is regularly the most surprising line in it.

What the report is#

A scored roadmap, prioritised by risk against effort, that your team could execute without us. Every finding has an owner, a rough size, and the sentence explaining why it matters that a non-engineer can read.

Roughly a third of the findings, in the median engagement, cost nothing to fix. They are defaults nobody revisited, alerts that fire into a channel nobody reads, a retry policy that turns a slow dependency into an outage.

The point of two weeks is not to find everything. It is to find the order.

We do the assessment as a fixed-price piece of work, and it stands alone. Plenty of teams take the report and run it themselves. That is a good outcome.