Recovery Runbooks for Small Sites: Writing Steps Someone Else Can Follow at 2 AM

GhostHosts editorial ·

Wikimedia Foundation Servers
Illustrative image. Victorgrigas · CC BY-SA 3.0 · Cropped and resized. Image source

Most small sites do not lack ideas about what to do when something breaks. They lack a written, specific sequence that a second person can follow when the first person is asleep, traveling, or simply unsure. A recovery runbook is that sequence. It turns memory and habit into steps with clear inputs, clear decision points, and clear end states. This article walks through how to write one for a small website, what it should contain, and how to test it so it stays useful. The advice here is general and applies regardless of which hosting plan or tooling you use. Where a question depends on your provider's actual capabilities, the runbook should record the answer you verified, not an assumption.

Start with the failures you can actually name

A runbook written as a catch-all for 'the site is down' will be vague and therefore slow. Begin instead with a short list of specific failure scenarios. Typical ones for a small site include: the homepage returns an error, the login page loads but sessions fail, the contact form submits without delivering, checkout hangs on payment, a deployment breaks the layout, or a certificate is no longer trusted by browsers. Each scenario deserves its own section, because the first diagnostic step for a form failure is different from the first step for a certificate failure.

For each scenario, write three things: the symptom as a visitor or teammate would describe it, the most likely causes ranked by how often they occur in your setup, and the first check that distinguishes between them. Keep the first check cheap. Loading the page from a different network, checking whether the problem affects one browser or all of them, and reviewing the most recent change are all low-cost checks that often narrow the field before anyone touches configuration.

Assign ownership and escalation before you need them

A runbook without named responsibilities tends to produce two failure patterns: everyone assumes someone else is handling it, or three people start working on it at once and interfere with each other. Decide who is the first responder for each scenario, who is the backup, and who has authority to make decisions such as rolling back a deployment or restoring from an older state. Write these names into the runbook along with contact methods that do not depend on the site itself being reachable.

Escalation rules should also be explicit. Specify the point at which a first responder should bring in another person, for example after a set number of minutes without a confirmed cause, or whenever a fix would involve changing account-level settings. Small teams often skip this because the group is trusting and informal, but informality is exactly what makes a stressful incident slow. The runbook should make the handoff boring.

Write steps that include the verification, not just the action

A common weakness in recovery documentation is that it lists actions ('restart the service', 'clear the cache', 'restore the previous version') without saying how to confirm each one worked. Every step should have an expected observable result. After clearing a cache, the expected result might be that a specific page now shows the current content from a clean request. After a rollback, the expected result is that the version identifier shown in the footer or a status endpoint matches the earlier release.

Include the exact location of things a stressed person would otherwise have to search for: where the deployment history lives, where the backup inventory is recorded, where DNS records are managed, and where certificate expiry dates are tracked. If a step depends on credentials, state where those credentials are stored and who can access them, without writing the secrets themselves into the document. Reviewing this every quarter catches the places where tools, dashboards, or menus have moved.

Separate stabilization from root-cause repair

During an incident the goal is usually to restore service to an acceptable state, not to fully understand the cause. A runbook should make that distinction visible. Stabilization steps might include rolling back the last change, disabling a recently added plugin, or temporarily serving a static notice while the team investigates. Root-cause work belongs in a separate phase that begins once visitors can use the site again.

Writing these as two distinct phases prevents a common mistake: spending forty minutes reading logs while customers cannot check out. It also gives the team a natural checkpoint to decide whether the situation has been contained. Note in the runbook which stabilization actions are reversible and which are not. Deleting data, overwriting files, or changing DNS records can be harder to undo than a configuration toggle, and that difference should shape how cautious the first responder is.

Test the runbook with a realistic drill

A runbook that has never been used is a hypothesis. Schedule a drill in which someone who did not write the document follows it against a staging copy or a low-risk test scenario. Observe where they pause, where they ask a question, and where a step assumes knowledge the document does not provide. Those pauses are the most valuable output of the exercise. Update the runbook immediately afterward while the friction is fresh.

Useful drill questions include: Could the reader find the first check within one minute? Did they know who to contact? Did any step require a tool they lacked access to? Was the expected result clear enough to tell success from failure? Repeat the drill after significant changes to hosting plan, deployment process, or team membership. Over time the runbook becomes a record of what your site actually needs, rather than what someone once imagined it needed.