Technical runbooks: the playbooks our engineers work from
These are condensed versions of procedures our engineers follow. They're written for people who administer systems, so they include the commands and the order that matters. Test them somewhere that isn't production first. Your environment will differ from ours in ways that matter.
/ Runbooks on this page
01
Harden an AlmaLinux VPS before it goes live
02
Put Nginx in front of a web application
03
Roll out Conditional Access without lockouts
04
Offboard a Microsoft 365 user in the first hour
05
Deploy a Next.js app on a VPS with PM2 and Nginx
/ Direct answer
What is a technical runbook?
A technical runbook is a step-by-step procedure for one operational task, written so a competent engineer who didn't write it can carry it out and get the same result. Good runbooks list prerequisites, give the exact steps in order, explain how to check each one worked, and include a rollback. They turn one person's knowledge into the team's.
- One task per runbook
- Verify after every step
- Rollback written in advance
/ 01
How do you harden an AlmaLinux VPS before it goes live?
Plan a supported operating-system baseline, named administrative access, network rules, update handling and recovery. Before changing SSH or firewall settings, confirm console recovery and test a second administrative session. The example below is for review on a fresh AlmaLinux 9 host; adapt it to the actual image and test it before production use.
# 1. Patch, and create a named admin user
dnf -y upgrade
useradd -m -G wheel deploy
passwd deploy # sudo will ask for this
mkdir -p /home/deploy/.ssh && chmod 700 /home/deploy/.ssh
# paste your public key into /home/deploy/.ssh/authorized_keys, then:
chmod 600 /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh
restorecon -RF /home/deploy/.ssh
# STOP: test a second key-based login and sudo access before continuing.
# Confirm working provider-console recovery.
# 2. SSH: keys only, no root
cat > /etc/ssh/sshd_config.d/10-hardening.conf <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
MaxAuthTries 3
EOF
sshd -t
sshd -T | grep -E 'permitrootlogin|passwordauthentication|kbdinteractiveauthentication'
# Review Include and Match rules if the effective values differ.
# Reload only after validation, then test another new session.
sshd -t && systemctl reload sshd
# 3. Inspect active zones before changing rules
firewall-cmd --get-active-zones
firewall-cmd --list-all-zones
# Preserve SSH on the actual management port and restrict sources where possible.
# Add web services to the intended zone; review other services before removing any.
# Test runtime rules and a new SSH session before saving permanent rules.
# 4. SELinux stays on
getenforce # expect: Enforcing
# 5. Automatic security updates
dnf -y install dnf-automatic
# Review /etc/dnf/automatic.conf and the timer schedule.
# Choose download-only or installation after testing and maintenance planning.
# Security-only selection depends on repository advisory metadata.
systemctl cat dnf-automatic.timer
# 6. Brute-force protection
dnf -y install epel-release && dnf -y install fail2ban
# Configure the sshd jail for the actual SSH port and logging backend.
# Validate before enabling, then confirm the jail is reading events.
fail2ban-client -tKeep the original session open and confirm console recovery, a new key-based login and sudo access before disabling other login methods. Review the effective SSH configuration, including Match rules, and the active firewall zone. These excerpts are not a complete hardening baseline or a production test record.
/ 02
How do you put Nginx in front of a web application?
Install Nginx, bind the application to a loopback address and configure the proxy headers for your network design. Check SELinux permissions, issue a TLS certificate and test the configuration before reloading. A successful syntax check does not verify upstream availability or application behaviour, so follow it with an HTTP check.
Check the app itself is listening on 127.0.0.1 and not 0.0.0.0. If it listens on every interface and the firewall is ever loosened, the app becomes reachable without going through Nginx and its protections.
dnf -y install nginx
systemctl enable --now nginx
# Let Nginx connect to the local app port under SELinux
setsebool -P httpd_can_network_connect 1
# /etc/nginx/conf.d/app.conf
server {
listen 80;
server_name app.example.com;
server_tokens off;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
# Direct public proxy; configure trusted upstream proxies separately.
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
nginx -t && systemctl reload nginx
# TLS from Let's Encrypt (certbot comes from EPEL)
dnf -y install certbot python3-certbot-nginx
# First verify DNS, challenge reachability and a backup of the Nginx configuration.
certbot --nginx -d app.example.com
certbot renew --dry-run
systemctl list-timers --all | grep -i certbot
# Confirm the renewal mechanism installed by your package; timer names vary./ 03
How do you roll out Conditional Access without locking anyone out?
Create and test two emergency access accounts with separately secured phishing-resistant credentials. Exclude them from Conditional Access policies that block or restrict sign-in; report-only policies do not need an exclusion. Review report-only results over a representative period, resolve unexpected failures, then pilot enforcement before a wider rollout. Document exceptions, owners and review dates, and test the recovery route.
Emergency access accounts
Two cloud-only accounts on the .onmicrosoft.com domain with Global Administrator, FIDO2 or certificate credentials stored separately, and an alert on any sign-in.
Policies in report-only
Model MFA, legacy-authentication blocking and phishing-resistant MFA for admin roles. Review the results before enforcement, and exclude emergency access accounts from policies that block or restrict sign-in.
Review the results
Investigate failures and user-action results. Distinguish intended blocks from legitimate access that would be disrupted, including policy interactions.
Pilot
Enforce for IT and a volunteer group. Keep the rest in report-only.
Roll out in stages
Team by team, with instructions sent ahead of each stage and someone on hand to help.
Maintain
Review exclusions on a documented schedule and after changes. Investigate missing owners before changing an exclusion; preserve and test the emergency-access route.
Microsoft now requires MFA to sign in to its admin portals whatever your own policies say, so emergency access accounts need an MFA method too. Use a FIDO2 key or certificate, not a phone that could leave with one person.
/ 04
How do you offboard a Microsoft 365 user in the first hour?
At the authorised leaving time, disable sign-in and revoke sessions, then check application access. Preserve records and confirm retention requirements before changing mailbox access, transferring files or removing licences. Device recovery, retirement or wiping needs an authorised decision based on ownership and data requirements. Treat the commands below as partial examples, not a complete offboarding procedure.
| Order | Step | Why it's in this position |
|---|---|---|
| 1 | Disable sign-in, revoke sessions | Blocks new sign-ins; existing tokens and application sessions need separate checks |
| 2 | Remove MFA methods and admin roles | Removes these access methods; check other applications and credentials too |
| 3 | Mailbox to shared, grant manager access | Check mailbox size, archive and retention features before removing its licence; keep the associated account |
| 4 | OneDrive access to the manager | Preserve required files and verify the recipient can access them before account deletion |
| 5 | Wipe or recover devices | Choose an authorised action after checking ownership, retention and recovery needs |
| 6 | Remove licences, log completion | Check each service’s retention and licensing rules before removal; record outstanding tasks |
Connect-MgGraph -Scopes "User.ReadWrite.All","User.RevokeSessions.All"
$upn = "leaver@contoso.com"
Update-MgUser -UserId $upn -AccountEnabled:$false # block new sign-ins
Revoke-MgUserSignInSession -UserId $upn # revoke refresh tokens; verify app sessions separately
# Confirm an approved handover and retention decision first.
# Check size, archive/hold requirements and licences; retain the associated user.
Connect-ExchangeOnline
Set-Mailbox -Identity $upn -Type Shared
Add-MailboxPermission -Identity $upn -User "manager@contoso.com" -AccessRights FullAccess/ 05
How do you deploy a Next.js app on a VPS with PM2 and Nginx?
Prepare the application in a release directory as the deployment user, provide its production configuration, install dependencies and build. Run it on a loopback address behind Nginx. For later releases, build and validate separately, then switch the application process during a planned deployment. Keep the previous release and its configuration available for rollback.
# First have an administrator prepare the directory with the intended ownership.
git clone git@github.com:your-org/your-app.git /var/www/app
cd /var/www/app
# .env.production goes here BEFORE building: NEXT_PUBLIC_ values are baked in at build time
npm ci
npm run build
# Run on localhost only, and restart on boot
sudo npm install -g pm2
pm2 start npm --name app -- start -- -p 3000 -H 127.0.0.1
pm2 save
pm2 startup systemd # run the command it prints, once
# Later releases: build and test in a separate release directory.
# Switch PM2 to that validated release using your deployment configuration.
# Verify HTTP responses and keep the previous release for rollback.In PM2 fork mode, allow for interruption during restart; the duration depends on the application. Build and validate a separate release directory before switching the process to it. Keep the previous release available for rollback. Building in the active directory can replace files that the running server still needs.
/ 06
How should you write your own runbooks?
One task per runbook, written for someone competent who has never seen your environment. Start with the purpose and prerequisites, number every step, say how to check each one worked, include the rollback, and put a "last tested" date at the top. Treat a runbook nobody has run in a year as a draft.
Frequently Asked Questions
Yes, adapt them to your environment. Test somewhere that isn't production first, and replace the example domains, users and ports with your own.
Both are good choices. AlmaLinux suits teams that want Red Hat compatibility, SELinux by default and long support lifecycles. The hardening principles apply to either; the package and firewall commands differ.
No. Client-specific runbooks stay private because they describe real environments. The ones here are general procedures we're happy to share.
They're still here. The blog covers news and explainers for business readers, and the resources section has buyer guides and checklists. Runbooks are the hands-on engineering layer underneath.
Yes. Server builds and hardening sit under custom VPS deployments. Conditional Access is part of Entra ID identity management, and offboarding is one of the first things we automate in workflow automation projects.
/ Next step
Want this reviewed against your own environment?
Share your users, tools and the problem you are trying to solve. We will tell you plainly whether this service fits, and what we would look at first.