The page came through at 2:47am on a Tuesday.
Our client was a Series A fintech. Their payment processing latency jumped from 120ms to 8 seconds. Their customer support line started filling up. Their lead engineer belonged to a two-person backend team spread across three time zones.
They paged everyone. Nobody fixed it.
The CTO woke up at 7am. The incident had been running for four and a half hours. Revenue loss: $240K. Engineering time spent debugging: six more hours, because the root cause was a configuration change someone merged into production Friday at 4:59pm.
Startups pay this tax when they treat on-call as an afterthought.
Incident response requires someone who knows your architecture, at any hour, with the authority to act and the processes to limit blast radius. You do not fix it by installing PagerDuty.
We handle incident response for 40+ clients. The ones with fewer 3am pages share a pattern.
What Startups Do Instead of Incident Response
When an outage hits, startups reach for one of three responses. None of them scale.
1. Hire an SRE
Senior SREs at early-stage startups run $150K to $200K per year. That covers one person for 24/7, which means two people minimum for shift coverage. For a pre-seed or seed startup, that’s a meaningful chunk of the engineering budget before you have shipped your product.
You need someone who knows your stack, not necessarily an SRE. The skill sets overlap, but the cost structure is different.
2. Rotate on-call between engineers
Most startups do this. Two junior engineers on your backend team take turns being on-call. They wake up at 3am, fire up Datadog, and start googling error codes.
Neither of them knows the architecture well enough to fix it at 3am. Neither has the authority to change infrastructure without calling the other person, who is also asleep.
On-call rotation without architectural context makes engineers sleep poorly.
3. Install PagerDuty and write a runbook
PagerDuty pages people. It does not fix anything. Runbooks help when someone already understands the system. When the person who built the system left six months ago, a runbook is a breadcrumb trail you follow while revenue burns.
What Incident Response Requires
Incident response has four layers. They compound into each other.
Layer 1: Auto-remediation
Most production incidents repeat. Disk fills up. Memory leaks. A deployment fails halfway through. A health check times out while a dependency restarts.
Scripts handle these. You write the automation and let it run. When automation cannot fix something, it pages someone.
Auto-remediation eliminates the 3am page for the incidents that happen most often. It costs engineering time upfront but pays for itself in avoided nights.
Layer 2: Named specialist with context
This person knows your architecture. They have been through production incidents and know where the failures live.
They have read/write access to everything. They deploy, roll back, scale, and tear down without asking permission. They know which services talk to each other and which dependencies are fragile.
At Kernul, every client gets a named specialist who shows up at 3am knowing your infrastructure, not reading a runbook.
Layer 3: Fast response and escalation
Our target is under-10-minute response across all tiers. The specialist handles the incident first. If it exceeds their scope, a senior engineer who understands the architecture takes over, not an on-call rotation.
Layer 4: Post-incident learning
After the incident, record what happened, why the automation missed it, what runbook entries need updating, and what should become auto-remediation next time.
Without post-incident review, you solve the same incident repeatedly.
Case Study: Payment Processing at Lumina
Lumina was an 18-person logistics company running a real-time routing API on AWS. Their engineering team had three full-stack engineers. No DevOps. No dedicated infrastructure role.
A memory leak in their container orchestration crashed three of five routing instances on a Friday afternoon. The remaining two instances handled double their normal load, which caused cascading timeouts in their dependency on a third-party geocoding API.
The team spent six hours troubleshooting. They rolled back the deployment. They scaled up. 504s lasted another two hours.
Revenue impact: $80K in delayed shipments and refunds. Engineering time spent: 18 person-hours across the three engineers, plus the weekend they spent on-call trying to stabilize.
What we built
We embedded a specialist as their infrastructure support.
Immediate (Week 1):
- CloudWatch alarms for container memory with auto-remediation: restart the container instead of scaling up, since the leak was internal
- Circuit breakers for the geocoding API dependency: queue requests and retry with backoff instead of cascading
- Document the architecture and known fragile dependencies
- Automated log rotation to prevent disk-fill incidents
Short-term (Week 2-3):
- 24-hour monitoring coverage with under-10-minute response
- On-call rotation handled by the named specialist, not the engineering team
- Post-incident review template and first tabletop exercise
Ongoing (Month 2+):
- Auto-remediation coverage grew to 70% of all production incidents
- Zero incidents requiring human response in the following two months
- The engineering team stopped taking pages at 3am
Lumina’s CTO told us later that the biggest productivity gain was the team stopped dreading Fridays. Uptime improvement followed.
Lumina did not need a fourth engineer. They needed someone who knew their stack, could act at any hour, and would push back when their engineers tried to merge a deployment at 4:59pm on a Friday.
When to Outsource
The right answer depends on your stage and your team size.
Pre-seed or Seed, solo or 2-3 engineers
Your auto-remediation should cover the basics: disk space, memory, health checks. For anything beyond that, you need coverage without the hiring overhead.
A named specialist for $1,500 per month costs less than one day of lost productivity from a single unresolved outage.
Seed or Series A, 3-10 engineers
Document runbooks. Set up on-call rotation. Add monitoring and alerting. Do not rotate your core engineers until you have the automation to back it up.
If you do not have a DevOps person on the team, outsource the coverage while you hire. Delaying incident response for hiring costs a real outage.
Series A+, 10+ engineers
You are building toward an internal SRE team. The teams with the fewest 3am pages use a hybrid model: internal engineers handle architecture changes and auto-remediation, external coverage handles the hours they are not on Slack.
Where Startups Go Wrong
Waiting for the first major outage. You debug under pressure with no runbooks, no automation, and no one who knows the system.
Buying monitoring tools without monitoring strategy. Datadog and CloudWatch alert on the same metrics. Without auto-remediation and a named responder, you pay for better visibility into the same problems.
Confusing alerts with incident response. PagerDuty fires. Someone wakes up. Nobody knows how to fix it. Alerts are notifications. Incident response requires context, authority, and the ability to act.
Treating on-call as punishment. The engineers who always get paged burn out and quit. Rotate on-call. Reduce page volume through automation.
Not testing anything. You configure backups and consider it done. You write a runbook and move on. Without testing, incident response does not exist.
The Numbers
Direct costs mislead you. Lost revenue from outages, engineering hours spent debugging instead of shipping, and churn from burned-out engineers make outsourcing cheaper on total cost.
| Approach | ⠀ | Monthly Cost | ⠀ | Coverage | ⠀ | Response Time |
|---|---|---|---|---|---|---|
| On-call rotation (2 engineers) | $0 direct | Unreliable | 30+ minutes avg | |||
| PagerDuty + runbook (internal) | $200-$500 | Internal burnout | 20-60 minutes avg | |||
| Hire SRE + on-call support | $12,500-$17,000 | Full | Varies by person | |||
| Named specialist (Kernul) | $1,500-$7,000 | 24/7 | Under 10 minutes | |||
| Hybrid (internal + outsourced) | $5,000-$10,000 | Full | Under 10 minutes |
⠀
We average under-10-minute response time across all clients. Clients with enough history show 99.9% uptime.
A Practical Checklist
- Auto-remediation for known failures: Disk space, memory leaks, health check timeouts. Target 60%+ of all incidents.
- Monitoring with meaningful alerts: Alert on things that impact users, not things that look bad in a dashboard.
- Named responder with context: Someone who knows your architecture, at any hour. Not a helpdesk queue.
- Clear escalation path: Specialist handles it. Escalates when needed. Do not page the whole team for a disk space issue.
- Documented runbooks: For incidents that cannot be automated. Written by people who have fixed them.
- Post-incident review process: Record what happened, what to automate next time, what runbooks need updating.
- Automated backups with tested restorations: Configuring backups does not create backups. Test them regularly.
Our solopreneur production infrastructure guide covers the foundational setup. The seed-to-scale checklist walks through monitoring and on-call as part of the first 90 days after funding.
Next Steps
Your startup needs someone who knows your infrastructure, can act at any hour, and can fix the problem without paging the whole team.
If your team wakes up to production incidents and you are tired of being the one everyone pages, get in touch. We have helped 40+ clients reduce their incident response time to under 10 minutes. We do not sign long-term contracts. If we are not worth it, you walk. No hard feelings.