What does it mean to be on-call as an SRE, and what are the key responsibilities and best practices?

Question

Accepted Answer

Being on-call means you are the designated first responder for production issues during a defined time period. When an alert fires, the on-call engineer receives a page through a tool like PagerDuty or Opsgenie and is responsible for acknowledging it, assessing the situation, and either resolving the issue or escalating to the right people. On-call is a fundamental part of SRE because it connects the team that builds reliability into systems with the real-world consequences of reliability failures.

Typical on-call rotations last one week, with the schedule distributed among team members so no one is on-call more than once every four to six weeks. A healthy rotation has enough people that the burden is shared fairly. Many teams have a primary on-call who takes the first page and a secondary or backup who takes over if the primary does not acknowledge within a set time.

Key responsibilities during on-call include monitoring alert channels and responding to pages within the SLA (typically 5-15 minutes for acknowledgment), performing initial triage to determine severity and impact, following runbooks for known issues, escalating when needed either because the problem is beyond your expertise or because it requires more people, communicating status updates to stakeholders during incidents, and documenting actions taken for the postmortem.

Best practices for effective on-call start before your shift. Review recent changes and deployments, check ongoing issues or known risks, verify your alerting setup by confirming PagerDuty is correctly configured on your phone, and ensure you have VPN access and necessary credentials. During your shift, stay within reach of your laptop and phone, respond to pages promptly even if you initially think it might be a false alarm, and take notes during incidents.

Sustainable on-call practices are equally important. The Google SRE book recommends no more than 25% of an engineer's time should be spent on operational work including on-call. If on-call is too noisy or stressful, that is a signal to invest in reliability improvements. Track metrics like pages per shift, percentage of actionable alerts, and mean time to acknowledge. If someone has a particularly bad on-call week with many pages, compensate with time off or lighter duty.

Handoff between on-call shifts should be a documented process. The outgoing on-call writes a summary of the shift including any ongoing issues, recent alerts, and things to watch. This prevents the incoming engineer from starting blind and missing developing problems.