When the power cut out at one of Google’s U.S. data center zones on March 29, the company’s promise of always-on cloud reliability met a sharp test. A six-hour disruption at the us-east5-c region in Columbus, Ohio left more than 20 Google Cloud services either degraded or completely offline—highlighting a surprising point of failure: the very systems designed to prevent one.
According to Google’s incident report, the disruption began with a routine enough problem: a loss of utility power. The uninterruptible power supplies (UPSes)—which should bridge the gap until diesel generators activate—experienced a critical battery failure. Not only did the UPSes fail to provide emergency backup power, they also blocked power from reaching the data center racks once generators came online.
Engineers got the first alert at 12:54 p.m. PT and had to manually bypass the UPS systems before restoring power at 2:49 p.m. While most services came back soon after, some required additional manual interventions for a full recovery.
In response, Google has promised to strengthen its power infrastructure, including audits of automatic failover systems and a closer look at its UPS hardware. The company says it will work directly with the battery vendor to prevent similar issues from recurring.
This incident highlights a larger cloud reliability issue. Even world-class redundancy can’t stop hyperscalers from suffering breaches when the right conditions align. For all of us who depend on the cloud—particularly in the enterprise—be clear: maintain a plan B for your plan B.
