‘A single point of failure can’t ruin your night again’: Xbox CTO says Monday’s major outage was ‘unacceptable’

The chief technology officer of Xbox has explained what led to the major Xbox outage on Monday, and vowed to investigate it to prevent it happening again.

Xbox acknowledged early on Monday that it was suffering “major outage” status in a number of its services, preventing players from being able to sign in, view their library or launch some games or apps.

The outages continued for a large part of the day, with 13 hours between Xbox’s tweet acknowledging the issue (it had been going on for some time before this) and its tweet stating that it had been resolved.

Now Xbox CTO Scott Van Vliet has taken to X to write a lengthy post explaining explaining what happened, vowing to keep players in the loop in the future.

“Earlier today, many players began experiencing issues with sign in, enumerating their game libraries and playing games,” Vliet wrote on Monday night. “The team has worked hard throughout the day to fix the issue, and we’ve restored service as of 2:30 PM PDT. Nonetheless, this is an unacceptable situation, and we must do better to support you.

“Over the coming weeks, you’ll hear more from me about our services, our platform, and the work we’re doing to make Xbox better. And that starts with some transparency about this incident.”

“Our on-call teams caught the first signals overnight through automated monitoring and declared a major incident,” he explained. “Once we isolated the failing infrastructure, we shifted traffic onto the healthy parts of the system while the team continued to work towards a root cause. Service came back unevenly across regions which is why some of you recovered hours before others.”

“We’ll do better”

Van Vliet added that while the issue had now been resolved, he’s more interested in why it happened in the first place and how Xbox’s tech can be restructured to ensure a single issue doesn’t lead to widespread outages in the future.

“We’re running a full post-incident review,” he wrote. “I care less about the one-line root cause and more about the real questions – why a failure in one service was able to take down this much, why recovery took as long as it did, and what we change so a single point of failure can’t ruin your night again.

“That means hardening the dependencies underneath sign in and game launch, improving how we detect and contain this class of failure, and being faster and clearer with you when something breaks. Thank you for your patience, and for the reports that genuinely helped us narrow this down. We’ll do better.”

Xbox’s outage followed similar issues on the PlayStation Network last week, which went down just hours after the open beta for Marvel Tokon: Fighting Souls went live.

The outage led to the service being unavailable for several hours, leaving players unable to access the PlayStation Network on a PS5 console or via a game that required a PSN sign-in.