Optus reveals cause of mass outage

Save articles for later

Add articles to your saved list and come back to them any time.

Optus has revealed a routine software upgrade by a third-party infrastructure provider was behind last Wednesday’s outage that crippled its network and impacted some 10 million customers alongside businesses, hospitals and rail networks.

An Optus spokesperson said that the telco had spent the past six days investigating the incident, which left some customers unable to access triple-zero emergency services.

Optus CEO Kelly Bayer Rosmarin faced criticism over her response to the outage.Credit: Natalie Boog

“We now know what the cause was and have taken steps to ensure it will not happen again,” the spokesperson said in a statement.

“At around 4.05am Wednesday morning, the Optus network received changes to routing information from an international peering network following a routine software upgrade.

“These routing information changes propagated through multiple layers in our network and exceeded preset safety levels on key routers which could not handle these. This resulted in those routers disconnecting from the Optus IP Core network to protect themselves.

“The restoration required a large-scale effort of the team and in some cases required Optus to reconnect or reboot routers physically, requiring the dispatch of people across a number of sites in Australia.

This is why restoration was progressive over the afternoon. Given the widespread impact of the outage, investigations into the issue took longer than we would have liked as we examined several different paths to restoration.”

The spokesperson added that Optus had changed the peering network to avoid the problem happening again, and would continue to work with international vendors and partners to increase the resilience of its network.

The telco pledged to fully cooperate with the reviews being undertaken by the government and the Senate.

It had previously been coy about the root cause of the outage, with CEO Kelly Bayer Rosmarin telling this masthead last week that the failure was “a network event” that “triggered a cascading failure which resulted in the shutdown of services to our customers”.

The under-fire telco is offering free data to disgruntled customers – but some commentators say it needs to do more.Credit: AFR

Wednesday’s outage not only paralysed the nation’s telecommunication networks, but prompted long queues at Telstra and Vodafone retail stores as customers looked to shift providers.

It also affected other providers using the Optus network, including Amaysim, Vaya, Aussie Broadband, Moose Mobile, Coles Mobile, Spintel, Southern Phone, Gomo and Dodo Mobile.

The outage came a year after Optus suffered a massive data breach, in which more than 9 million current and former customers had their records accessed.

As previously reported by The Age and Sydney Morning Herald, Optus is offering aggrieved customers a free data top-up, but the industry watchdog says it is prepared to force the telecommunications company to offer large compensation payments (up to $100,000 for a business that could prove a loss and up to $1500 for individuals with a claim) if it refuses to settle customers’ claims.

“If you can see a customer has clearly been impacted, we’d be encouraging them to really own the complaint and deal with it,” telecommunications industry ombudsman Cynthia Gebert said.

“But if we need to take a strong line with Optus to get the right outcome for their customers, that’s what we will do.”

Optus’ offer was immediately slammed by Greens communications spokesman Sarah Hanson-Young, who said the “PR play” was not enough, and tech analyst Foad Fadaghi, who said “knee-jerk offers” could prompt more customers to ditch the business.

Embattled CEO Bayer Rosmarin is due to front a Senate inquiry into the 16-hour outage, while also answering to a separate government inquiry announced by Communications Minister Michelle Rowland.

The Senate inquiry kicks off this Friday.

The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.

Most Viewed in Technology

From our partners

Source: Read Full Article