Save articles for later

Add articles to your saved list and come back to them any time.

A major outage on Wednesday has impacted millions of Optus customers nationally including businesses, Melbourne’s rail network and hospitals. Here’s what we know.

What caused the Optus outage?

Optus is yet to confirm the issue behind the outage, but the fact it started about 4am points to a likely issue with a software or firmware update, or an incorrect message sent from either inside or outside the Optus network hat provided dodgy traffic routing instructions, according to network engineers. The vast majority of network updates occur overnight, between 2am and 4am, while most of us are asleep.

In 2012, Dodo took the blame for an outage affecting many Australian internet connections, saying it was caused by a hardware fault on a router that triggered crippling flow on effects at Telstra. That outage only lasted 45 minutes, while the Optus outage has so far lasted at least nine hours.

Matt Tett, the managing director of Enex TestLab, said the issue appeared to be caused by a so-called “BGP prefix flood”.

Essentially it means that one of Optus’ routers has likely been fed incorrect routing information in an update, leading to total network gridlock. This could have been caused by either Optus or an external party. Optus has been contacted to confirm whether this is indeed the case.

Network operators pointed towards this possibly being the case after Optus sent a message to them stating that the suspected root cause of the issue lied with “route reflectors, which are currently handling an excessive number of routes, leading to session shutdown and a complete traffic halt”.

The message to Optus partners early this morning said: “Our on-site technician is actively prioritising establishing a console connection [a physical cable connection]. Rest assured that said technician is also being provided additional technical support remotely.”

Tett said Border Gateway Protocol is where network owners and operators’ routers share information.

“Take for example the fence you share with your neighbour. I have a note for you to pass to the neighbour next door that says ‘I love them’. Instead of putting the one address to next door, I accidentally put ‘everyone’, then you go off and try to deliver that note that I love them to EVERYONE. Which results in you handing off that note to everyone off to your BGP routes.”

An Optus source, who did not wish to be naame because they were not authorised to speak publicly, said a BGP prefix flood from a peer was likely causing the issues on its core network.

The dodgy instruction could have been sent from an internet exchange (a physical location similar to a data centre, where multiple internet providers and network operators interconnect their networks) or directly from an internet provider itself or a content provider.

Is a cyber attack to blame?

It’s too early to rule out a malicious attack, though Optus chief executive Kelly Bayer Rosmarin says there are no indications of the outage being due to a hack or cyber attack, despite her telco suffering one of Australia’s most significant data breaches late last year.

“There’s no indication that there’s anything to do with cyber at this stage,” the CEO told ABC Sydney.

How long will it take to fix?

Optus is yet to provide a timeline for a fix but it will likely take hours, according to Tett.

“The problem with routers, particularly if it is configuration [and] not an attack, is that one major change has to propagate throughout the network and then a fix also then needs to propagate,” he said.

“[The fix] can take hours, particularly if the network is slammed through mis-routing.”

Why is the Optus network outage connected to public transport and hospitals?

Optus’ network infrastructure runs far deeper than just 4G and 5G mobile phone towers. Fibre networks like that from Optus are the backbone for all telecommunications services, including 5G and 4G, as well as eftpos, public transport infrastructure, and hospitals which were all affected on Wednesday.

What’s the government saying?

Communications Minister Michelle Rowland has called on Optus to “step up” its public communications to customers as people are “hungry for information”.

“I think Optus needs to make sure they step up and communicate with people because, as I understand it, this started in the early hours of this morning, ” she said. “We’re now at 11 o’clock and for a lot of people who are trying to get on with their day and their business this is absolutely vital that they get back to normality.”

Will customers be eligible for compensation?

If you have been disadvantaged or lost money due to a phone or internet outage, you might be able to claim compensation, according to ACCAN, the peak advocacy group for Australian communications consumers.

“Compensation should make up for your loss,” ACCAN says. “For example, if your internet is out for one week you could ask for your money back for that week. You may be able to claim for costs incurred, like getting your internet fixed or using extra mobile data.

“Work out how much money you or your business has lost because of the outage, including any costs for an interim service. Keep documents such as bills and receipts as evidence, record when the outages happened, and how long they lasted.

“Contact your service provider to explain the problem, and to ask for compensation, and give your service provider the evidence you have collected.”

The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.

Most Viewed in Technology

From our partners

Source: Read Full Article