Featured
Possible causes of the outages included everything from software update errors, problems in the core network, to cyberattacks, experts said while the problem was ongoing.
This story was updated on Friday, Feb. 23.
Suspicions confirmed.
Widespread cellphone outages in the U.S. Thursday morning primarily affecting AT&T customers were a result a software update, or “the application and execution of an incorrect process used as we were expanding our network,” the company said.
Of course, Northeastern experts at the Institute for the Wireless Internet of Things had their hunches. As the outages were unfolding, they said that the countrywide connectivity issue is “not something you see every day.”
“This happens very rarely,” says Michele Polese, a principal research scientist at the Institute for the Wireless Internet of Things at Northeastern.
At its peak, more than 71,000 AT&T customers were affected just before 8 a.m. EST on Thursday — a number that fell rapidly as the day went on. Outages also reportedly affected T-Mobile and Verizon customers, but to a much lesser extent.
Experts offered a range of potential causes, which included everything from software update errors, problems in network infrastructure, such as the “core network,” or cyberattacks (our experts corrected suggested it was unlikely to be a cyberattack).
“These systems are extremely complex,” Polese says. “There are many aspects of the system that need to work seamlessly together so that the system as a whole behaves as intended.”
Because the outages are so widespread, it was unlikely that the problem was tied to more localized phenomena, such as weather conditions or a particular construction incident affecting critical network infrastructure, says Josep Jornet, a professor of electrical and computer engineering at Northeastern.
“This [was] a distributed problem,” Jornet says. “Put it this way. If suddenly everyone using AT&T in Boston loses access, then we know that this is a problem on the Boston side of the infrastructure — a result of, for example, some construction workers somewhere breaking the core of optical fibers.”
Jornet also noted that a cyberattack, while a very real possibility, is less likely.
Polese and Jornet note that global demand for cellular and wireless connectivity has meant that carriers are constantly expanding and building out their networks to accommodate more users and devices. While companies such as AT&T have ample diagnostic tools used to root out system failures, an occasional snag in the core network can result in outages.
“Cellular networks come with a core network which manages authentication, billing, roaming, paging, mobility, among others. Operators and vendors build additional components on top of this to be able to manage the complexity of the network and operate the system,” Polese says. “If something goes wrong at any point along the chain, it can have cascading consequences in all of the other parts.”
Polese says that when new devices are added to a cellular network, they can trigger what are called misconfigurations, or errors that can then affect devices on the rest of the network.
As the world becomes more connected, Jornet says customers expect faster speeds and more reliable service.
“Wireless connectivity has become a utility,” Jornet says. “Inside your home, you have water, you have electricity, and you have internet access. We expect to be connected everywhere, and we kind of assume that we are connected and if we’re not, we panic.”
By Thursday afternoon, the vast majority of AT&T customers had their service restored.