Software issue knocked out 911 service for several provinces last month, Bell report says
None of the missed calls during outage led to 'critical public safety incident,' provider tells CRTC
asserted
provider → knock → CRTC
911 outage that lasted nearly an hour and a half in several Canadian provinces last month was caused by a previously undetected bug in Bell's software, the company said in a report on Friday.
asserted
company → last → Friday
The report said a "latent software issue" triggered the outage across Manitoba, Ontario, Quebec, New Brunswick, Nova Scotia and Prince Edward Island at 3:13 a.m. ET on Sept. 17.
asserted
issue → say → Sept.
"At the outset, we note that this incident was not a cyber attack and did not result from any change activity, planned or unplanned," Philippe Gauvin, assistant general counsel for Bell, wrote in the report to the Canadian Radio-television and Telecommunications Commission (CRTC).
asserted
Gauvin → note → Commission
The software bug affected components of Bell's next generation 911 system, known as NG9-1-1, Gauvin said.
asserted
Gauvin → affect → NG9
The service is designed to allow the public to send more detailed information about emergencies to dispatchers to help police, fire and paramedics better understand an emergency before arriving at the scene.
asserted
police → design → scene
Bell said it identified the root cause and restored service at 4:40 a.m., after 87 minutes.
asserted
it → say → minutes
The company implemented "a permanent solution to the problem" later that same morning.
asserted
company → implement → problem
On the day of the outage, Bell confirmed the issue disrupted service across six provinces.
asserted
issue → confirm → provinces
Its report to the CRTC on Friday said the "vast majority" of calls during the disruption were successfully managed by backup systems, but "legacy" voice calls routed through older systems were missed if they were destined for a call centre already using the next generation technology.
The report said none of the missed calls led to a "critical public safety incident."
asserted
none → say → incident
In a statement to CBC News, Bell declined to say how many calls were missed.
asserted
calls → decline → News
"Bell takes its responsibility as a 9-1-1 carrier seriously and understands the gravity of any interruption to that service.
asserted
Bell → take → service
We have taken steps to ensure the issue experienced on September 17 does not recur," read an email, which noted the outage as the first of its kind since the system launched in early 2022.
asserted
system → take → 2022
"We are committed to the reliability, resilience, and continuous improvement of emergency calling services so Canadians can call for help when they need it.
asserted
they → commit → it
"
Early alarms didn't work, report says
asserted
report → work → ?
Friday's report also said two alarms at the beginning of the outage, one at 3:13 and one at 3:19 a.m., did not trigger a "war room" crisis response as they should have.
asserted
they → say → response
Engineers and other experts weren't notified of the issue until a separate alarm system flagged an unusual drop in the number of incoming calls 24 minutes later.
asserted
system → notify → calls
The response team convened at 3:48 a.m. and fixed the issue at 4:40 a.m., the letter said.
asserted
letter → convene → a.m.
Bell said the company is still investigating why the first two alarms did not work and has since installed "independent monitoring mechanisms as a further failsafe."
asserted
alarms → say → failsafe
The company said it is also reviewing the system "end to end," debriefing call centre staff and sharing lessons with other providers.
asserted
it → say → providers
In the report released publicly, Bell redacted specific information around the flaw that caused the outage and other details about its wider system.
asserted
that → release → system
The company said revealing those details could create the risk for hackers to find and exploit "weaknesses in the 911 infrastructure.
uncertain
hackers → say → infrastructure
"
"Disclosure of such information could result in direct harm to Bell Canada and the 9-1-1 network serving the public and, accordingly, to the public interest," wrote Gauvin.
uncertain
Gauvin → result → interest
Across Canada, 911 service is connected to regional legacy telephone providers.
asserted
service → connect → providers
Bell provides that service in Manitoba, Ontario and Quebec through to the Atlantic provinces.
asserted
Bell → provide → provinces
B.C. and Alberta are serviced by Telus, while Saskatchewan has SaskTel.
asserted
Saskatchewan → service → SaskTel
Holly Barkwell, the Canadian region director for the National Emergency Number Association (NENA), said last month that the nation's 911 system "across Canada is extremely resilient," but noted no network can be perfect.
"There are always issues with technology," Barkwell told CBC News at the time.
asserted
Barkwell → say → time
The CRTC has said all legacy 911 systems in Canada need to be decommissioned by March 31, a deadline the agency extended by two years in early 2025.
asserted
agency → say → 2025
The Aporia
