Research in Motion is blaming technical problems in its highly secretive communications network for a three-day disruption in service to millions of BlackBerry users around the globe.
The company runs a highly secure network and data centers that are used to encrypt and route traffic to every BlackBerry device worldwide.
Here are some questions and answers about the epic failure:
Q. What happened?
A. RIM said that a switch used to direct messaging traffic failed at a data center in Europe. Its back-up switch also failed, causing a huge backlog of traffic.
Q. What caused the failure?
A. RIM has yet to disclose the cause of the failure, but a company executive said at a press conference on Wednesday that company technicians believe they have identified the cause. IDC analyst Rohit Mehrasaid he suspected that the failure in those switches could have been caused by a software bug.
Q. Was hacking involved?
A. A RIM executive said at a press conference on Wednesday that there is no evidence or hacking or a system breach.
Q. Why did the problem spread from Europe to the Americas?
A. All of the RIM data centers are connected. So eventually traffic got so backed up that it had an impact on messages of customers in the Americas, the RIM executive said.
Q. Will RIM compensate its customers for their inconvenience
A. A RIM executive said at the press conference that the company has yet to make a decision on that matter.
Q. How big is the RIM network?
A. Nobody outside the company knows for sure.
Jefferies & Co analyst Peter Misek estimates that there are more than six major data centers around the world and "little nodes all over the place." On top of that, he said RIM operates the world's largest telecommunications network. "It includes dark fiber. It includes connections to multiple data centers around the world. It includes connections to carrier networks, software at the cell site."
RBC Capital Markets analyst Mike Abramsky said RIM has two centers at its Waterloo, Ontario, headquarters for traffic in the Americas and Asia-Pacific, and another in Britain for traffic in Europe, the Middle East and Africa.
Q. Why does RIM send the traffic through its data centers?
A. That is part of the "special sauce" behind the BlackBerry formula -- all BlackBerry traffic is encrypted through servers controlled by corporate clients or telecommunications carriers, then funneled through RIM data centers that monitor the traffic to make sure it is secure.
But the central handling makes BlackBerry traffic vulnerable to widespread outages -- something that rivals such as Apple and Google don't need to worry about with the iPhone and Android devices.
Q. Is that the only way to secure email on mobile devices?
A. No. When RIM first launched the system in the 1990s, no competitors could often similar security features. Many alternatives have since been launched for the iPhone, Android and other devices that do not require customers to use centralized data centers. Instead they can host the services at their own, internal data centers.
(Reporting by Jim Finkle and Alastair Sharp; Editing by Frank McGurty)