WASHINGTON (Reuters) - U.S. intelligence agencies will forever be scarred by their failure to connect the dots and detect the September 11 plot, but a decade later efforts to break down barriers to information-sharing are taking root.
Changing a culture of "need-to-know" to "need-to-share" does not come easily in spy circles. Some officials say they worry, a decade later, about a future attack in which it turns out that U.S. spy agencies had clues in their vast vaults of data but did not put them together, or even know they existed.
Yet significant changes, both big and small, have broken down barriers between agencies, smoothed information-sharing and improved coordination, U.S. intelligence experts say.
From issuing a blue badge to everyone working in the sprawling intelligence community to symbolize a common identity, to larger moves of mixing employees from different agencies, the goal is singular -- to prevent another attack.
"We're much further ahead," David Shedd, Defense Intelligence Agency deputy director, said of the ability to connect the dots compared with 10 years ago. Still, signs of a plot to attack the United States could be missed again.
"My worst fear, and I suspect probably one that would come true, is that in any future would-be or actual attack, God forbid, we will be able to find the dots again somewhere because of simply how much data is collected," Shedd said.
The political response to the failure to stop the attack was the 2002 creation of the Department of Homeland Security, pulling together 22 agencies to form the third largest U.S. Cabinet department behind the Pentagon and Veterans Affairs.
That was followed by the creation in late 2004 of the Director of National Intelligence to oversee all the spy agencies, as recommended by the bipartisan 9/11 commission.
Previously, the CIA director held a dual role of also overseeing the multitude of intelligence agencies. But in the aftermath of the 2001 attacks, policymakers decided that was too big of a job for one person to do effectively.
Critics argued then and now that the reforms were the government's usual response to crises -- create more bureaucracy. But others see much-needed change.
"It has been a tremendous improvement," said Lee Hamilton, who was the 9/11 commission vice chair. "It's not seamless, there are problems, and we've still got a ways to go."
The 2001 attacks involving airliners hijacked by al Qaeda operatives killed nearly 3,000 people in New York, Pennsylvania and the Pentagon. Various U.S. intelligence and law enforcement agencies had come across bits of information suggesting an impending attack but failed to put the pieces together.
The CIA had information about three of the 19 hijackers at least 20 months before the attacks; the National Security Agency had information linking one of the hijackers with al Qaeda leader Osama bin Laden's network; the CIA knew one hijacker had entered the United States but did not tell the FBI; and an FBI agent warned of suspicious Middle Eastern men taking flying lessons.
Have the reforms made America safer? Officials say yes, and point to the U.S. operation that killed bin Laden in Pakistan in May that demanded coordination among intelligence agencies and the military. But there is an inevitable caveat: no one can guarantee there will never be another attack on U.S. soil.
On Christmas Day 2009, a Nigerian man linked to an al Qaeda off-shoot tried unsuccessfully to light explosives sewn into his underwear on a flight to Detroit from Amsterdam. It turned out U.S. authorities had pockets of information about him.
President Barack Obama used a familiar September 11 phrase to describe the 2009 incident as "a failure to connect the dots of intelligence that existed across our intelligence community."
Roger Cressey, a former White House National Security Council counterterrorism official, resurrected another September 11 phrase: "It was a failure of imagination."
The intelligence community had not seen al Qaeda in the Arabian Peninsula, a Yemen-based al Qaeda off-shoot, as capable of striking the U.S. homeland. If the "underwear bomber" threat had originated in Pakistan "they would have gone to battle stations immediately," Cressey said.
Some proposed changes in how authorities would respond to another successful attack still are pending. For example, creation of a common communication system for police, firefighters and other emergency personnel remains tangled up in political wrangling in Congress over how to implement it.
"This is a no-brainer," Hamilton said. "The first responders at the scene of a disaster ought to be able to talk with one another. They cannot do it today in most jurisdictions."
Former leaders of the 9/11 commission issued a report card saying nine of its 41 recommendations remain unfinished.
The Office of the Director of National Intelligence has experienced growing pains as overseer of the 17 spy agencies, churning through four chiefs in six years.
Tensions over turf, confusion about the DNI's role, and problems herding agencies with very powerful chiefs of their own all came to a crescendo when retired Admiral Dennis Blair, the third DNI, tried to assert authority over CIA station chiefs, who represent the agency in different countries.
"The position of chief of station is one of the crown jewels of the CIA, and they don't want anyone playing with their crown jewels," said Mark Lowenthal, a former senior U.S. intelligence official.
After a dust-up with CIA Director Leon Panetta, who now is defense secretary, it was Blair who was sent packing.
"I think the mistake that some have made is to have viewed the DNI and the Director of CIA as an either/or proposition rather than the power of the two working together," the DIA's Shedd said in an interview in his office.
"There is a history of where that hasn't worked so well, I believe it is working much better today," said Shedd, who has worked at the DNI, CIA and National Security Council.
Intelligence experts say in the current administration, Obama's top homeland security and counterterrorism adviser John Brennan arguably has more power than any of them because he has the president's ear. It's a reminder that, bureaucratic reform or no, personalities count in making national security policy.
The improved sharing of secret data has led to yet another set of problems. The deluge of bits and bytes has subjected intelligence analysts to information overload as they try to sift through it all for relevant pieces.
"Our analysts still are spending way too much time on finding the information rather than on the analysis of the information," Shedd said. "There is just too much data to go find it all."
The intelligence community wants a system developed that would automatically process information from multiple agencies and then make the connections for the analysts.
But greater inroads into sharing data across agencies does not guarantee that another attack will be averted.
The threat has evolved and officials now are increasingly concerned about a "lone wolf" plot by an individual, not tied to any militant group, that may be more difficult to uncover.
"Those threats will not come to our attention because of an intelligence community intercept," said John Cohen, a senior Department of Homeland Security counterterrorism official.
"They will come to our attention because of an alert police officer, an alert deputy sheriff, an alert store owner, an alert member of the public sees something that is suspicious and reports it," Cohen said.
One measure of the success of post-9/11 reforms is that a decade later the United States has not had a similar attack.
"Now that could be luck, that could be skill, we don't really know," Hamilton said. "But in all likelihood what we have done, including the establishment of the Department of Homeland Security and the transformation in intelligence and FBI, has certainly been helpful."
Editing by Warren Strobel and Will Dunham