By Joseph Menn

SAN FRANCISCO (Reuters) - Security industry pioneer RSA adopted not just one but two encryption tools developed by the U.S. National Security Agency, greatly increasing the spy agency's ability to eavesdrop on some Internet communications, according to a team of academic researchers.

Reuters reported in December that the NSA had paid RSA $10 million to make a now-discredited cryptography system the default in software used by a wide range of Internet and computer security programs. The system, called Dual Elliptic Curve, was a random number generator, but it had a deliberate flaw - or "back door" - that allowed the NSA to crack the encryption.

A group of professors from Johns Hopkins, the University of Wisconsin, the University of Illinois and elsewhere now say they have discovered that a second NSA tool exacerbated the RSA software's vulnerability.

The professors found that the tool, known as the "Extended Random" extension for secure websites, could help crack a version of RSA's Dual Elliptic Curve software tens of thousands of times faster, according to an advance copy of their research shared with Reuters.

While Extended Random was not widely adopted, the new research sheds light on how the NSA extended the reach of its surveillance under cover of advising companies on protection.

RSA, now owned by EMC Corp <EMC.N>, did not dispute the research when contacted by Reuters for comment. The company said it had not intentionally weakened security on any product and noted that Extended Random did not prove popular and had been removed from RSA's protection software in the last six months.

"We could have been more skeptical of NSA's intentions," RSA Chief Technologist Sam Curry told Reuters. "We trusted them because they are charged with security for the U.S. government and U.S. critical infrastructure."

Curry declined to say if the government had paid RSA to incorporate Extended Random in its BSafe security kit, which also housed Dual Elliptic Curve.

An NSA spokeswoman declined to comment on the study or the intelligence agency's motives in developing Extended Random.

The agency has worked for decades with private companies to improve cybersecurity, largely through its Information Assurance Directorate. After the 9/11 attacks, the NSA increased surveillance, including inside the United States, where it had previously faced strict restrictions.

Documents leaked by former NSA contractor Edward Snowden showed that the agency also aimed to subvert cryptography standards. A presidential advisory group in December said that practice should stop, though experts looking at the case of Dual Elliptic Curve have taken some comfort in concluding that only the NSA could likely break it.

"It's certainly well-designed," said security expert Bruce Schneier, a frequent critic of the NSA. "The random number generator is one of the better ones."

RANDOM NUMBERS

Cryptography experts have long been suspicious of Dual Elliptic Curve, but the National Institute of Standards and Technology and RSA only renounced the technology after Snowden leaked documents about the back door last year.

That was also when the academic team set out to see if they could break Dual Elliptic Curve by replacing two government-issued points on the curve with their own. The team published a summary of their study online on Monday (http://www.dualec.org)

and plan to present their full findings at a conference this summer.

Random numbers are used to generate cryptographic keys - if you can guess the numbers, you can break the security of the keys. While no random number generator is perfect, some generators were viewed as more predictable than others.

In a Pentagon-funded paper in 2008, the Extended Random protocol was touted as a way to boost the randomness of the numbers generated by the Dual Elliptic Curve.

But members of the academic team said they saw little improvement, while the extra data transmitted by Extended Random before a secure connection begins made predicting the following secure numbers dramatically easier.

"Adding it doesn't seem to provide any security benefits that we can figure out," said one of the authors of the study, Thomas Ristenpart of the University of Wisconsin.

Johns Hopkins Professor Matthew Green said it was hard to take the official explanation for Extended Random at face value, especially since it appeared soon after Dual Elliptic Curve's acceptance as a U.S. standard.

"If using Dual Elliptic Curve is like playing with matches, then adding Extended Random is like dousing yourself with gasoline," Green said.

The NSA played a significant role in the origins of Extended Random. The authors of the 2008 paper on the protocol were Margaret Salter, technical director of the NSA's defensive Information Assurance Directorate, and an outside expert named Eric Rescorla.

Rescorla, who has advocated greater encryption of all Web traffic, works for Mozilla, maker of the Firefox web browser. He and Mozilla declined to comment. Salter did not respond to requests for comment.

Though few companies appear to have embraced Extended Random, RSA did. The company built in support for the protocol in BSafe toolkit versions for the Java programming language about five years ago, when a preeminent Internet standards group - the Internet Engineering Task Force - was considering whether to adopt Extended Random as an industry standard. The IETF decided in the end not to adopt the protocol.

RSA's Curry said that if Dual Elliptic Curve had been sound, Extended Random would have made it better. "When we realized it was not likely to become a standard, we did not enable it in any other BSafe libraries," he added.

The academic researchers said it took about an hour to crack a free version of BSafe for Java using about $40,000 worth of computer equipment. It would have been 65,000 times faster in versions using Extended Random, dropping the time needed to seconds, according to Stephen Checkoway of Johns Hopkins.

The researchers said it took them less than 3 seconds to crack a free version of BSafe for the C programming language, even without Extended Random, because it already transmitted so many random bits before the secure connection began. And it was so inexpensive it could easily be scaled up for mass surveillance, the researchers said.

(Editing by Edwin Chan and Tiffany Wu)

Word Count: 1022