The forces that led to the DigiNotar hack

Last week, the New York Times finally covered the DigiNotar hacks, more than two weeks after security experts and the tech media first broke the story. Unfortunately, the top 2-3 newspapers in the US (which is what legislative staff, regulators and policy makers read) have missed most of the important details. The purpose of this blog post is to fill in those gaps, providing key context to understand this incident as part of the larger Internet trust (and surveillance) debate.

Lawful access

As consumers around the world have embraced cloud computing, large Internet firms like Google, Facebook, Twitter, Yahoo, all of them based in the United States, increasingly hold users' most private documents and other data. This has been a boon for law enforcement agencies, which can often obtain these files without a court issued search warrant, or have to provide the investigated individual with the kind of prompt notice that would otherwise occur had their home been searched.

Law enforcement and intelligence agencies in the US, EU, Canada, Brasil, India, Japan, Israel and several other countries all regularly obtain private user data from Google. The company will insist on a court order for some kinds of user data, but will disclose many other types of data and subscriber records without first insisting on an order issued by an independent judge. This isn't because Google is evil, but because privacy laws in these countries, the US included, are so weak.

Google does not treat all governments equally though. For example, the company will not honor requests from the governments of Iran, Libya, Zimbabwe, Vietnam and several other countries. You might be inclined to believe that Google has taken this position because of the poor human rights record in these countries - that is part of the reason (but not the whole one, otherwise, Google would refuse requests from the US government which has a documented track record of assassination, rendition/kidnapping and torture). Google's policy of refusing these requests, I believe, largely comes down to the fact that Google does not have an office or staff in those countries. Without a local presence, employees to threaten with arrest or equipment to seize, these governments lack leverage over Google.

This situation is not specific to Google - Facebook, Yahoo, Microsoft and other large US firms all disclose user data to governments that have leverage over them, and ignore requests from others. Thus, lacking any "legitimate" way to engage in what they believe is lawful surveillance of their citizens, these governments that lack leverage have turned to other methods. Specifically, network surveillance.

An unintended consequence of HTTPS by default

When users connect to Facebook, Twitter, or Hotmail—as well as many other popular websites—they are vulnerable to passive network surveillance and active attacks, such as account hijacking. These services are vulnerable because they do not use HTTPS encryption to protect all data as it is transmitted over the Internet.

Such attacks are trivially easy for hackers to perform against users of an open WiFi network using tools like Firesheep. They are also relatively easy for government agencies to perform on a larger scale, when they can compel the assistance of upstream ISPs.

As I described above, because Google will not respond to formal requests for user data from certain governments, it is likely that the state security agencies in these countries have come to depend on network interception, performed with the assistance of domestic ISPs.

Unfortunately for these governments, in January 2010, Google enabled HTTPS by default for Gmail and a few other services. Once the firm flipped the default setting, passive network surveillance became impossible. Thus, in January 2010, the governments of Iran and a few other countries lost their ability to watch the communications of domestic Google users.

For now, these governments can still spy on Facebook, Twitter and Hotmail, as these services do not use HTTPS by default. That is changing though. Following the release of Firesheep in October 2010, (as well as two senior US government officials calling for encryption by default) all three services now offer configuration options to force the use of HTTPS. These firms are all moving towards HTTPS by default - for some firms, it will likely be a matter of weeks until it happens, for others, months.

Governments can see the writing on the wall - HTTPS by default will become the norm. Passive network surveillance will lose its potency as a tool of government monitoring, and once that happens, the state intelligence agencies will "go dark", losing the ability to keep tabs on their citizen's use of foreign, mostly US-based Internet communications services.

HTTPS Certificate Authorities and surveillance

As these large providers switch to HTTPS by default, government agencies will no longer be able to rely on passive network interception. By switching to active interception attacks, these governments can, in many cases, easily neutralize the HTTPS encryption, thus restoring their ability to spy on their citizens. One active attack, known as a "man in the middle attack" requires that the government first obtain a HTTPS certificate issued by a Certificate Authority (CA) trusted by the major web browsers.

In March of 2010, Sid Stamm and I published a paper on what we called compelled certificate creation attacks, in which a government simply requires a domestic Certificate Authority issue it one or more certificates for surveillance purposes. When we released a draft of our paper, we also published a product brochure I had obtained in the fall of 2009 at the ISS surveillance conference, for a Packet Forensics interception device that described how it could be used to intercept communications using these kinds of certificates.

The browsers trust a lot of Certificate Authorities, probably too many. These include companies located in countries around the world. They also include Certificate Authorities that are operated by government agencies. For example, Microsoft trusts a couple dozen governments, that include Tunisia and Venezuela. It is perhaps worth noting that Microsoft continues to trust the Tunisian government even after it was caught in December 2010 actively hijacking the accounts of Facebook users -- an act that led to Facebook enabling HTTPS by default for all users in the country.)

In any case, as Sid an I described, governments can compel domestic Certificate Authorities to provide them with the certificates necessary to intercept their own citizens' communications. However, not all governments around the world are as lucky as Tunisia to be trusted by the browsers, nor do all of them have a domestic certificate authority that they can bully around. Some countries, like Iran, have no way to obtain a certificate that will let them spy on Google users (yes, I know that you can buy intermediate CA issuing powers, but I am assuming that no one will sell this to the Iranian gov).

In recent weeks, we have learned that the encrypted communications of 300,000 people in Iran were monitored by an entity using a certificate that DigiNotar issued. While the Iranian government has not admitted to conducting this man in the middle surveillance against its citizens, it seems reasonable to assume they were behind it. The reason for this certificate theft seems pretty clear, when you consider the other details described in this blog post:

Iran wants to spy on its citizens. It wants the same interception and spying capabilities that the US and other western governments have. Unfortunately for the Iranian government, it has no domestic CA, and Google doesn't have an office in Tehran. So, it used a certificate obtained by hacking into a CA already trusted by the browsers - a CA that had weak default passwords, and that covered up the attack for weeks after it learned about it, giving the Iranian government plenty of time to use the stolen certificate to spy on its citizens.

As Facebook, Twitter and other big sites embrace HTTPS by default, the temptation will grow for for governments without other ways to spy their citizens to hack into certificate authorities with weak security. Can you blame them?

NSA and other US government agencies have gambled with our security

In December 2009, after I had obtained Packet Forensics' product marketing materials, I met with a former senior US intelligence official. I told him that I believed that governments around the world were abusing this flaw to spy on their own citizens, as well as foreigners. When I told him I would be going public in a few months, motivated by my concerns about China and other governments spying on Americans, he said I would be aiding "terrorists in Peshawar" by helping to secure their communications. Needless to say, our meeting wasn't particularly productive.

US intelligence agencies have long known about the flaws associated with the current certificate authority web of trust. For example, in 1998, James Hayes, an air force captain working for the National Security Agency published an academic paper in which he described the ease with which certificates could be used to intercept traffic:

Certificate masquerading allows a masquerader to substitute an unsuspecting server’s valid certificate with the masquerader’s valid certificate. The masquerader could monitor Web traffic, picking up unsuspecting victims’ surfing habits, such as the various net shopping malls and stores a victim may visit. The masquerader could change messages at will without detection, or collect the necessary information and go shopping on his or her own time.

Of course, it isn't too surprising that NSA has known about these vulnerabilities. If the agency hadn't know about these risks, it would have been grossly incompetent.

The question to consider then, is what has and hasn't the NSA done with this knowledge. In addition to attacking the computers of foreign governments, NSA is supposed to protect US government electronic assets. In the 10 years since NSA first acknowledged it knew about the problems with certificate authorities, what steps has the agency taken to protect US government computers from these attacks? Likewise, what has it done to protect US businesses and individuals?

The answer, I believe, is "nothing". The reason for this, I suspect, is that NSA wanted to exploit the flaws itself and didn't want to do anything that would lead to the elimination of what is likely a valuable source of intelligence information -- even though this meant that the governments of China, Turkey, Israel, Tunisia and Venezuela would have access to this surveillance method too.

Perhaps this was a reasonable choice to make, when the intelligence agencies abusing the flaw could be trusted to do so discreetly (The first rule of State-run CA Club is...). The Iranians have upset that delicate understanding. They have acquired and used certificates in a manner that is anything but discreet, thus forcing the issue to the front page of newspapers around the world.

Now, any state actor or criminal enterprise with a budget to hire hackers can likely get its hands on fraudulent certificates sufficient to intercept users' communications, as Comodo and DigiNotar will not be the last certificate authorities with weak security to be hacked. Hundreds of millions of computers around the world remain vulnerable to this attack, and will likely stay this way, until the web browser vendors decide upon and deploy effective defenses.

Had the US defense and intelligence community acted 10 years ago to protect the Internet, instead of exploiting this flaw, we would not be in the dire situation that we are currently in, waiting for the next hacked certificate authority, or the next man in the middle attack.