Network Security Entering No-Content Era

Firewalls have changed a great deal over the past 20 years. Originally, firewalls were just a simple set of rules to do packet filtering. Though they were not designed to deal with many of the problems that came later, firewalls are still a useful and necessary tool in the security arsenal.

Soon administrators needed to identify attacks on the network, stop viruses, stop spam, and block certain types of web traffic. They needed to stop certain applications or monitor and record network activity. Firewalls evolved to meet these needs by adding related network security technologies and content inspection.

Firewalls then entered a new era which combined the traditional routing and packet filtering base with new technologies like intrusion prevention, antivirus, antispam, web filtering, etc. Many new terms were coined to describe this evolution like “next-gen” and “UTM” or unified threat management.

Next-gen firewalls essentially reconstruct the application data from packets transparently and feed it to these other security technologies for inspection. This allows the antivirus engine to scan the content, and the web filter to identify and classify the web page, and the intrusion prevention engine to look for suspicious activity.

However, the ability to inspect the raw unencrypted content on the network is vanishing rapidly. Security technologies that rely on content-inspection are quickly becoming irrelevant.

SSL Drivers

SSL/TLS is a technology to verify and encrypt communications. SSL is not new; HTTPS (HTTP over SSL) has been around since most of us have been using HTTP. SSL is also used to verify and encrypt many other communications and protocols.

While this technology has been around for a long time, usage has skyrocketed recently. A few years ago, we were all searching Google using unecrypted HTTP. We were shopping on Amazon and eBay with HTTP (excluding the actual checkout). We were watching videos on YouTube in HTTP. We were reading Wikipedia with HTTP. In fact, we were doing pretty much everything in HTTP except logins, checkouts, and security-sensitive use cases like banking.

Today, you can do none of those things using HTTP even if you wanted to. All the major sites are switching to HTTPS for all traffic.

SSL adoption is being driven by a few things:

  • Security and Privacy Issues are Getting More Awareness

    It seems there is a new security breach of some major organization in the news every week now. Many countries (including our own) also struggle with internet freedom, net-neutrality, and censorship.

    Given these security and privacy concerns, it is natural for users to expect and desire SSL and for organizations to respond.

  • Google & Search Engine Optimization

    Google has started rank boosting HTTPS sites. Since search is a major source of traffic for almost all sites, most sites are highly incentivized to switch to HTTPS.

    Google’s Chrome browser also adds more warnings and annoyances for users doing things over HTTP. It appears one of Google’s goals is to drive HTTPS adoption, and it’s working.

  • Network Effect

    As more people adopt SSL, especially HTTPS, it is advantageous for others to adopt SSL as well. Given the way browsers handle HTTP- and HTTPS-mixed content, it is much easier for a site to also be HTTPS to avoid cross-protocol issues with referers and other corner cases, or for a site to link or load content from another HTTPS site. Now that all major sites and ad platforms are using HTTPS, smaller sites are really better off going to HTTPS.

  • Going SSL is Cheaper

    Acquiring an SSL cert is easier and cheaper than it’s ever been. There are even some free services like that are gaining traction.

SSL Adoption

The majority of web traffic is now encrypted. Looking at yesterday’s web traffic on an anonymized, semi-random subset of the Untangle fleet deployed in the real world, about 65.6% of all web traffic was SSL encrypted. (Sample size: ~170 terabytes, ~10000 organizations)

Web Traffic

Looking at the Untangle reports for an individual site can show the continued adoption over time. Below shows a single site percent SSL vs non-SSL web traffic by day through the past year.

SSL encrypted traffic

If we look at other protocols, we see the same trend at varying levels of adoption. Using the same dataset as above (yesterday’s anonymized data of ~170 terabytes of ~10000 organizations), 83.1% of IMAP email traffic is now SSL encrypted.

POP3 lags behind at 35.4% SSL, but is SSL adoption increasing (yet POP3 usage on the whole is declining). SMTP usage has been clear-text for ages and is now finally starting to see some real growth in using SSL to protect SMTP.

SSL effects on network security

At this rate, the huge majority of content sent over the network will be encrypted with SSL. As a thought exercise, let us assume 100% of application layer content is SSL encrypted. What does this world look like?

Traditional intrusion prevention rules are not very useful. 88% of our ruleset relies on the “content” match which will be useless in an SSL world. Gateway antivirus is useless because it relies on access to the content to evaluate if the content is malicious. Antispam is very difficult at the network layer because the content cannot be evaluated.

It is not all bad news though! Web filtering is still possible, just not at the very granular level that it was before encryption. Application Control can still identify the application and, in some ways, is even easier because it has the certificate from which to derive information. Many reputation-based security technologies are even better in this world because the certificate provides a way to verify authenticity and provide identification of the organization from which the reputation can be derived.

Long-time Untangle users can look at their own data and see this effect. You will likely see many more blocked accesses to malware sites based on reputation via Web Filter than you will see accesses blocked based on content scans in Virus Blocker.

SSL adoption will force network security vendors to shift to non-content based security approaches. While this trend may be obvious, we are often seeing the opposite. Many vendors are investing in network sandboxing in an effort in increase the efficacy of the antivirus engine––which is already not relevant at the network level and is becoming less so over time.

SSL Inspection

But wait! Won’t SSL inspection, also called man-in-the-middle or MITM, allow us to inspect SSL traffic as if it were unencrypted and give us full visibility to the content? Indeed it does. SSL inspection transparently decrypts and then re-encrypts SSL traffic so that things like antivirus and antispam can scan the content.

The problem with SSL Inspection is that it very difficult to deploy in the real world. SSL inspection requires adding a new certificate authority (CA) to the device. In some scenarios, like a school adding the CA to devices that it owns, SSL inspection can work great.

Outside of scenarios like this, SSL inspection can often be more of a headache than it’s worth. Why?

  • The diversity of devices on the network makes deploying the CA difficult since each device is different. Even browsers vary with each device.
  • The organization may not own the device on the network and may not be able to deploy the CA at all.
  • Many applications don’t use the OS’s “root store” and hard-code (or “pin”) certificates. In this case, SSL inspection just breaks the application.

Given these issues, many administrators simply abandon the effort. We are currently seeing less than 10% adoption of SSL inspection amongst Untangle customers.

However, the biggest issue facing SSL inspection is Google. As mentioned earlier, Google is one of the key drivers of SSL adoption. Now with Android Nougat (7.0), Google is making it very difficult for even administrators to add CAs to the root store on the device, making SSL inspection essentially impossible for these devices.

The purpose of SSL is to authenticate and encrypt, and SSL inspection essentially negates this purpose. I believe that it is one of Google’s missions to encourage SSL adoption so that users can safely connect to their service from anywhere without interference. For the record, I agree with this. SSL inspection, while it does have useful security applications, is also something that is likely to be abused. We need SSL to work for a safe and free internet. If Google continues down this path, and I believe they will, then SSL inspection is not going to be a viable long-term approach to handle SSL.


The era of network security products being able to analyze clear-text application data is ending. Many of our old content-based technologies won’t be as useful.

Luckily, SSL adoption brings many new opportunities and can empower some of our existing security applications. More on that in future posts!

– Dirk