Security Appliance Tests - What We're Learning About Threat Prevention

January, 14, 2015 Franklyn Jones

In my last blog, I highlighted some interesting research in a new report published by Delta Testing in the UK. In case you missed that, they tested several different, well-known security appliances to see if they could detect and block advanced cyber threats. The results were horrible, with successful threat detection rates as low as 5% on one appliance.

Coincidentally, another report was recently published by another UK testing lab, MRG-Effitas. Like the Delta Testing report, this one also focused on security appliances designed to detect and block advanced persistent threats (APTs). And like the Delta Testing report, the MRG report also highlights the failure of these appliances to detect and prevent 100% of attacks.

In fairness, it is true that the Delta test demonstrated a 99% success rate in APT detection for one appliance. And that is very good, right? Yes, absolutely, 99% if exceptional – until you're business is successfully breached by that remaining 1%. And when that happens, it is highly unlikely you will run to your boss and say, "Great news! We blocked 99% of all cyber attacks this week!"

After becoming a victim of a costly cyber attack that apparently began with that 1% of browser-borne malware, JP Morgan Chase (among many others) now realizes that 99% APT protection is the same as 0% at the end of the day. Anything less than 100% protection is failure.

But let's get back to these tests for a minute. I'm sure you've seen many independent security test results that often prove detection success rates as high as 100%. I've seen those reports myself. But if you look closer at their test methodology, they typically use known malware samples. Any halfway decent security appliance should be able to easily identify and block known malware. That's table stakes.

The problem is cyber criminals tend to be a bit more creative than that – they develop complex, never-before-seen cyber threats that increasingly escape detection and gain control of network resources. If you read the MRG-Effitas report, you'll see that they developed their own unique piece of software called BABO, designed to escape detection. According to the report:

BAB0 is a custom designed sample written in C++ with a server side written in PHP. It was designed to be as stealthy as possible, and utilizes multiple methods to avoid detection. Actually, this test case simulates attackers with moderate resources and some understanding of the state-of-the-art detection tools and how advanced malware work.

The result? None of the security appliances detected and blocked this threat. So it seems that, with a bit of intelligence and creativity, escaping detection is quite doable. But you already knew that.

The only way to get from 99% to 100% is to shift your security strategy from detection to isolation. Since the web browser is a favorite attack vector for cyber criminals, start by isolating the browser and all browser content outside your network. As a result, any and all browser-borne malware also stays outside your network, which means that your chances of winning the cyber security battle just got a whole lot better. To learn more about how we can help, check out the short video on our home page at

Franklyn Jones, CMO, Spikes Security


Keep informed.