Security product testing


Just a quick rant about the comparison of different security products (in the largest sense of the word):

Many times we see things said like product X stops 100% of all (known) malware. First there is of course the problem that some people omit the known part which makes them by default ripe for lawsuits. But lets take a much weaker claim (which is a superset of the previous claim): this product stops 99% of all known malware.

The argument usually goes like this: the product was tested on N malware samples and it stopped them all. The fallacy in this argument usually is that the malware samples the tool was tested with were not created with this tool in mind. In general such new wonder tools are used by a very restricted circle of people and thus it newer got targeted by malware writers. But if the hype would to have its effect and a significant amount of people would start using it, it would become a prime target and cracked / bypassed very fast.

Now mind you, I’m not railing against diversity. However when a security product is only as effective as running with lower privileges, you have to wonder:

  • Why is somebody paying for this?
  • Why is somebody wasting her/his time to write this? Why isn’t s/he instead writing a good tutorial about using the security features already built in the OS?
  • Is the coder as ignorant that s/he doesn’t realize the limits of the program or is s/he simply a shady entrepreneur who wants to get as much money out of the hype as possible before the bubble burst? I wouldn’t want to run the code on my system in either case.

Emphasizing my original point again: layered security is good, however if the security promised by a tool is equivalent to the one delivered by something already present in your OS, then (a) why are you paying (literally and/or metaphorically – by taking time to install and configure it) for the product? and (b) why are you creating possibly new vulnerabilities by adding new code on to your system?

PS. The most BS type of hype I’ve seen is when some company admits to being the target of a targeted attack and then a security vendor comes out and says: our product would have prevented this. No it wouldn’t! Targeted attack equals reconnaissance before the action. The specific Trojan/RAT/etc was employed because the attackers knew that the given company uses security product X which doesn’t detect it. Would they have used product X and product BS (or just product BS), the attackers would have chosen a different path and still been effective!


One response to “Security product testing”

  1. Systems Administrators have very grounded opinions about what people should or should not do with their machines. However, if they wish to turn off local administrator access (for example), they have to explain to people, and for some subset of their userbase, “I’m doing this for your own good” is going to translate to “You’re not qualified to do this, you idiot, so I’m turning it off.”

    Depending, of course, on how tactless the systems administrator is, and how high maintenance the user population is.

    On the other hand, if they buy “security products” and install them, and the “security product” disables something the users wants, the systems administrator can nod his head and say, “Yes, I understand why you find this annoying, but our security software doesn’t allow it” -> the systems administrator gets to play “good cop” and put the security software in the role of “bad cop”.

    It’s a trust issue: it is very common for people to want to argue with their IT guy (and why not, they run their own computer at home), but they can’t argue with the faceless security software. Plus, the security software was written by experts, right?

    Remember the old saw, “An expert is someone who doesn’t work here” 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *