Appliance-Based Fuzzing: Will Spirent Bite?

Thomas Ptacek | April 10th, 2006 | Filed Under: Bitching About Protocols, Industry Punditry

Now this sounds like Dug writing.

I have two reactions to appliance-based protocol fuzzers:

1. I don’t understand why speed is going to be an important differentiator for these solutions. Say I want to test TLS. I can:

   • buy a Mu-4000, which is fast but may or may not actually fuzz X.509v3, or

   • call up Codenomicon, not pay the appliance premium, and buy something built on PROTOS, which has probably the best ASN.1 test suite available, or

   • wait for someone to publish the 12,000-file testcase suite for TLS handshake negotiation and run it through netcat for free

   Do I want the test suites to complete faster, or do I want them to find more vulnerabilities?

   If I want to test performance while under attack, why wouldn’t I just generate docile background traffic while running the tests?

2. It’s not terribly hard to figure out the market opportunity here.

   About 1/3 of Spirent’s ‘05 revenue (310MM, of ~900MM) came from performance testing. 2/3 of that revenue went to vendors and service providers. Security testing:

   • skews slightly more into enterprise customers (companies do more security testing than performance testing), and

   • there are 100 times as many vendors who could “use” security testing (anybody who opens a socket, vs. anyone who forwards ethernet frames).

   And, unfortunately, your average enterprise software vendor is probably exactly the demographic who finds $50,000 of value in a plug-and-play appliance, instead of running PROTOS and Metasploit.

   Which is why Spirent cites security testing as their #1 growth driver in their investor summary. I’m waiting to see which of Codenomicon, Mu, or Breakingpoint they pick up for $50MM.

You want to remember that this kind of testing, while clearly valuable for IPS evaluation, is really geared at enterprise software. Things like Mu and BreakingPoint are going to do a good job of finding the next CommuniGate overflow, and, sure, they’ll uncover the occasional TippingPoint bug. But IPS testing is about evasion (being able to slip attacks past an IPS is almost as bad as being able to run code on it), and from what I can tell, the state of the art here is still open-source.

ps: the funny rumor I keep hearing is that BreakingPoint is actually hardware accelerated. Can this be true? Why?!