Richard Bejtlich Sticks Up For IDS. I Retaliate.
Thomas Ptacek | October 28th, 2006 | Filed Under: Industry Punditry, Uncategorized
Richard Bejtlich pulls another DailyDave thread into his blog, irritated by the security community’s dismissal of intrusion detection. Couple things.
We Get To Talk About IDS, Richard
Richard Bejtlich does not get to dismiss Dave Aitel and Halvar Flake as “offensive computing” experts who don’t understand IDS, and, personally, if he calls me an “offensive computing” person again I’m going to ask for an apology. Richard, these people are your peers, and you read DailyDave for the same reason I do: it’s an excellent list.
It’s Not About Signatures
The IDS/IPS problem, which is now a decade old, is that lots of smart people in operations, research, and development are skeptical that it works at all. You cannot bait your way out of this problem by saying “that’s just signature IDS”. Sorry, Richard, I know how Bro works and I’m pretty familiar with anomaly detection. These ideas have, if anything, amplified the fundamental signal/noise problem “attack detection” creates.
Does Anyone Actually Rely on IPS?
On DailyDave, I asked the same simple question I always ask about IPS:
“I am waiting for someone to tell me the story about how an IDS saved their bacon. I’m not interested in the story about how it found the guy with the spyware infection or the bot installation; secops teams find those things all the time in their firewall logs and they don’t freak out about it when they do.”
Richard says, “it saved my bacon all the time at Ball Aerospace”. Alright, Richard, now tell us a story. And if the story ends in “so I scrubbed the infection off the desktop and nobody ever had to think about it again”, please find another story.
What Are These “Great Ideas” In IDS People Talk About?
I made a somewhat inflammatory point:
“Intrusion detection has been an active field of research for over 15 years now and apart from Tripwire I can’t point to anything operationally valuable it has produced.”
This should be an easy one to knock down. But all you say is, “this sounds like the ‘Snort is worthless’ argument”. You’re a smart guy who thinks about this stuff all the time, Richard. Can you actually address the argument I really made? I know a whole bunch of my readers can (and probably will, with expletives).
Firewalls and IDS Are Not Comparable
IDS is not “pigeonholed the same way firewalls are”. There’s a difference: firewalls are hugely successful, perhaps the single most important piece of security technology enterprises buy. It is absolutely unimaginable for a large company to be connected to the Internet without them. You cannot say the same thing about IDS/IPS; lots of enterprises don’t use it, and they aren’t suffering.
ps: the stuff about tape drive speed? that was a joke. about tape drives.
10/28: NB: We have lots of very talented friends who are real believers in monitoring and even IDS; some of them are talking here, some of them on DailyDave, and doubtless some are just reading and shaking their heads. I don’t think any less of any of these people, including Richard, because they buy the IPS story and I don’t. But I have an opinion and intend to be ruthless about transmitting it.
William
October 28th, 2006 1:33 amOk, let me be the first one to lob a grenade!
IDS/IPS are TOOLS like so many others in security they are not the end all be all, holy grail, save my bacon, white knight, take it home to mama applications and appliances that vendors would love for you to believe. They ARE part of what I can generally call a “general detection / protection mesh”.
I personally fail Thomas’ challenge to describe an instance where I can safely say, “If I didn’t have snort(or X IPS/IDS) product in place, my company may not still be around” . Does this mean that they are not effective? NO. They have a place in the security world, if they are properly maintained they can be invaluable in providing detection for such things as “X apache module is vulnerable to DOS using XXXX payload” detection of this payload and quick response, either by an IPS or by a human in the case of an IDS could in fact, save the day.
The very things you chide IDS/IPS for in statement 3 are things that security personnel SHOULD be worried about. Botnets, infections, spyware, et al. Downplaying their importance is not helpful to community. Are they the “end of the world” aspects that you concern yourself most with? Obviously not.
As far as statement 4 and 5 are concerned, I have to agree with you for the most part. Innovation is lacking in the IDS/IPS arena, the same ole same ole gets rehashed in every new application or appliance. Firewalls are arguably the most important security innovation of all time. However I believe it unfair to use the success of Firewalls to downplay the need or effectiveness of IDS/IPS. Not every product or idea will be as grand as its predecessors, but they still have their place.
ps: Im sure that the comments that follow will be weighted heavily towards one side of the aisle or the other. IDS/IPS is a very polarizing issue in the security community, I stress the need for everyone to keep an open mind to arguments made on both sides, and draw your own conclusions. What it comes down to is the importance you place on specific countermeasures for your environment, you are in the end responsible for your own actions.
Bamm
October 28th, 2006 11:48 amIn a perfect world…
…FWs/IPS (host and network) would prevent 100% of any attack on against the enterprise. No one would need to patch applications or operating systems. Instant Protection Systems rock, monitoring the network would be a waste.
In a perfect world…
…effective patch management and vunlerability scanning would thwart 100% of the vulnerabilities within an OS or application. Exploits would never happen and the term zero-day would be wiped from wikipedia. Monitoring the network would be a waste.
In a perfect world…
…$SOMETHING would prevent 100% of $EVERYTHING. And it’d be an appliance that would cost under $5000 so I could expense it. Or better yet, it’d be open source and could be installed by blinking.
Unfortuneately, I live in the real world and shit happens (I have the t-shirt to prove it). I don’t think anyone is going to argue that FWs and patch management aren’t a necessity. I also doubt that anyone will garauntee (backed by dollars) that your network cannot be compromised with them in place (without a laundry list of perfect world outs).
I personally cringe whenever I hear the term IDS. Vendors have commoditized IDS into an underachiving wannabe wiz-bang appliance. IDS should be part of a monitorying solution. I hear there is a book that does a good job of outlining this thing called Network Security Monitoring.
As far as someone telling you about how IDS saved their bacon, the first rule of the Fight Club is not to talk about the Fight Club ;). If you honestly want to hear a story about how NSM saved some bacon (with IDS as part of the process), then please feel free to contact me directly. I’ll even give you my phone number. My only requirement is that you keep an open mind. Yes, there are thousands of different ways that the incident(s) I’ll tell you about could of been prevented and there are just as many ways the attacker could of avoided detection. But that’s not the point, I live/work in the real world. And shit happens.
Bammkkkk
Comedic Effect
October 28th, 2006 12:13 pmQ. Whats the difference between IDS and IPS?
A. $30K and a few flashing lights.
IDS is a function, a process, it is not a product « Observations of a digitally enlightened mind
October 28th, 2006 1:42 pm[…] Richard from Taosecurity (here) and Ptacek from Matasano (here) are apparently blog-debating the value of IDS, a discussion that spawned from the daily dave list. I normally do not get into the middle of a debate, but I was a little surprised we are discussing the advantages/disadvantages of IDS in 2006. So I am tossing in my 21 pesos. […]
ivan
October 28th, 2006 3:55 pmI think that the ‘IDS/IPS debate’ that re-surfaces reguarly in security mailing lists and blogs is interwhined with, at least, two other adjacent debates. Besides the very valid points about the stagnation of IDS/IPS technology and lack of innovation for effective operational solutions that some people made (I totally agree with you there but I dont dismiss IDS/IPS tchnologies as useless because of that) there is also a discussion about the legitimacy and value of ‘offesive computing’ people and technologies. Dismissing the offensive computing viewpoint is equally bad than dismissing the defensive one and only leads to the zero-sum game that has been going on for more than a decade and a half.
I think this zero-sum game has been fueled by a pervasive lack of humbleness (or abundance of arrongace) from the security community ’superstars’ on both sides of the fence. Daily Dave and other ‘offensive’-biased forums demonstrate the telltale signs of that view: “I’m too good for any defensive stuff, I can break anything and everything and therefore nothing really works and its all useles crap”. Defensive ’superstar’ Marcus Ranum’s lively diatribes against the role of offensive security people are a good example of the counterbalancing force in this ‘debate’. And guess what?… nobody cares! Most end user organizations are looking for ways to solve their real world problems not for the definite proof of what’s The Right Security Philosophy
On top of that, there’s also a ridiculous search for silver bullets that inevitably lead to failure in the real world. Sadly, it is more convenient for everybody (security vendors, customers, ‘analysts’ and the expert community) to think that you’re building, buying or selling a silver bullet than to allow yourself to think that there are no silver bullets and that all security solutions are partial and flawed and you *have to* do your homework to find the right mix for your environment.
The “death of the IDS” has been greatly exagerated but that doesnt necesarilly mean that IDS/IPS technologies are inmortal.
Tom: Regarding point #5, yes it is unimaginable for a large company to be connected to the Internet without firewalls, but what does that prove? That firewalls are intrisinc networking technology? Or perhaps, that large companies lack the imagination to envision a different network security paradigm? The former would lead me to think that the network security field is possed for evolutionary change only, the later leaves room for a revolutionary change… somewhere, sometime.
alan shimel
October 28th, 2006 5:01 pmThomas, I don’t think it is quite as black and white as you make it out to be. I also am not sure you are thinking of the many SME-SMB admins who use IDS as part of their arsenal every day of the week and what it means to them. I have put up a short response here:
http://www.stillsecureafteralltheseyears.com/ashimmy/2006/10/the_peak_of_inf.html
PaulM
October 28th, 2006 7:49 pmTom-
I think you’re asking the wrong question in looking for an existence proof for “IDS works”. You can use a ziplock baggy as a condom and sometimes you won’t get pregnant. The question isn’t whether IDS ever works, but whether it’s worth the $500 million that gets spent every year and the $billion+ to deploy and manage it.
It isn’t.
toby
October 30th, 2006 1:03 amTom, I won’t dismiss Dave and Halvar for being primarily pentesters/vuln finders/vuln researchers, but I will point out that they have a very different perspective and background than people who have to defend networks primarily.
Also, you need to distinguish between IDS and IPS since we _need_ to monitor and that’s what IDS gives us, whereas IPS is utter shite and a boil on the ass of the toad that is marketing in infosec products.
IDS may suck, tell me what we do instead. I know IDS has serious issues, I quote your papers and all recent work on evading and breaking and compromising IDS frequently. That doesn’t mean we get to _not_ monitor the network.
Adam
October 30th, 2006 1:43 amToby,
Encrypt it all for good security reasons, then exclaim “This damn IDS can’t read encrypted packets!” and throw it away. It’s a win/win!
gustavo
October 30th, 2006 2:28 pmThe problem here is trust, often we see people running IPS and having dreams about safety. I don’t expect anyone thinking about it.
PaulM
October 30th, 2006 3:15 pmIDS has been declared dead so many times that I have lost count. Most hilariously in 2003 by Gartner, who predicted that “IPS” would obsolete IDS without getting their own joke.
At the end of the day, IDS is good at plucking low-hanging fruit out of the noise and recording it. That’s useful.
“I am waiting for someone to tell me the story about how an IDS saved their bacon. I’m not interested in the story about how it found the guy with the spyware infection…”
Why these have to be two seperate things is beyond me. If an IDS caught a screen-scraper trying to send logins and private information back to some .ru site and blocked it, that’s rescued pork, no question. The associated regulatory fines alone would be 2x-3x what the average company spent to clean up MS-Blaster. Keeping all the servers in the DMZ from getting owned shouldn’t be the bar by which we judge the value of a technology, because if we did:
“There’s a difference: firewalls are hugely successful, perhaps the single most important piece of security technology enterprises buy.”
…then firewalls, suck, too.
djm
October 30th, 2006 5:21 pmIf you conflate IDS with IPS, then you give yourself the luxury of attacking each for the other’s failings. Nice strawman!
mcwresearch.com » Ptacek on IDS/IPS
October 30th, 2006 5:48 pm[…] Thomas Ptacek over at Matasano Chargen has strong opinions against IDS/IPS and has some valid points but his fundamental argument is wrong. When Thomas stated the following, he made it clear (to me anyway) that he is just dramatizing the issue: “Can you actually address the argument I really made? I know a whole bunch of my readers can (and probably will, with expletives).” […]
Thomas Ptacek
October 30th, 2006 8:54 pmdjm: provide a definition of “IPS” that doesn’t capture RealSecure 1.0, and isn’t simply “inline”.
ivan
October 30th, 2006 9:37 pmThomas: how about this definition “Has the ability to stop from succeeding the attacks that it detects” does that capture RS 1.0?
Btw, you don’t really need to be “inline” to accomplish the above… i know, it sounds like blasfemy to the protocol police but just think about it for a few minutes…insertion, evasion & dos need not be just offensive techniques. Possibly not “the right thing” or the most elegant but… what if it actually works? Doesn’t RNA do any of this?! There.. Mr Roesch, if you do it is will be expecting some complementary stock from that upcoming IPO
Hey, why not? redhat did it!@#
Martin Roesch
October 31st, 2006 12:10 amThomas Ptacek
October 31st, 2006 12:33 amThe sweet
of liquidity. Marty’s :)’ing all the way to the bank. 
PaulM
October 31st, 2006 12:11 pmThat’s Dave Aitel’s real beef, isn’t it? Not that people use IDS, but that Marty’s company is trying for a pay day (that it was earlier denied w/ the Check Point deal).
So Marty should smile. And Dave should keep on hating. No doubt someone will hate all over him when Immunity gets bought or goes public.
Thomas Ptacek
October 31st, 2006 12:32 pmI don’t think Dave has a beef with Marty, and I think Dave gets real paid on a regular basis.
The Units of Risk and Learning how to Measure Them! at RiskAnalys.is
October 31st, 2006 12:51 pm[…] The FAIR whitepaper dedicates an entire section to controls. I won’t duplicate what it says here, but I will mention this: Controls are either preventative, detective, or aid in our ability to respond to an incident. Now, this weekend there was quite a bit written about IDS/IPS and the value of that technology or process. I don’t have much to say about that particular technology except the following: […]
SecuriTeam Blogs » Is the IDS/IPS Still Relevant? Was it ever?
November 1st, 2006 12:11 am[…] Although both are written by security experts that have earned respect in their fields, I can’t help but feel like I’ve been watching a security fashion show after reading these two posts: Response to Daily Dave Thread Richard Bejtlich Sticks up for IDS. I Retaliate. […]
PaulM
November 1st, 2006 11:31 am@Tom - I didn’t *mean* that Dave has any personal issue with Marty or even a problem with Sourcefire, though I guess I did sort of *say* it.
But I can see how Dave has a hard time seeing the value in NIDS as a technology because he has developed effective methods for evading NIDS products.
It’s akin to an expert car thief* who can’t understand why people still buy car alarms because he has no trouble getting around them. But people buy them because they work some of the time, and that’s better than none of the time.
* For the record, I’m not comparing Dave Aitel or anyone else to a criminal of any sort.
Leave a reply