Archive for October, 2005
Thomas Ptacek | October 31st, 2005 | Filed Under: Disclosure
Buried in a comment, Peter says:
Why on earth should people be forced to “make a case” against
helpfulness? If it was really just help, couldn’t they decline the
favor? No, the onus is on bugfinders (finding a buffer overflow
really isn’t research anymore) to offer their service and wait for a
request - that’s how “help” works.
Peter has misunderstood something fundamental about the process. Let’s “help”
him get his head around this:
Vulnerability researchers aren’t “helping” the vendors and producers
of software by publishing vulnerabilities (though that is often a
side-effect of their work). They’re “helping” the customers who deploy
that software. Those customers very much want that “help”, which is
why providing that “help” pays so fucking well.
If Peter Lindstrom is advocating for a process whereby researchers will post
to Bugtraq and say, “I’ve found a new remote code execution bug in Snort,
would anybody like to take me up on my offer to help them identify and
eliminate it?”, that’s a valid, though stupid, argument to make. Peter Lindstrom
will simply have added about 34 seconds to the exact process we have now[*].
I don’t think that’s Lindstrom’s argument, but I’m waiting to hear the
coherent alternative argument to synthesize from the points he’s making.
Global 2000 enterprises spend tens of millions of dollars a year —-
just shy of 100 million dollars, not counting in-house headcount
expenses or revenue from products whose value props depend on security
research —- on targeted vulnerability research. This is an existence
proof of the demand for information about vulnerabilities in deployed
software.
3 Comments
Thomas Ptacek | October 31st, 2005 | Filed Under: Disclosure
You can sum up Lindstrom’s
arguments
this way: the industry is best served by making life difficult for
weak attackers, even at a cost of making life easier for strong
attackers.
We can go round and round about whether this is a useful
strategy, but my read is that this argument has become boring,
especially when it’s carried on between me and Peter
Lindstrom.
Suffice it that I don’t believe “weak attacker” and “strong attacker”
are useful distinctions for software security. Unlike cryptographic
data security, where “strong attackers” benefit from millions of
dollars in equipment, and where it is therefore useful to employ
gradations like “foreign government” or “organized crime”, clever
software exploits are no more expensive than cookbook exploits.
To the rest of Lindstrom’s argument, I’ll lapse into smug laziness and
say that Lindstrom’s opinions about what Neel Mehta, myself, or anyone
else will inspect has no impact on what we’ll be publishing. On the
other hand, Mehta’s opinion on what is “fair game” has a LOT of impact
on what will get published, and, in the immortal words of Nelson Muntz,
“hAH-hAH!”
You don’t need to consider motives to defend “opposition research”:
Snort is a high-value target which is intentionally exposed to peer
review by nature of an open codebase. It’s definitely a “security hot
spot”, and overtly positioned that way. Does Mehta get an emotional
kick out of beating a competing security system? I certainly do. Does
that impact the value of strengthening a widely deployed security
tool? I’m waiting for someone to show me how.
I don’t even think Marty buys the idea that finding bugs in Snort is
unproductive.
No Comments
Thomas Ptacek | October 29th, 2005 | Filed Under: Disclosure
I feel silly sticking up for Neel Mehta, who doesn’t need any help from
me, but to clear something up: just because ISS published the advisory
doesn’t mean that Mehta was forced to research Snort. I would be
surprised if Mehta doesn’t have a huge explicit say in what his
targets are, and it is clear to everyone in the industry that he has a
total implicit say, since he’s a team MVP.
The process of moving a finding from vendor disclosure to public
advisory usually takes a long time. The timeline here seems to be:
This unusually fast turnaround is a credit to the SourceFire team; on
the other hand, Snort is open-source, so once the fix is committed
there’s an incentive to publish formally, before black-hats notice.
I don’t condone CERT’s involvement here, but for a totally different
reason than Bejtlich. Bejtlich sees it as evidence that ISS went out
of their way to throw an elbow at SourceFire. But CERT, ISS, and
SourceFire coordinated the advisory and didn’t release it until
SourceFire patched; that’s the same process as would have occurred
without CERT. And that’s my problem with CERT: I don’t see the value
they add, apart from acting as an unneeded dampener in the
disclosure-publication process. Maybe someone can clear that up for
me.
In any case, coordinating an advisory is a huge pain. My
company handles it for me. Neel’s handles it
for him. I think it’s safe for me to put words in his mouth and say
that Neel would rather be getting real work done, or for that matter
be getting eaten by a bear in British Columbia, than to be on the
phone with CERT and SourceFire for a week.
Finally: nobody has made any case for why “opposition research”
(Jaquith’s term) is anything other
than helpful to the community as a whole. Snort is widely deployed and
perfectly fair game, by which I mean, people who take the time to
improve its code are doing the community a service, regardless of what
you think their motives are.
3 Comments
Thomas Ptacek | October 25th, 2005 | Filed Under: Bitching About Protocols, Matasano, New Findings
iSCSI is a network protocol that creates the illusion, on behalf
of a network storage system, that network storage clients have
directly attached SCSI disks (LUNs). It does this by proxying SCSI commands
(CDBs) over the network. Since modern Ethernet is faster than modern
SCSI drives, this works pretty well.
Until you get to the security aspects, that is:
We broke the market-leading implementation. I believe this
is the first real iSCSI security finding (I’m happy to be
corrected about this). The break is, as I mentioned
earlier, pretty bad: a small piece of exploit code that
works on any vulnerable Filer gives you carte blanche to every
iSCSI LUN, without a password.
iSCSI is a complicated protocol. It combines everything we
love about text key-value negotiation with the rich,
chocolatey goodness of a binary protocol with multiple length
fields that can conflict with each other, one of which is
expressed in different units from the rest.
Session management in iSCSI is painful. iSCSI runs over
TCP. TCP goes through some lengths to ensure that all TCP
connections share bandwidth fairly. This struck the iSCSI
designers as a bad idea, so they came up with a session
management system that allows a single iSCSI client to bundle
together multiple TCP connections to claim more bandwidth. For
some reason, which may or may not be defensible, this requires
multiple session-ID fields and a state machine. Is it possible
for an iSCSI attacker to “hijack” a preexisting iSCSI session
by playing games with these fields? I can’t say.
iSCSI creates new, interesting vectors for attackers to
exploit. For example: at some point, an iSCSI-wrapped SCSI
CDB becomes an unwrapped SCSI CDB, where it is sent to
SCSI firmware. In order words, attackers now have a viable
avenue to attack SCSI firmware directly (before, to get to
this point, you already had to be at the game-over point of
owning the SCSI firmware’s host OS kernel). To put this in
perspective: the errata list for Adaptec SCSI firmware was
so scary they refused to publish it to OpenBSD.
iSCSI creates new, interesting vectors for attackers to
exploit. For example: iSCSI asks you to run a filesystem
over the network. This is a subtle, awful difference from
the way NFS and CIFS work: in those protocols, you work
in terms of filenames and the actual underlying filesystem
is transparent, encapsulated in the server. In the SAN
universe, the underlying filesystem metadata is just another
set of storage blocks, flying over the network. Put
differently: has anyone audited the NTFS driver code for
clientside vulnerabilities?
In theory, iSCSI is supposed to be run on backchannel networks, and
over IPSEC. In practice, you can scan networks for TCP:3260 to see how
true this is.
Is this alarmist? Sure. That’s why it’s in a blog post. But —- and I
know I’m a total geek about this —- I am fascinated by the extent to
which things like iSCSI create new, weird security vulnerabilities. To
quote one of my business partners: on internal networks, things are
getting worse before they get horrible.
2 Comments
Thomas Ptacek | October 25th, 2005 | Filed Under: Uncategorized
Beep! Beep! MREEEEE-OWWW MREEEEE-OWWWW! OooooOOOP! OooooP!
This is bad (if
I do say so myself). If you are affected (that is, if you use NetApp
filers for iSCSI), you should patch. Potential exploits for the hole
can be made very user-friendly.
I’ll have more to say on the advisory and on iSCSI in general later.
No Comments
Thomas Ptacek | October 22nd, 2005 | Filed Under: Uncategorized
There is nothing wrong with looking for vulnerabilities in your competitor’s products,
and Neel Mehta has built enough of a rep for himself that he doesn’t need to take “marching orders” from anybody.
Snort is the most popular IDS deployed on the planet, and it is perfectly fair game: researchers are doing the community
a service when they find holes in it.
Quickly, though: I agree with a lot of Bejtlich’s wormability analysis, but the fact that Snort’s code is freely available (to be compiled on a variety of different toolchains on a variety of different operating systems in a variety of different configurations) makes it less wormable, not more. Each variant installation is a different offset to get wrong.
1 Comment
Thomas Ptacek | October 20th, 2005 | Filed Under: Uncategorized
A caveat: these comments are not a reaction to any current vendor I’m
dealing with right now (caveat necessary due to previous two weeks worth of stalling comments)
Five Cheap Tips For Vendors Handling Security Disclosures
Have A Security Liason
Far and away the most important advice I can give:
Have someone on staff who owns vulnerability disclosure
Publish their contact information
Escalate disclosed vulnerabilites to them immediately
Your security contact does not need to be a developer. But it
does need to be someone who at least appears to be outside
the normal support escalation chain. Simply allocating this
role is a very cheap way to make your company seem responsive
to disclosures.
We know your top-tier support staff is excellent, that you’ve
invested lots of time and resources into equipping them to
solve customer problems, and that you’ve built up years of
experience on how to handle critical customer problems. But
shunting security disclosures through standard support
guarantees not just delay, but the strong perception of
delay, and thus a major incentive to write an exploit to
expedite the process.
A good vendor security contact knows what to say and, more
importantly, what not to say to a reporter —- things like,
“SSL means nobody can attack the service”, or “You can
prevent this server vulnerability by deleting the client
binary”. A good vendor security contact creates the perception
that you are taking the disclosure seriously, and that my time
is better spent finding other vulnerabilities than it is in
demonstrating how severe this one is.
Have A Dedicated Security Enhancement/Bug-Fix Process
Your internal ticket to fix blind, pre-authentication remote
code execution vulnerabilities probably looks just like other
tickets, like the ones to fix spelling errors in your help
files. And that’s fine. But there is no reason customers
should be exposed to that piece of information.
Don’t call security fixes “enhancements”. Your customers do
not perceive “closing major vulnerabilities” as
“enhancements” any more than Ford customers perceive the
recalls on exploding Firestone tires to be “special bonus
upgrades”.
This is so simple to fix that it’s amazing everyone doesn’t do
it. Create special bug numbers for security
vulnerabilities. Don’t even call them “bugs”. Call them
“urgent security issues” or “incidents”. For god’s sake, don’t
call them “feature requests”. You don’t have to do anything
more than changing a label to create the perception that
you are on the case.
Confirm Vulnerabilities Rapidly
The longer it takes you to acknowledge a potential
vulnerability exists, the more worth my while it is to write
an exploit. Basic risk/reward:
| | Admit
Now | Admit Later |
| I’m Right | Appear
Responsive | Downplay Exploit |
| I’m Wrong | Appear Responsive, Incur
Sympathy | Feel Better |
I don’t know what kind of a premium you put on “feeling
better”, but I hope it outweighs the cost of discussing the
cost of the vulnerability in the context of how many measured
seconds it took to compromise every installation of the
product across the whole network.
Provide Documentation Liberally
It’s 2005. At this stage in the game, when investigating a
problem in a network service, I have access to:
Pretty much the only secrets you’re keeping are the precise
number and type of swear words in the source code
comments.
However, at some point around the time I got to “bothering to
attach to your program in GDB”, I figured out exactly how easy
it would be to prove my point with an exploit.
Do Not Downplay
Further examples of things not to say to a customer reporting
a security vulnerability:
“Don’t you have some kind of IDS in place that would
detect people trying to exploit this vulnerability?”
“Our product is designed to be deployed BEHIND a
firewall.”
“Because our product is not built on Windows, it is
not vulnerable to buffer overflows.”
“Our new 9.3 version of the software is not vulnerable
to this problem. We’ve been meaning to talk to you
about why you should renew your maintenance contract.”
To be perfectly frank, you are expected to have security
vulnerabilities. People who look for vulnerabilities wind
up in a bad mood when they don’t find them. So if you want
to make yourselves feel better at the expense of vulnerability
researchers, beat the hell out of your software internally
before researchers get their hands on it.
No Comments
Thomas Ptacek | October 20th, 2005 | Filed Under: Uncategorized
I agree
with Lindstrom’s post about liability, at least in principle. Dan Bernstein can’t write perfect
software, and even if he could, it wouldn’t be reasonable to hold every programmer to that standard.
There certainly is software out there developed with neither the prudence nor care that a reasonable engineer would exercise developing critical software. But for whatever it’s worth, as far as I’m concerned, the issue of whether it’s possible to build secure software systems “in the large” with today’s technology is a settled question. It isn’t going to happen. We need to look beyond the software development process for answers.
It’s still important to assure applications, and companies should suffer when they deliver defective applications. I just think we need better ways of letting the market provide this kind of feedback.
No Comments
Thomas Ptacek | October 20th, 2005 | Filed Under: Uncategorized
So you can’t post an advisory on a Thursday or a Friday, and therefore you are all spared a crowing rant about a piece of internal infrastructure.
On the other hand, I do have something I want to get off my chest today, and, as soon as I go off the clock, I plan on unloading. So, stay tuned (or duck).
No Comments
Thomas Ptacek | October 18th, 2005 | Filed Under: Uncategorized
CardSystems sold for assets
to Pay By Touch,
essentially for the value of their mailing lists. After a previous
unsuccessful buyout negotiation, the best the new buyers can say
is that CardSystems assets, if not their business, are “very far from distressed”.
I suppose that means the mailing list addresses were still legible?
A random sampling of customers still return CardSystems’ phone calls?
Is this our first real example of a large company essentially being put out
of business as a result of a security breach?
No Comments
