OpenBSD’s Amusing Handling Of Remote Kernel Overflow
Thomas Ptacek | March 13th, 2007 | Filed Under: New Findings, Uncategorized
From a tip, this CORE Advisory, an IP6 remote code execution vulnerability in OpenBSD. I wonder if this is considered part of “the default install”. Timeline excerpted in its entirety, emphasis mine:
2007-02-20: First notification sent by Core.
2007-02-20: Acknowledgement of first notification received from the OpenBSD team.
2007-02-21: Core sends draft advisory and proof of concept code that demonstrates remote kernel panic.
2007-02-26: OpenBSD team develops a fix and commits it to the HEAD branch of source tree.
2007-02-26: OpenBSD team communicates that the issue is specific to OpenBSD. OpenBSD no longer uses the term “vulnerability” when referring to bugs that lead to a remote denial of service attack, as opposed to bugs that lead to remote control of vulnerable systems to avoid oversimplifying (“pablumfication”) the use of the term.
2007-02-26: Core email sent to OpenBSD team explaining that Core considers a remote denial of service a security issue and therefore [Core] does use the term “vulnerability” to refer to it and that although remote code execution could not be proved in this specific case, the possibility should not be discarded. Core requests details about the bug and if possible an analysis of why the OpenBSD team may or may not consider the bug exploitable for remote code execution.
2007-02-28: OpenBSD team indicates that the bug results in corruption of mbuf chains and that only IPv6 code uses that mbuf code, there is no user data in the mbuf header fields that become corrupted and it would be surprising to be able to run arbitrary code using a bug so deep in the mbuf code. “The bug simply leads to corruption of the mbuf chain.”
2007-03-05: Core develops proof of concept code that demonstrates remote code execution in the kernel context by exploiting the mbuf overflow.
2007-03-05: OpenBSD team notified of PoC availability.
2007-03-07: OpenBSD team commits fix to OpenBSD 4.0 and 3.9 source tree branches and releases a “reliability fix” notice on the project’s website.
2007-03-08: Core sends final draft advisory to OpenBSD requesting comments and official vendor fix/patch information.
2007-03-09: OpenBSD team changes notice on the project’s website to “security fix” and indicates that Core’s advisory should reflect the requirement of IPv6 connectivity for a successful attack from outside of the local network.
2007-03-12: Advisory updates with fix and workaround information and with IPv6 connectivity comments from OpenBSD team. The “vendors contacted” section of the advisory is adjusted to reflect more accurately the nature of the communications with the OpenBSD team regarding this issue.
2007-03-12: Workaround recommendations revisited. It is not yet conclusive that the “scrub in inet6” directive will prevent exploitation. It effectively stops the bug from triggering according to Core’s tests but OpenBSD’s source code inspection does not provide a clear understanding of why that happens. It could just be that the attack traffic is malformed in some other way that is not meaningful for exploiting the vulnerability (an error in the exploit code rather than an effective workaround?). The “scrub” workaround recommendation is removed from the advisory as precaution.
2007-03-13: Core releases this advisory.
Wondering how many other “reliability fixes” are actually remote kernel overflows? Talk amongst yourselves.
Jon
March 13th, 2007 10:18 pmHere’s a “read in-between the lines” paragraph from the advisory:
OpenBSD systems using default installations are vulnerable because the default pre-compiled kernel binary (GENERIC) has IPv6 enabled and OpenBSD’s firewall does not filter inbound IPv6 packets in its default configuration.
The word “default” is used three times in a very long sentence… Indeed.
Chris Rohlf
March 13th, 2007 10:26 pmI have not tested it, but interestingly enough it can be triggered when on the vulnerable boxes local lan, even if it doesnt have an IPV6 address configured?
From the advisory…
“However, in order to exploit a vulnerable system an attacker needs to be able to inject fragmented IPv6 packets on the target system’s local network. This requires direct physical/logical access to the target’s local network -in which case the attacking system does not need to have
a working IPv6 stack- or the ability to route or tunnel IPv6 packets to the target from a remote network.”
The fact a function pointer sits in the attacker controlled space screams “yes remote code execution is possible!” to me, and the ret2text example to trigger ddb is a great POC.
dimitry
March 14th, 2007 12:56 amAt first it sounded like OpenBSD team did not want to be wrong… but now their banner lists “Only two remote holes in the default install, in more than 10 years!”
Dr. Strangelove
March 14th, 2007 8:36 amOpenBSD was created out of Theo’s desire to “not be wrong”, and the rest of the NetBSD team not being able to stand it before. OpenBSD probably has the absolute most sinister vulnerability disclosure model in the history of OS products. I’m a bit fuzzy on the numbers but IIRC, there have been at least (if not more than) 2 dozen exploitable bugs in OpenBSD that were silently fixed via CVS in the last 10 years. No notification, no ERRATA, no Changelog, OpenBSD team member discovers bug, patches, _never tells anyone_.
They are so concerned with their outwardly viewed security posture and aesthetic image, that they feel the need to hoodwink their customers about the nature and extent of security problems in their product.
Has Theo ever not been a double-talking self-righteous security debutante?
dfc
March 14th, 2007 6:07 pmwhat is “pablumfication”
Thomas Ptacek
March 14th, 2007 6:15 pm“Making a pejoritive term apply to the OpenBSD project”.
Jordan Wiens
March 14th, 2007 9:40 pmNot only that, but it looks like they screwed Core over on the release of the announcement too, not coordinating so the vulnerability announcement could go out with patches. I blogged it a bit here:
http://networkcomputing.com/blog/dailyblog/archives/2007/03/openbsd_remote.html
Matt
March 14th, 2007 11:00 pmOpenBSD has avoided the term “vulnerability” for some time. Distinguishing between a denial of service and a remote exploit is important. “Vulnerability” is too vague.
That being said, I personally consider a ping of death a “vulnerability”.
.:Computer Defense:. » Introducing the Amazing, Wonderful and Completely Magnificent Microsoft the Magician…
March 14th, 2007 11:13 pm[…] Perhaps Microsoft feels that by limiting the assigned MS Advisory Numbers, they can limit the number of vulnerabilities that affect their platforms. If that’s the case we have an explanation for the Windows XP Patch that was released quietly on Patch Tuesday… sans MS Advisory Number and therefore not a patch. The patch is related to a race condition in the memory manager. Now perhaps a race condition isn’t all to serious, perhaps it causes a DoS when it occurs, but that’s still a vulnerability and in this case it’s a vulnerability that causes a blue screen. In the old days (”ping of death”) a blue screen was a serious issue… now it seems that Microsoft is ignoring them as being vulnerabilities. Perhaps Microsoft and OpenBSD got together and discussed what is and is not a vulnerability. I guess Microsoft has made it clear via their conversations with ISC (relayed via the ‘Missing Microsoft Patches‘ list) that they don’t consider a Denial of Service worth patching these days. Is this a cost cutting measure on their part or just pure laziness? In either case I wonder how long until privilege escalation is no longer worth covering and then… maybe we won’t even need to cover remote code execute. Regardless failure to provide patches for public issues is irresponsible and disgust with this process should be expressed by the public. […]
ivan
March 15th, 2007 12:38 amRandom interpretation of security advisories seems to be a popular sport and also it seems opinions are like security bugs: everyone has one and sometimes… more than one
In any case, Theo (OpenBSD’s project leader) was quite clear in his emails about why he thinks a denial of service should not be called a vulnerability: Because in doing so the term vulnerability is rendered irrelevant, if you call both a DoS and a remote exec bug the same users will not be able to differentiate what “really matters” from meaningless vulnerability reports (search for ‘pablum’ for the specific term Theo used).
We disagreed on that aspect and now, after the IPv6 mbuf bug report is done and gone, we continue to disagree. We consider a remote DoS a security issue not only because it has a direct effect on availability but also because a remote DoS can be aconvenient building block for a composite attack (for those lacking in creativity: think DNS).
But disagreement did not prevent them from fixing the bug. They did fix it very quickly but they were not very forthcoming about what was exactly the problem and why they thought it wasn’t exploitable and since they didn’t consider it exploitable they did not seem to care about doing anything more than just committing the fix, issuing a patch and noting the “reliability fix” in the project’s Errata page.
Core considers every bug exploitable unless throughout investigation produces convincing enough technical results to make us think otherwise. Even in that case we remain doubtful because there’s always the possibility of somebody out there that is smarter, more motivated or that has more time to look in to it and that will prove us wrong (look up “humbleness” or “cautiousness” for appropriate terms to describe this viewpoint), so obviously we were not satisfied leaving it at that and Alfredo decided to continue investigating.
For those that are now saying that exploitability was evident and that it should have been obvious: I challenge you to backup that statement based _only_ on the original patch. It wasn’t even easy to figure out why and where the kernel crash was actually being triggered, ddt traces showed so_receive as the culprit and not even in a systematic manner.
If you have not been through the vuln. reporting process: What you can read in the advisory describing the timeline and nature of the communications with the vendor is ABNORMNAL. The typical process is usually MUCH WORSE than that. It’s just that we decided to make the part of the process more transparent than what most vendors and researchers usually do.
@strangelove: Would you care to point to the diffs or CVS commits that show those 2 dozen silently fixed exploitable bugs in OpenBSD? I’m not doubting you or hmm well..maybe I am, I’ll have to think about it, but even if you are right how is that different than what *every other software vendor* has been doing for years?
@jordan: I wouldn’t say that they screwed us over because they did not coordinate release dates with us, a “forced release” -the bug reporter disclosing due to an external event- or a “user release” -the bug reporter disclosing due to his unilateral decision- is not necessarily bad or any more “irresponsible” than a fully “coordinated release” planned for, say December 2009, and completely devoid of any meaningful technical details, disclosure of which can be also conveniently promised for some random future time and afterwards systematically ignored as a follow up item
And…ohh yes..there is PoC in the advisory. why? it seems like the best way to prove beyond any shadow of doubt the the bug is exploitable for code execution, because it is useful to test if a system is vulnerable and because it makes it _a lot_ easier to develop a network IDS signature (or please tell me how to write an IDS signature using _only_ the original diff as your source of information)
Dave G.
March 15th, 2007 10:12 amIvan:
One difference between OpenBSD and everyone else is no one else has “Only 2 remote holes in a default install, in 10 years” on their main webpage. If they silently fixed a remote… then they aren’t telling the truth (big if)
Jordan Wiens
March 15th, 2007 10:47 am@Ivan: I suppose “screwed Core” wasn’t quite the best way to put it. Mainly, it looks like they were “operating outside of the best practices for responsible disclosure” with you guys. There, is that summary pablum enough for everyone?
I realize that most disclosures happen at a much slower rate and with less communication than this one (my final note in my blog entry was that the OpenBSD guys fixed it really quite rapidly, all things considered), but I think we can hold them to a higher standard and expect much more than just a little better than average, much like Dave G. points out.
If OpenBSD wants to be the paragon of security best practices (and hey, they’re probably closer than anybody else), that includes coordinating with bug reporters better, and most definitely includes (as you point out) erroring on the side of caution and treating a remote kernel DoS as a serious security issue, whether or not you call it a vulnerability, and whether or not you can immediately identify it as remote code execution exploitable.
All-in-all, they needed to step up to the plate and handle this one better.
Matasano Chargen » When Did Denial Of Service Attacks Stop Being Vulnerabilities?
March 17th, 2007 2:58 pm[…] Furthermore, there are security implications for machines being shutdown. Ivan hints about it here: […]
James Holt
March 17th, 2007 5:27 pmdfc: “pablumfication” is the act of making something tasteless goo, turning something to mush. It’s used when you over blend something. It’s also used when you talk about over simplifying things, babifying it, so that it no longer hold it’s true meaning but is simple enough that even a child can ingest it. Canadians use the term, a lot in the northern Ontario area.
Thomas Ptacek
March 17th, 2007 6:58 pmApparently, every one of those Canadians is involved with OpenBSD:
http://www.google.com/search?client=safari&rls=en&q=pablumfication&ie=UTF-8&oe=UTF-8
James Holt
March 17th, 2007 9:46 pmNah, though apparently some use pablumification.
Thomas Ptacek
March 17th, 2007 9:51 pmA whole three of them, that I can find:
http://www.google.com/search?client=safari&rls=en&q=pablumification&ie=UTF-8&oe=UTF-8
James Holt
March 17th, 2007 10:17 pmWell, having just gotten off the phone with my father, who lives in Thunder Bay (google it if you dunno), I asked him what a few other terms for babying are. His response: “Well, there’s coddling, pabluming, dumbing down, simplifying, shit, I don’t know, you doing a word search?”
Just because the Internet doesn’t give a detailed readout on a word, doesn’t mean it doesn’t exist. Hell, I am sure there is some Newfie slang (google if in doubt about what that is) you’ll never get to read online.
Thomas Ptacek
March 17th, 2007 10:48 pmNewfie slang = sea shanties and pirate talk.
James Holt
March 17th, 2007 11:48 pmWell, no, it’s largely Gaelic thrown into English in a haphazard manner and English with random letters removed. I doubt you’ve ever read someone respond, “aie dunnae ken,” instead of, “I dunno.”
Matasano Chargen » George Ou Goes All-In On Dave Maynor’s WiFi Findings
March 20th, 2007 9:17 pm[…] Why hasn’t the 2006 presentation been repeated and verified by experts? At every step of the way since Black Hat 2006, new disclosures about Maynor and Ellch’s presentation have left more questions than answers. For example, the most recent authoritative disclosure, Maynor’s “once and for all” presentation at Black Hat, crashed his target instead of taking control over it. Sometimes that’s a relevant distinction, and sometimes it isn’t. […]
Leave a reply