Patch Your (non-DJBDNS) Server Now. Dan Was Right. I Was Wrong.

Thomas Ptacek | July 9th, 2008 | Filed Under: Feature, Uncategorized

Thanks to Rich Mogull, Dino and I just got off the phone with Dan Kaminsky. We know what he’s going to say at Black Hat.

What can we say right now?

  1. Dan’s got the goods. You know that scene near the end of High Fidelity where Jack Black listens to the skate punk’s electroclash demo? Yep. It’s really f’ing good.

  2. This is strong year at Black Hat: Dowd and Lawson in particular have awesome talks linked up. But Dan may have Best of Show here.

  3. If you were running DJBDNS before, you’re safe. If you made fun of me for running DJBDNS: sucker.

  4. Ryan Russell pointed out earlier on our blog that Dan takes a lot of crap for doing so much public research. You can’t be in the public eye for long without taking fire from people who write shellcode instead of Black Hat talks and press releases. Ryan is right: it’s not fair. I don’t know how you can give Dan crap about his work after this.

I think Dan should come clean on this and publish the details. The 30 days he’s given before Black Hat won’t make much of a difference. But his reason for not doing it is at least plausible. And he did the work. So, it’s his call.

I think I owe Chris Eng $100 now.

Viewing 88 Comments

    • ^
    • v
    Wow. That was an epic fuck-up ;)
    • ^
    • v
    PowerDNS, by Bert Huber, is also safe. Bert has also been pushing the source port randomization fixes for years, and deserves credit for that.
    • ^
    • v
    There was a reason that I used DNS as an example of how painful it can be if one encounters design flaws...
    • ^
    • v
    In fairness: the **design flaw** here is ancient and really well documented. I'm going to make a case for this being a political flaw more than a technical one. Again: DJBDNS isn't vulnerable.
    • ^
    • v
    @David LeBlanc

    Is it possible for the average person to comprehend these design flaws without an advanced engineering degree?

    Could someone of as humble a vocation as "security researcher" get their mind around this, or would it require, say a trained engineer such as yourself?
    • ^
    • v
    @Statler and Waldorf

    Nah, I've got it on good authority that if you aren't lost on the pass dns cache poisoning attacks and maybe familiar with the workings of the protocol, recursive lookups, etc. Then I'm sure Dan's coming talk will be suitably enlightening.

    By the by, Tom's initial but retracted post was close to my idea of the problem, and I wasn't all that interested till someone with some insight on the matter, combined with Dan's interview with Rich, disavowed me of my foolhardy assessment of its seriousness.

    just to be clear, the discussions may or may not have had were very vague and I'm happy to wait for the offical word like everyone else.
    • ^
    • v
    this better be substantially different than the old "16bits is not enough" because randomizing the source port per query will likely create several new problems and there are other (in my opinion better) ways to address the 16bit qId space problem.
    Somebody decided that a random source port per query was the best (and only) way to solve this issue and since I don't know what the issue is I can't assess if that is the right solution for me.
    • ^
    • v
    • ^
    • v
    Ok. I have a new version for this issue. I took all my special troll and assclown abilities to predict what Dan is going about.

    Simple theory behind this: It would be more easy(in time) to spoof dns, if we wouldn't need to race against real dns server!

    So what Dan found, is how to make legitimate DNS server ignore victims request, so that attacker could have more time sending spoofed responses.

    Did you think why remedy is to randomize source port? Because, if we know victims source port (dst port we already know it will be 53), we can spoof UDP request to legitimate DNS server and make that legitimate DNS server reject victims request!
    • ^
    • v
    I am looking forward to BlackHat this year :D.

    I hope to meet a few of you guys working for Matasano.
    • ^
    • v
    Ivan: it's significantly more interesting than simply the XID space.
    • ^
    • v
    http://adm.freelsd.net/ADM/ADMID.txt ?
    (this is why people laugh at both you & him)
    • ^
    • v
    Ivan: that link to Vixie talking about XQID? Pure gold. "There's simply no way to avoid downgrade attacks."

    "The easier by far solution is to implement DNSSEC."

    Awesome stuff.
    • ^
    • v
    For the record, I'm a *huge* fan of ADM. I learned a ton of my early stuff from them.
    • ^
    • v
    I remember a paper a few years back that was poisoning Windows client DNS resolvers by noting that if you hit them in the early morning, they were likely to be using the first 10 or so source ports (1024-1033) and they were using a constant or otherwise predictable query ID. The attack was remarkably simple: spray responses at all the clients on your target network, with those ten or so target ports with 1-5 different DNS query IDs.

    I'm sure it's not that simple here, but I'd put a small wager on a bad random number generation technique for those IDs whereby knowing a set of responses from the target DNS server shrinks the keyspace massively...
    • ^
    • v
    @Tom

    ok, ok, significantly more interesting than QID space, I'll buy that for now (since now in addition to Dan, you and Dino say so too) but the question that I still don't have an answer for is:
    Is randomizing source port the right (and only) way to address this new significantly more interesting thing?
    What about implementing the XQID proposal (or a variation of it)? Would that address the problem?

    The reason I want to know this is because deploying the current patch may have several side effects with real security implications and I want to understand if I should do it and assume its cost because it the best technical solution or if I am been forced to do it due to somebody's political leverage or preference. I do not think these are unfair questions to ask. We could avoid all this if we had more details but alas we don't...Also, I know of at least one OS software vendor with subject matter expertise that is in the same situation whose only choice is to fall in line and implement the same solution or be perceived as not being diligent in addressing the issue.
    oh btw, the XQID thing did address downgrade attacks it simply wasnt immediately evident to the reader.
    • ^
    • v
    @Ivan

    Why do something simple and non-invasive like XQID when you can do something massively pointless and overengineered like DNSSEC?
    • ^
    • v
    @Ivan,

    Lets talk at BH about XQID.

    16 of us got in a room, and there was not a whiff of politics. Lots of -- wow, we screwed up -- but no politics.
    • ^
    • v
    @Dan: ok, lets do that then. Final question from my part: did the 16 of you analyze the impact for end user organization of the fix that you ended up implementing and decided there wasn't a better way to address the issue?
    • ^
    • v
    @Ivan,

    Oh yes.
    • ^
    • v
    Ivan, the powers-that-be know whats best for you. More patch, less think.
    • ^
    • v
    So, I'm still waiting for the "This Old Vulnerability" write up tqbf is working on to get posted in a few weeks. In the meantime, "more patch, less think" makes Statler a dull boy.
    • ^
    • v
    Dan, I think you may still be missing a valid point here - people are being asked to install a patch that has a genuine impact on how their network infrastructure functions, without being given the information to decide for themselves whether then fix being employed is actually the best one available. Sure, Vixie and crew say its the only one, but they also weren't interested in port randomization 7 years ago when DJB was calling for it. Maybe there's another way that a 17th person in that room would have come up with that would have been brilliant. Or 18th. Etc.

    For the record, I believe you have a legitimate exploit. At issue here is that we're creeping back to a mid-90's security mindset that says patch because we say so. The way DNS is deployed for some people is changing in a major way - and they aren't being told why.
    • ^
    • v
    Why must you feed Dan (Bernstein)'s extremely large head by admitting his software is superior.
    • ^
    • v
    Jeremy--

    I fully expect better ideas after August 6th. If I was going out with a huge bug, and no particular date in mind, or a date way off in the future -- alright. But there's a population that would like advance notice, and there's a population that ain't doing anything without technical details. If nothing else, both populations are getting very close to what they want.

    I'm totally open to a debate, after my talk, about whether this was the right path to take. But so far, "every vendor in the world patching on the same day and not whining about it" is about as distant from a Mid-90's security mindset as you can get.
    • ^
    • v
    bubba--

    Because he was right.
    • ^
    • v
    bubba you can't be saved unless you accept him as your lord + savior. You should be asking yourself "What would DJB do?" any time you make a decision in life - security or not.
    • ^
    • v
    Patches being available and people feeling comfortable applying them are two different things. Now people may have to alter how they deploy DNS without understanding why.

    Corporate security folks out there - how are you approaching actually applying this patch? Whats your timeframe? How reticent are you to apply this without better understanding of why its changing assumptions you may have made about how DNS is deployed in your company? Its entirely possible I'm wrong, and I misinterpret how our clients would react to patching a critical part of their infrastructure based on limited information. And changing their firewall rules too.

    (Wouldn't it be nice if there was a way to apply a rule like this in a single place, and have it pushed out to all your firewalls simultaneously? And roll it back if all hell breaks loose? matasanoplaybook)

    It's also worth pointing out that I don't really have any specific DNS or anti-Dan agenda here. I just feel very strongly about fostering and encouraging open security discussion, as I think its improved things immensely. Had I not been asked NOT to discuss this stuff, I probably would have continued my blog coma.
    • ^
    • v
    Jeremy--

    So, you think I should have said nothing until Black Hat, and gotten the vendors to wait until then too?

    Because, you know, that was an option. Would your clients have preferred that?
    • ^
    • v
    Jeremy,

    I use OpenBSD for DNS servers in my company, and the fact that they have chosen NOT to use the BIND solution, and instead implement their own, tells me that the ISC solution is sub-optimal.

    Since the limited information available infers that random source ports mitigate the issue, I've enabled NATing on the external interfaces of my recursive DNS servers, since pf uses random per-connection source ports for NATing. If this avenue wasn't available, I would have migrated the company to djbdns (and still may).

    As a corporate network and security guy, rather than a researcher, I frequently feel insulted by vulnerability releases like this. Apparently, researchers and vendors assume that everyone else to too dumb to make informed choices, or apply their own mitigation strategies. I have no problem with big behind-the-scenes coordination for SPs (like the TCP reset bug, where ISPs got the fix first), but after the initial wave, I find little value in waiting to release enough detail for admins to make their own decisions.

    I expect the other talks at Black Hat will be spacious as everyone crams into Dan's talk to see what the big fuss is about.
    • ^
    • v
    At issue isn't that there's a patch out. Its that you spoke publicly about it having a flaw, gave out limited details, and then asked the rest of us not to discuss it. As is, that was enough to totally derail everyone discussing the flaw, and instead get spun up on an ethics issue.

    When patch Tuesday comes around, people look at whats been patched. If you hadn't tried to stifle useful discussion, maybe a Tom would have done enough research on his own to gain a better understanding of the issue at hand. In turn, the corporate security + IT types could get a cogent discussion of the issue laid out for them, and decide for themselves how to prioritize patching, or apply a different mitigation strategy - because like Steve above, they're intelligent enough to make their own decisions.

    I'm not saying full disclosure is the only way - just that thoughtful technical discourse on security issues is key to making the world a safer place. I think we'll just have to agree to disagree on whether or not that's the right attitude to have, in this case - but I promise you, whatever you release at Black Hat will not make me any less upset that you succeeded in sidetracking the Tom's (and Ivan's, and Steve's) of the world from discussing the technical issue.
    • ^
    • v
    After this all blows over, I'd be interested to see the numbers for how many have already applied the patch versus how many haven't, and again after Dan's talk at Black Hat.

    Asking people to accept Dan's (and Tom and Dino's) words on blind faith is asking a lot. I think the position a lot of people are taking is "details first, patch later" as opposed to the "patch first, ask questions later" cowboy mentality.
    • ^
    • v
    • ^
    • v
    "We know what he’s going to say at Black Hat."

    My take on what Dan is going to say: he is going to show that when the attacker and target DNS client are on a GbE network (think of a company network), and when the source port of DNS requests is known and constant, the attacker can bruteforce the 16-bit TXID in 34ms average or 67ms seconds worst case (assuming he sends 128-byte DNS replies -- (128*65536*8)/1e9 -- packets can be made ever shorter by removing unnecessary fields, probably 60-90 bytes).

    Which means that DNS spoofing is DAMN EASY and it has always been the case for quite a while (basically since GbE became widespread). Most people just don't know it takes only 67ms. I think this is what shocked Thomas :-)

    Then Dan is going to explain that randomizing the source port adds ~32-64k (depending on your UDP settings) values to bruteforce, transforming the attacker's pb to a 31-32 bruteforcing pb (making it impractical again).

    Am I right, or am I right ? :-)
    • ^
    • v
    Not enough GbE adoption to make vendors jump. Dan's would have to be a very broad and widely usable approach.

    Plus Tom probably whiteboarded that Tuesday morning in his jammies.
    • ^
    • v
    @Steve: OpenBSD was not giving the details of the problem which is quite interesting considering that they were the first ones to tackle this issue in 1996, did not consider using random source ports as a good option back then and have been proposing alternative solutions for years and also considering that the current BIND patch uses their QID randomization technique.
    Also, the patched named (9.5.0-P1) uses the entire pool of non-privileged ports for its randomization purpose without an option for users to configure it, you can verify this by reading get_randomport() in lib/dns/dispatch.c. that is what OpenBSD developers may consider an ugly hack developed in haste. Your NAT approach may not work entirely well because unless pf understands the protocol there is no easy way (except a timeout) to remove entries from the NAT state table. I'm interested to hear more about how that option is working out for you.
    • ^
    • v
    sorry I meant OpenBSD was not given (not "giving") the details
    • ^
    • v
    Ivan, any clue if they were invited to be part of the DNS patch cabal, and decided not to participate?
    • ^
    • v
    Uhg and I forgot to mention that ISS X-Force has rightly pointed out that firewalls and NAT devices that do not pick their NAT ports randomly will render the named patch useless if they are sitting between your name servers and the big bad internecs but I am sure and Dan's 16 had figured this out already and will provide good guidance on what to do about it at BlackHat
    • ^
    • v
    Statler and Waldorf: to make bruteforcing practical on 100Mbps networks, the attacker needs to cause the latency of the DNS server to jump to x10 the values I mentioned (so 340ms or 670ms for a 50% or 100% chance of success). Example: (a) small company network with 1 or 2 DSL/cable links to the Internet, (b) attacker launches temporary aggressive download (not too aggressive else the latency easily jumps to 1000-2000ms, (c) and bam! the attack can be pulled off.
    • ^
    • v
    Sooooo...

    I'm confused here. Maybe I'm just too old school for my own good, but, what happened to the fundamental tenet that EVERYONE in the know use to subscribe to: "more information is good thing"? Remember when smart people would disclose what they knew to the rest of the world so that other smart people could discuss and come up with other viewpoints and angles? Kind of like an "open community" (not my term).

    I like and respect Dan. Back In The Day when I was cool, we did some dirt together. But I'm just not convinced this is the right way to go about this. Amiwrong or does it kind of feel like of media exploitation? I mean, honestly, if we are going to jump ship to the other camp, wouldn't the Internet and the World be safer if there was no disclosure at all? I don't think so, but I do think the disclosure and discussion needs to come at the time of remediation so that organizations are free to determine what is the best path to a solution and they are free to weigh the second order effects from patching vs. not patching vs. .
    • ^
    • v
    ivan,

    I'm probably not the best test case, since my DNS servers only serve my enterprise of ~2000 systems, performing about 5-10 queries per second. At that level, the NAT rule seems to be averaging about 100 - 120 states at any given time, and most are less than a minute old. For my purposes, it seems to be accomplishing the goal of source port randomization, anyway.
    • ^
    • v
    I agree with the criticism of how Dan is handling the disclosure. Even after talking to him and hearing his story, I still believe a good part of this is being played for his PR advantage.

    But having said that, he's still going to get on stage in a couple weeks with a really, really nice finding. So I'm not going to go toe to toe with him on this, because I'll lose. It's not the best handling of disclosure ever, but it's really good research work.
    • ^
    • v
    FWIW I have to agree with Jeremy and Mike S. on this disclosure process and the general handling of the bug. I have no doubt the bug is real and severe, especially now that Dan has brought others into the circle (Thomas and Dino) and they have confirmed it.

    However a lot of people are running around right now trying to get a complete understanding of this bug and how it impacts them, and in addition to that how the fixes impact their current deployments. And at this point I don't believe they have enough information to fully answer those questions.
    • ^
    • v
    Mike, Tom and Chris:
    Can you elaborate on what you would like to see done differently?

    Dan says, "I have a patch for a serious vulnerability. If you believe me, install it. If you don't believe me, which is perfectly understandable, give those who do believe me just 30 days to get a chance to brace themselves and I will explain everything you want to know."

    What is wrong with this procedure? Would the right approach be to feed the black hats the knowledge while the whole world is still vulnerable? Would you rather see the affected systems breached than wait 30 days?

    It seems to me you are criticizing, but not in fact offering any alternatives.
    • ^
    • v
    Anders, not releasing details himself is OK with me. Asking others not to discuss is not. That's what's wrong with the way this was handled.

    If the goal is to prevent systems from being compromised, and it really isn't possible to reverse engineer this flaw from the patches, what's accomplished by talking about it at Blackhat? Do you think 30 days is enough time to patch and secure the entire internet?
    • ^
    • v
    @Anders: The problem here is that I do not have enough details to determine what is the best course of action: should I deploy the patch and invest a substantial amount of effort in re-configuring my network security to properly handle random source ports in each DNS query that is made or should I just take my chances not install the patch and accept the risk of having the cache of my DNS servers poisoned? Are those the only two possible options? If I cannot come to an answer those two questions by myself then I've not been given any extra help to reduce my risk, just uncertainty (and the chance to reduce that uncertainty by trusting 16 people that already decided what was best for me). No problem, I can deal with that for a couple of weeks until BH but you wont convince me that I should be grateful or happy about it.
    On top of that, this is not a run-of-the-mill SQL injecting bug in some barely known web application, it is related to a DNS design flaw whose solution has been surrounded by controversy and political bias for over a decade. It was likely to stir more controversy that normal if the details and the mechanics of how it was reported, discussed and disclosed were not entirely clear.
    • ^
    • v
    Chris:
    I have no idea, but yes - with a patch backed by leading vendors in the industry, I do imagine 30 days is sufficient to secure the most critical systems on the Internet.

    The purpose of talking about it at Black Hat is exactly the reason that you and/or others have given when asking Dan to disclose full details: peer review and disseminating the specifics about this vulnerability to system administrators so even the most skeptical ones have a chance to assess its severity and determine whether they should patch or not.

    With regards to not discussing, I assume you refer to the last bit of Dan's initial blog post. First of all, I read that as a very humble (he admitted it was unusual and maybe unreasonable in the post itself) request, rooted in the same (reasonable) safety concerns as described above - not a command or instruction as some people is trying to portray it (Marcin calls it 'cowboy mentality').

    Again, I see Dan saying: "If you can trust me, Microsoft, Cisco, ISC and CERT for 30 days, I recommend the suggested response. If you don't, do what you must, but don't say I didn't warn you."

    With that said, can you elaborate on what is wrong with this request? What could have been done if Dan had not made this request?
    • ^
    • v
    Ivan:
    I appreciate your situation, but that argument is flawed. You are putting Dan at responsibility for circumstances that are out of his hands. Dan Kaminsky can not both disclose and not-disclose. The complaint for that situation you must lodge with God or Aristotle or general relativity or whatever. Meanwhile, Dan Kaminsky only has two options, and he chose the one that will leave the most critical system unsusceptible to attack.

    Once more, enough with the finger pointing, please. Offer specific alternatives - because then it will be clear that in practice, there are none, and Dan has handled things quite well.
    • ^
    • v
    Anders:

    Isn't that a false choice? This isn't about trusting Dan, Microsoft, Cisco, ISC and CERT as to whether or not this is a serious vulnerability or not. I think the decision is "trust me that you would agree with me that the risk is worth the costs of patching immediately... and take my word for it, there are no other possible mitigations that your organization would find acceptable".

    Also, the idea that no one else could have found this bug is naive in 2008. Even if they didn't already have it, once people know something exists, they will hunt it down. We have seen this before with WabiSabiLabi.

    The knowledge then becomes asymmetric. Attackers are more adept, more motivated and have more time to figure it out than an F500 security admin. There is a really good argument that you can level the playing field by giving people more information and let people make better decisions and come up with potentially better mitigation strategies.

    For example, maybe detecting that it is happening is sufficient for some organizations. Maybe they aren't in a position to patch right now.
    • ^
    • v
    Dave G, others:

    I believe that once the seriousness of this issue comes to light, it will be clear why Dan and others chose this path. The effort being extended here to protect the "worldwide community" from the brunt of this issue by getting these patches out in a coordinated fashion should be commended.

    Yes the patches may change some of the assumptions some have made in their environments when using naive stateless firewalls or ACLs, but that is a small price to pay for the valiant attempt at keeping all of us connected to our email, entertainment, security news and blogs we all crave. Not to mention trying to keep the b' herders at bay.

    Yes, I believe this is that fundamental of an issue, and the response has been an appropriate step.

    Believe me, I know.
    • ^
    • v
    @Anders: please show *proof* that they chose the option that will leave the most critical system not susceptible to attack or go back to your philosophical corner. This is not a dogmatic, religious or ethical argument it is a pragmatic, rational debate.
    I am not attributing responsibility or pointing fingers at Dan or the other 15 people I am just saying that they've given me just two options and scarcely more information than what I already had to let me find a solution of my own or to assess the cost/benefit ratio of adopting of each of the options I've been given.
    I can't offer you specific alternatives because I don't know what really is the problem.
    CAN'T YOU UNDERSTAND THAT?

    Btw, I mentioned XQID and so far I have not been told that it wouldn't work (nor that it would).

    I am sure there are more than 16 DNS security experts in the world (I am not claiming to be one of them) and if more of them analyzed the problem maybe more alternatives could be found.

    Besides, I don't like to be told what is best for me I like to come to my own conclusions and I feel that I've been temporarily denied of that possibility in this matter... but that's just me, you are free to do whatever you want with your DNS servers right? I may decide not to patch mine and simply hope that nobody will poison my server's cache to make your domain point to some kiddie porn site, let's just hope that you're not servicing critical security updates, financial or otherwise privacy sensitive services on your domain.
    Btw, I am sure that P2P software developers and botnet operators will love your patched BIND.
    • ^
    • v
    Dave:
    Yes, "maybe" it is. "Maybe" it isn't. Again, recommend a specific alternative to Dan's approach. Then, and only then, can we compare pros and cons. Anything else is idle speculation.
    • ^
    • v
    @Steve: I think that each NAT states is pinned to a particular remote nameserver's IP so you are still open to attack. For any given domain that you query this is equivalent to the internal named picking a random port at start up time. Unless pf closed the NAT hole when an ICMP unreachable for that port is seen coming from the inside the hole can be kept open forever.
    • ^
    • v
    Ivan:
    Your tone doesn't make you look very rationally or pragmatically (or any other arbitrary euphemism for that matter) inclined either.

    The point that you have made so far is that you aren't happy about the situation and you have no idea what to do about it. Of what relevance is that to anyone here?
    • ^
    • v
    Anders:

    I think what folks here are suggesting is that after the patches came out, release more information about what the issue is. Enough for people to make an educated decision about how to fix it that makes sense for their environment.

    Brian:

    The co-ordination isn't really what some people are objecting to. It is post co-ordination. The fact is, right now, numerous people know about the details about this. Many don't. People that do can now do whatever it is against the people who don't. Their only option is to patch. Patching may be expensive. There may be another solution that works better for organizations.

    I think we should poll people who actually are responsible for the security of DNS servers and see what they would prefer (disclosure that would allow them to pursue other options even if the entire Internet would know more about the issues, or have to decide on patching or not patching based on the information currently available.
    • ^
    • v
    Anders, I think a major point here is that we can not offer you another solution because we simply are not as informed as we should be. (When I say 'we' I am speaking about the security community as a whole).

    30, 60, 90 days... doesn't matter, it will never be enough time to secure 'the most critical systems' if those system administrators don't know anything about the vulnerability they are patching. Proper disclosure has taught us a valuable lesson (at least I thought it did), that more information is better. The bad guys are going to obtain it anyway, mine as well let the good guys prepare.
    • ^
    • v
    @Anders: I am saying that implementing XQID was a better way to fix this problem. There, I've given you an alternative, now prove me wrong. Dan, Tom, Dino and 15 other people can (but wont before BH, fair enough) you can't.
    • ^
    • v
    my .02 from the point of view of someone who needs to advise their customers on how to handle this issue;

    more details are needed; our telecons with our customers makes us sound incapable of our jobs - we cannot be firm and offer advice that we have confidence on about a resolution to the problem because we dont completly understand the problem; likewise, we cannot have faith in the vendors that say they're not vulnerable because we fear they dont understand the problem either.

    releasing information in this manner is harmful to information security from the business aspect, harmful to the PR image of security researchers, and detrimental to overall security as a whole due to the spreading of misinformation and speculation, imo.
    • ^
    • v
    I personally don't know the merits of XQID enough to know if it would be a fix to this or not, but given how recent some of the discussion was (I see much in April prior to the March Summit..), I doubt that it could have been properly baked and implemented in record time by so many vendors to address this issue.

    The fact is that the issue that was recently patched has been making the rounds for nearly 18 years! It got some good attention with CA-1997-22, and then more later with CA-2001-09 and various adaptations of that, yet it was still never properly fixed. How much time do we need to know of (at least one) key problem and talk about solutions? It's about time that this workaround has been implemented, because frankly, the industry just hasn't come to a consensus fast enough left to it's own devices. (The recent collective effort aside.)

    The s in DNS has never stood for Security, but the proliferation of the web has made it a defacto part of the security infrastructure. We all trust in the address bar of our browsers, the security level we apply to certain domains with Zones and tools like NoScript, the vain attempts to graft in spam protection and the sources we download updated bits of code from, etc.

    Like it or not, the DNS is part of our security infrastructure. It is about time that we as an industry step up the efforts, skip the workarounds and go about solving the problem at the heart of it all.

    This debate I don't see ending soon is great, but is it accomplishing much? Agreed. Some people know the details, some people don't (isn't that part of everyday life?). The hope is that the fewer who know will remain that way until high volume recursing servers are all patched. Those who keep asking for full disclosure will look back on this conversation when the truth comes out and say, "Oh, Ah, yeah, in retrospect, this was one issue where full disclosure was not appropriate."

    I hope that most will take these patches as a gift you would be foolish to refuse! Should you choose to delay, just be prepared for the consequences and resign yourself to franticly deploying them around 3am some morning in August.
    • ^
    • v
    O/T: Nice new theme, but (A) "I need a web app Penetration Test" is not a blog, and thus is a somewhat suspect entry for your *cough* blogroll, and (B) I miss the PunditCon (stationary as it was).
    • ^
    • v
    Dave and Chris:
    I guess I'm just wondering how everyone here and elsewhere calling media exploitation can be so certain that immediate public disclosure would be to their benefit. But I guess we'll know by the 6th.

    Ivan:
    Who cares if you are right or wrong? The question was how to improve the disclosure, not what the best fix for the vulnerability is.
    • ^
    • v
    Guys, cost of fix and/or mitigation - totally agree with most organisational and customer points raised. Trust or be damned? At least in the customers eyes! Because we say so? More infosec Voodoo? Cynical portion over.

    Surely *internal* unicast reverse path forwarding, internally blackholed non-organisational subnets, no internal default route and some *internal* dns features on *internal* FWs may help to mitigate. Even RFC 2827 "Network Ingress Filtering" applied closer to your internal COREs, or some form of host/network IPS?

    The point is, in my reading, the attack must exploit a trusted internal host.. almost trivially, think browser, to generate valid traffic requests and capitalise somewhat on subsequent transitive trust.

    Sometimes a network level solution can temporarily mitigate and create time/support to address a more costly patching excercise, or even allow for surveillance and incident response to occur at least...
    • ^
    • v
    @Anders: ok sorry, it seems we've been talking about different things then.
    I am interested in figuring out what is the most effective and less expensive solution to this problem for vulnerable organizations so I can provide guidance to them when asked. I am not interested in using Dan Kaminsky's finding and the process he and the other 15 people to disclose it as a proxy for a debate on how to improve the vulnerability disclosure process.
    • ^
    • v
    Ivan:
    If you have a suggestion that will make the Internet a more secure place before the 6th, please, put it forward. If not - please quit addressing me.
    • ^
    • v
    Findings-v1: bb821e88d15c198951660424cd9692736356b253290ac3cad5a2cbbb87ab5a4e

    Findings-v2:
    0fca0b42a9795b35f12c1d7fdb7abcd6a85e1f7810ede46fb59b8d318487da36

    (SHA256 - Just for posterity ?)
    • ^
    • v
    Guys, I have been put in a very sensitive situation because of this. How do you expect me (a non-DNS nor DNS security expert) to go to our CTO (who helped draft the RFC for DNS) and tell him we need to patch, but there are no details available? Heresy!
    • ^
    • v
    I hope the internet doesn't break. I'm not good at anything else..
    • ^
    • v
    @anonymous

    Still reading RFCs to see if I can guess who your boss is. In the meantime, why not having him call up Vix for the 411 on the DL?

    Lots and lots of patchii come out with vague info. This one has more press and affects something critical, but do you really delay installing MS or Cisco patches unless you have working exploits, srsly?
    • ^
    • v
    Probably not the right forum for this, but does anyone understand the logic in why not all vendors are rating this Critical? Microsoft for example only lists the patch as Important. This simple point can make it harder for enterprise admins to get the support and resources they need behind changes out of regular patch cycles.

    I realize that these vulnerabilities are not direct code execution issues, but they chain so darn well.

    With the understanding that full details will be released in little more than 3 weeks, one would think more emphasis on the criticality, the better.
    • ^
    • v
    Because "Critical" vulnerabilities are worm-able. Vulnerabilities that are not themselves critical, but can be "added up" to critical, are almost the definition of "Important".

    This vulnerability is still less important than the RSA signature forgery issue, which wasn't critical either.
    • ^
    • v
    Dan stuck between a rock and hard place? So are we. At least the suspense is exciting. :P
    • ^
    • v
    Is Kaminsky going for the Pwnie of Most Overhyped Bug?
    • ^
    • v
    I see on daily-dave there are people who goes along with my theory.
    So if they say ICMP, I say pure UDP and fragmentation.
    • ^
    • v
    My opinion on the Important v. Critical thing is that severity is only linked to impact, not distribution. For example, it's a Critical in WINS, even though you probably aren't running it on any particular server. This is an Important in everything, but it's still an Important.
    • ^
    • v
    Lots of people saying [paraphrasing] "how can we know our risk and prioritize the patch if we have no details" and, stuff may break. I agree.

    Dan rightly indicates that it takes a lot of meetings, time, and money to get these things done in an enterprise space (I work in a shop with 90,000+ endpoints). Again, lots of stuff may break.

    The advantage of enterprise, though? I have a huge test lab.

    What I'm doing? Patching EVERYTHING in the lab now, so if stuff is going to break it'll break now. It also gives me some burn-in/testing time of my own. As soon as the BH presentation hits the fan (so to speak) and the press starts to turn into pressure to act, I can pull the trigger WITH full information.

    ISC says that the "beta releases include optimized code that will reduce the impact in performace".

    What I would like to know is when the when the beta code is expected to become GA? Anyone clear on ISC's release practices?

    Dan / Paul Vixie- any insight here?
    • ^
    • v
    rGod,

    We're actively trying to get you what you need. Seriously though, you really want to pull the trigger before my talk. Really you want to pull it as soon as possible. Yes, I've bought you some testing time...but don't treat it like a snooze button :)

    How's the perf impact hitting you right now? Most name servers are not run to the edge of capability. I'll look into the GA date.
    • ^
    • v
    Performance is hard to judge in the lab. From looking at pure packet in/out stats in production, though, at peak times we see just a little over 5000. I'd rather patch once with optimized code rather than revisit this - and this is a question my management should ask about.

    I assure you, I'm not treating the time like a snooze button. It's really hard to justify some of the risk ($$ loss if something breaks in production) based on the facts in evidence today. I can try to get everything patched, but it's ultimately the business who makes the decision based on the risk assessment I can show them.

    Right now, I can show them vendors giving an Important (not critical) risk rating, and no details (much conjecture) on how we MIGHT be vulnerable and what our ASSUMPTIONS are regarding the risk to our business.

    Like it or not, from a non-technical business standpoint, that's hardly compelling.

    I WANT to patch before your presentation, but I NEED to be ready for when (not necessarily in order of likelihood):
    a) the business gets scares by all the press
    or b) I can show the security risks are high enough to assume some business risk

    This is helped by having everything up and running in the lab as long as possible before either of these things occur (due diligence). It's also helped along by making sure the various security teams (CSIRT, VulnMgmt team, etc.) are on side, and everything is in place so major apps can ensure zero impact.
    • ^
    • v
    Why don't you ping me privately?
    • ^
    • v
    • ^
    • v
    It isn't the fault of djbdns, but running dnscache behind a typical ADSL router still appears to be unsafe.

    I get a "POOR" security if I do the porttest:

    dig porttest.dns-oarc.net in txt
    • ^
    • v
    How do you use that port test?
    localhost:~ Michael$ dig porttest.dns-oarc.net

    ; <> DiG 9.4.1-P1 <> porttest.dns-oarc.net
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 30368
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;porttest.dns-oarc.net. IN A

    ;; Query time: 2323 msec
    ;; SERVER: 68.105.28.12#53(68.105.28.12)
    ;; WHEN: Wed Jul 16 13:10:51 2008
    ;; MSG SIZE rcvd: 39


    localhost:~ Michael$ nslookup porttest.dns-oarc.net.
    ;; Got SERVFAIL reply from 127.0.0.1, trying next server
    ;; Got SERVFAIL reply from 4.2.2.2, trying next server
    ;; Got SERVFAIL reply from 127.0.0.1, trying next server
    ;; Got SERVFAIL reply from 68.105.28.12, trying next server
    Server: 4.2.2.2
    Address: 4.2.2.2#53

    ** server can't find porttest.dns-oarc.net: SERVFAIL
    • ^
    • v
    dig @4.2.2.1 porttest.dns-oarc.net in txt

    try that?
    • ^
    • v
    Incidentally, theReformed was running a poll on Dan’s find posted the other day, so far Dan seems to be coming out on top with the people http://thereformed.org/2008/07/19/poll-dan-kami... .
    • ^
    • v
    Disclaimer: I am a confessed DJB fanboy.

    It's a shame DJB's (admittedly abrasive) demeanor has so often distracted so many from the quality of his work. I've deployed djbdns exclusively since forever, and I have never been disappointed with its performance and simplicity, particularly over conflated bloated pigware like BIND (imho). The simplicity of the zone file format alone is delightful. But I digress.

    Imagine my delight to learn of this massive urgency and then conclude that I need to do NOTHING about it for the security of my networks (all systems here are showing GOOD on the TXT test).

    To discover that DJB was right on this issue doesn't surprise me in the least. At the same time, to discover some personal distaste for him remaining at large doesn't surprise me much either.

    Good on him, I say. And shame on so many others for not listening years ago.
    • ^
    • v
    any reason why a couple of minutes ago, on my google reader feeds i got a feed/post named: "Reliable DNS Forgery in 2008: Kaminsky’s Discovery" written by ecopeland that starts with the catching phrase "The cat is out of the bag" and then go out and tell us that what Halvar Flake wrote on his blog today is the "Real Thing" and then when i go to this post in matasano´s blog i get a 404?

    the link that i got in my reader is/was: http://www.matasano.com/log/1103/reliable-dns-f...

    does it means that what was written in the post is *not* the actual "Real Thing" and was "recalled" by the author? or does it means that it *is* the "Real Thing" and was recalled by the author whatever reason it may be?

    just curious...
    • ^
    • v
    http://blog.invisibledenizen.org/2008/07/kamins...

    No one's confirming, but it sure does smell like the real deal.
    • ^
    • v
    Fail. It's all about creating covert channels. This is known since at least 1990 when Steve Bellovin found flaws in the DNS design. But Dan found a way to do it effeciently since 2004. and hence the reason why DNSSEC came to be in 1999.

    Why don't you just wait for Dan's talk instead of blowing your trumpets and claiming his thunder. Dan deserves this more than anyone I know given his research on DNS over the years.

    Anyway, that's my take on it. Put it to rest and wait for Dan's talk.

Trackbacks

close Reblog this comment
Powered by Disqus · Learn more
blog comments powered by Disqus