The Merits Of Threat Modeling
Dave G. | October 2nd, 2007 | Filed Under: Industry Punditry
As a consultant, I have been involved with many-a threat modeling exercise. Oftentimes, they are boring, process intensive sessions where you stare out the window praying that the meeting ends or that the lunch you ate contained botulism. They are also boring, process intensive meetings that have more impact on the longterm security of your organization than just about anything you are likely to do.
The reason for this is simple:
Most developers don’t give a damn about security. Threat modeling is a great way to get everyone on the same page. It is great when dev teams actually start talking to each other about security. Not everyone will think like an attacker, and that’s ok. As a matter of fact, I think having the security police in the room is potentially less valuable early on in the process. Let the people who produce the app think through this stuff. They are smart. And besides, if you provide the structure and have an approval process, you can always take a look at the TM documentation they produce.
Pen-testing re-inforces the wrong message [1]. Sure it’s a necessary part of the process [2], but just performing penetration testing trains developers to think about security they way they think about other code quality issues. Which is to say, “My code is perfectly secure until someone reports a vulnerability in it, at which point I will fix it and my code will be secure again.” [3].
Prioritize your teams. It helps focus the valuable time of your external security people to the right subject and prevents audit RFPs that ask you to look for buffer overflows in systems where the authentication, authorization and role system is broken by design, but the code is written in C#. —FX added [see comments].
Why am I talking about this? Adam Shostack blogged about Microsoft’s experiences rolling out the threat modeling process. Obviously, one thing that makes this work there is that they have management buy-in. But the point is, It Works There. I hope Adam is in a situation to be candid so that other we can all learn what to expect when trying to insert security into the development process.
(1) Don’t worry, I’ll defend penetration testing in the next sentence.
(2) Never doubt me.
(3) Yes, I am simplifying and generalizing.
FX
October 2nd, 2007 8:43 amHow about adding:
3. It helps focus the valuable time of your external security people to the right subject and prevents audit RFPs that ask you to look for buffer overflows in systems where the authentication, authorization and role system is broken by design, but the code is written in C#.
Gunnar
October 2nd, 2007 9:42 amI think you are confusing “don’t give a damn about” with “don’t know enough about”; I don’t think developers want to write broken code. In my experience, the good ones will always latch on to perscriptive ways to improve authN, authZ and so on.
The real problem is that the security people and the developers are isolated from each other, and whatever knowledge the developers lack about security, the security lack at least that much wrt development practices - how many security people understand build cycles, software architecture, etc.?
I would not write off developers, in fact I think they are _far_ more likely than security people to create solutions in this space. Look at the track record and how much innovation comes out development there are new frameworks, languages, and arch. styles all the time - for one small concrete example unit testing was unheard of few years back and now it is sine qua non. Contrast this to security where you basically have two workign mechanisms (reference monitor and crypto) and think its fairly obvious where the solutions will come from. So it is all about how to pragmatically engage developers and getting them to innovate and extend on behalf of some (clearly articulated) security goals. Threat models may or may not be the best way to do this.
Adam
October 2nd, 2007 12:35 pmGlad you like the post Dave. It’s the start of a series, and if there’s things I don’t cover, let me know. The goal is to talk about what worked and didn’t, and how I’ve been fixing it.
Nick
October 2nd, 2007 8:28 pmWe(the security guys)often only have one opportunity to review something. Threat modeling gives us an opportunity to identify the big issues that we should address.
The artifacts produced through threat modeling allows us to defend our recommendations (rather than making gut judgement calls without supporting evidence). Also, in many situations, you’ll find that the developers haven’t produced adequate documentation (architecture, data flow), so the development teams find it provides added value.
My team has adopted a process based off the Microsoft model. I’d be interested to know of alternative approaches exist.
Dave G.
October 2nd, 2007 9:28 pmFX:
I agree so much, I am adding it to my post!
Gunnar:
I was mostlyish joking.
FX
October 3rd, 2007 1:06 pmWow! Thanks.
batz
October 5th, 2007 10:41 amI’m glad that someone of Mr. Shostak’s technical background is writing from a broader security perspective. As someone who does code reviews for not just security, but for privacy (as legislated) getting developer buy-in that they are responsible for delivering secure code that ensures there are controls on the data they process - is sometimes an uphill battle.
Threat modelling using stories has been a useful way to express the scenarios we use to do our TRA, however, the broader problem is selling the notion of code quality. When an auditor publishes findings that say there is a lack of protection of the data, can we say we developed an application that provides some assurance that the data is protected?
We can’t fix everything, but we can apprise people of the risk and leave it to them to either do something about it or accept it. A threat model helps security people organize their findings, but my experience has been that interpreting threats in terms of specific risk to business is what helps people “get it”.
Why Dynamic Program Analysis is Superior.. Part Three: Path Coverage « System of Systems
April 9th, 2008 2:55 am[…] have a very narrow view of the code they write themselves. I think Dave G put it best in his Merits of Threat Modeling post to Matasano’s Chargen: “My code is perfectly secure until someone reports a […]
Leave a reply