Finger 79/tcp # Christey/Martin: Evolution of the CWE Pie Chart

Dave G. | July 10th, 2007 | Filed Under: Guests

  Login: cwe                Name: Steve Christey / Bob Martin      
  Directory: /guests/cwe     Shell: /bin/sh
  On since Wed Jul 10 21:55:00 EDT from www.cwe.org
  No Mail.
  Plan:
  ----------------------------------
  Views expressed by guest bloggers not necessarily 
  those held by Matasano Chargen.

Evolution of the CWE Pie Chart

We thought we would provide some clarification on the CWE pie chart that we published, which was adapted and is now making the rounds in the blogosphere.

The 55/45 chart is a simplification of a pie chart that we presented at Black Hat DC this year. We’re not sure who simplified it, although it is consistent with the original chart, which we presented to demonstrate the utility of CWE, not as an authoritative analysis.

The original chart was constructed as follows. Application security vendors provided us with their internal vulnerability descriptions, which we then mapped to CWEs, creating new CWEs as necessary (we did this in the very early days of CVE, too). At the end of this first round, we had CWE mappings for 6 vendors. We then looked at how much overlap there was between the tools. We found that 55% of CWEs weren’t covered at all, 1% were shared by all 6 sources, and the rest were covered by 1 to 5 sources.

We built the CWE mappings based on what the vendors’ documents said they look for, which has certain limitations. For example, CWE supports multiple levels of abstraction, such as about 8 XSS variants and 15+ directory traversal variants; we only mapped a tool to a “variant CWE” if it specifically mentioned that variant. Also, CWE has about 104 categorization nodes (such as “Authentication Issues”); many of these would not be reported by a tool, which would be looking for more specific children. Finally, some tools include checks for issues that are useful for developers but not necessarily inherent weaknesses.

Finally, many CWEs are design-level or business logic issues, which are difficult if not impossible to detect automatically. CWE also included many issues that aren’t applicable to web software. So, it’s not a surprise that tool coverage would be less than 100%, nor should we expect it.

We expect that the adoption of CWEs will help to make communication more precise. However, the community is in the early stages of adoption, and we think it’s reflected in the original pie chart. Still, it demonstrates how CWE could be useful in identifying opportunities for vendors and consumers to widen their vulnerability coverage.

We did not expect our slide to generate such interest. In light of this, we will conduct a more careful analysis and provide more specific information on methodology, assumptions, and constraints of the statistics. We will also give a more qualitative description of the gaps that we encountered.

Thoughts or feedback are welcome at cwe@mitre.org.

Steve Christey, CWE Technical Lead

Bob Martin, CWE Program Manager