The path to PenTestConsole

Mike Tracy | March 27th, 2008 | Filed Under: Uncategorized

I recently gave a talk at OWASP Chicago called “Web app penetration testing with scripting languages”. Based on feedback since the talk (and dancing between the raindrops of a pretty hectic work schedule), I decided to try a series of blog posts cementing the ideas surrounding the development of PenTestConsole. To Mike, Mike, Al, Walter and the other 5 people I can’t remember right now, your tarball will be in the mail soon… I hope.

What is PenTestConsole?

In simplest terms, PenTestConsole is a set of Ruby libraries (and add-ons to existing libraries) that provide a scripting framework for web application testing. It allows you to work from an irb (Ruby shell) prompt or write scripts to automate tasks that would otherwise consume precious engagement time.

And Now… Some Context

Coming into this business from a QA background, I realize that testing is testing. In the QA world, positive testing is simply testing to show the product works (test to pass). Negative testing shows the product is broken (test to fail). One of my big lessons learned over the years is that you can’t positively test something unless you know what it’s supposed to do. Proving a negative is easy… it’s proving a positive that’s the hard part. These concepts hold equally true for application penetration testing and even though line between “positive” and “negative” blurs to a disturbing gray, the necessary piece of the puzzle is business context.

The problems that plague QA guys are a subset of the problems that hinder application pen testers. Not enough information, too little time and getting started in the wrong part of the development cycle just to name a few. The problems we face as security consultants are compounded by the breadth of expertise required to work any engagement that comes our way. As much as I would love to spend time pontificating about methodology, technology and software development practices (Niaaaagra Falls), I have work to do and that’s the reason for PenTestConsole.

By the time I get to an application it’s either deep into QA cycles or actually in production and I’m testing the production instance in an off-hours gig where I’m hamstrung by limits on what data I can enter into the system or how destructive I can be on a shift schedule that is grounds for divorce in 32 states (Slooooowly I turn). The key thing is that I am looking at an application that fills some business gap for my customer. Nirvana comes when I can use business justification to inform my assessment of security posture. I need to be able to find all the bread and butter issues (XSS check, session problems check, SQL|XML|XPath|Command|… injection check, CSRF check, error handling check…) and find the more complex and subtle (read fun) vulnerabilities that the customer needs to know about.

There are lots of tools out there to help me ranging from from proxies to fuzzers to scanners. Every one I have used has solved some particular problem during an engagement. My big challenge is that the more advanced or subtle functionality is not as accessible or obvious as the basic stuff. I need to spend cycles figuring out how the application under test works rather than ferreting out how my tool works. Even when I do figure it out, I keep running into situations where I need to code something specific because the app uses some new authentication mechanism or postbacks or includes HaveAFunTimeReadingMe.asp or you_need_to_be_on_meth_to_read_this.js (Step by step). Doing this over and over made me decide to write a framework that makes this job faster and less frustrating.

Coming next are the whys and hows of PenTestConsole’s development as well as some examples of how it works. Stay tuned.

24 Comments

Requirements for Injectable Virtual Machines

Wes Brown | March 14th, 2008 | Filed Under: Development

Much of the last two or three years of my research time has been devoted to the concept of injectable virtual machines and using them in a penetration testing context.  I’ve found that once created, they tend to be handy tools that solve other problems too outside of the original intended design goals, such as secure code deployment using lightweight portable virtual machines.

Requirements design can be an iterative process, and after working on several implementations of the injectable virtual machine, the following requirements were codified:

Compact.  This is the biggest requirement: that the implementation be compact.  How compact you need it to be depends entirely on what you want to do with it.  But in general, the smaller the footprint, the more you can do with it.  In the past, I’ve targeted 64K-128K of payload size, and considered that pretty good.  But the smaller the implementor makes it, the more likely it is to be useful in more contexts such as first stage injection.  This also extends to the libraries and bytecode that is generated for the virtual machine, as the virtual machine by itself is useless without logic to drive it.  This is why I’m targeting a much smaller payload size in my next implementation, of a few kilobytes.

Portability.  This is another huge design goal that I think many people who operate in this space miss out on.  The implementation of the virtual machine should be portable between platforms.  Behaviors should be as similar as possible and abstracted out so as not to distract the programmer-user from the task at hand.  A task such as opening a socket on a Windows machine should work on a Unix machine as well.  When we have multiple implementations of a virtual machine, we run into the risk of different behaviors, defeating the entire purpose of a portable bytecode virtual machine.

Dynamic.  The virtual machine should be dynamic.  We should be able to modify the behavior of the program and language while executing.  By being self-modifying, the language and core functionality can start out very small, but be extended upon.  Features that are not needed early in the process such as garbage collection can be added.  Squeeze a payload in, and watch it expand, much like a ship in the bottle.

Secure.  Using the virtual machine for injection purposes should never introduce a state of insecurity to the target machine or network!  While it is all but impossible to make a virtual machine immune to examination by a local user who controls the machine, the virtual machine should not be easily attackable from the outside or use a cleartext control channel.

Elegance.  This cannot be understated.  While it is easy to write a functional and spartan language for the purpose, it is not going to be pleasant for the programmer.  Programmers are happiest when they have a language that does not get in their way.  If the language and virtual machine environment does not have this characteristic, then people are not going to use this tool.  Elegant design also lends itself well to other desired traits such as dynamism and compactness.

I will be taking a critical eye at the implementations that I have worked on in the past and contrasting it with these requirements, where they succeeded and where they failed.  If you, my dear reader, feel that additional requirements should be applied, let me know!

6 Comments

Controlling ActiveX Controls

Dave G. | March 13th, 2008 | Filed Under: Industry Punditry

So, lots of talk about ActiveX controls these days.  I actually had a larger post in mind, but I am pressed for time, so here are some quick thoughts on why browser accessible ActiveX controls are so frustrating:

1. ActiveX controls aren’t (usually) tied to the websites that installed them.  Meaning, any website can instantiate one and communicate with it.  And by communicate with it, I mean perform memory corruption attacks that lead to remote code execution.
2. They are often written poorly.  Even more poorly than most 3rd party software.  Overflows, arbitrary file access… you name it.  You could probably find an ActiveX control that is actually vulnerable to every bug class.   
3. They persist (and can be difficult to remove(*)).  After they get installed, you forget about it.  Forever.  Long after you have even logged into the website that convinced you to install it.   Just waiting for someone to take advantage of issues 1 and 2 to make you part of their botnet.
4. They can be difficult to update.  Unlike a lot of software, ActiveX controls rarely have auto-update functionality.  As a result, most people that are vulnerable, stay that way. 
5. They are rarely necessary.  The worst part is, ActiveX controls are often add-ons that no one really needed and wouldn’t miss if they disappeared.  A lot of times that I have seen them used, they were mostly there to make a UI feel more Win32 and less webby.  The risk to benefit ratio has rarely been worth it.

(*) By difficult to remove, I am referring to the average user. 

15 Comments

Seven Deadly Pen Test Sins

Dave G. | March 11th, 2008 | Filed Under: Industry Punditry

This is a quick list of sins that I think most people that do this are or have been guilty of in the past:

1. Managing Time.  One of the most common yarns about the difference between pen testing and hackers is that pen testers have a limited amount of time to look for vulnerabilities in a small window of time in the applications’ life.  As a result, time is precious.  Knowing when you are on to something that you can confirm in a reasonable period of time is probably the biggest place where good pen-testers go bad.  It is really easy to turn a pen-test into a research project.  And while you toil away on a tiny sub-section of the application, you may never get to that remote code execution flaw that is lurking elsewhere.  This is also where coverage vs. depth plays a huge role.
   
2. Smugness.  Devaluing findings that customers care about.  Yes, XSS and CSRF are lame findings to people used to exploiting memory corruption or even compared to sql injection and auth bypass.   This also extends to “I found one, let the dev team find the rest of them”. Smugness can also be extended into overconfidence.  And overconfidence equals underestimation.  This all results into missing vulnerabilities that you needed to find.
3. Never understanding the app.  It is easy to just treat an application as a series of inputs, and not bother to understand what the application is actually for or what it is actually doing under the hood.  Good penetration testers are often trying to get into the developers’ heads.
4. Over-automation.  While I am a big fan of utilizing tools to make people more effective, there are two problems with relying on these tools:
   1. They generally create as much work as they eliminate.  False positives in the popular web app scanning tools are still common enough that you waste a lot of time, especially on a small website.  
   2. After you run them, you don’t have any better understanding about the application.  It encourages #3.
5. Sloth.  This raises its ugly head in a couple of ways.  It is usually either in avoiding the difficult parts of the test or conversely the easy parts of the test.  Total human nature.  One is hard and the other is boring.  As a result, usually the team that comes in after this sinner finds serious flaws that are either hard-core or are embarrassing.
6. Stagnation.  Given the difficulties of the job, it is easy to not evolve one’s skillset.  This is compounded by the fact that even in 2008 there still aren’t enough resources out there to keep evolving.   This is also an organizational problem inside of every company.  It is why the evil M word raises its ugly head.  
7. Communication/Soft Skills.  Where it’s a project manager or the customer, you need to understand what the customer cares about, manage their expectations, and oh yeah… if you can’t actually care, at least pretend to.  Lots of people think that doing a good job is simply breaking the app.   That is enough until someone who is as technically savvy as you rolls in and also has that polish (you know, the one that you can’t stand).  

The results of these sins all generally lead to missed findings and an overall craptastic job. 

9 Comments