Over the last year of working here at Matasano, I’ve had some time to refine the code enough for release. I’ve cleaned up a bunch of the code resulting from my inexperience with Ruby, implemented a couple features and fixed several bugs. Meanwhile, Chris Rohlf began a Linux port and its beginnings are included in the gem. And yes, Ragweed is now available as a gem through github. (sudo gem install tduehr-ragweed for the impatient with github as a gem source)

When reversing, the usual debugging tools for developers aren’t as useful. They’re built for stepping interactively through programs you have source code for. They don’t generally have methods to get data out.

Reversing also requires being able to do mean and nasty things to the running process. When tracing calls, you want to watch how they interact. The last thing you want to do is anything manual. Automation is a requirement.

Also helpful is the ability to automate information gathering tasks, or the ability to dynamically add, remove or change breakpoints. These features are why scriptable debuggers have been created: To play with black boxes in a more dynamic and seedier manner.

There are already scriptable debuggers out there.  The most notable are PaiMei/PyDbg, Immunity Debugger and IDA.

PaiMei is written in Python, bills itself as “a reverse engineer’s swiss army knife” and uses the Python ctypes library for low level win32 calls.  

Immunity Debugger is a GUI debuggger for win32 that uses Python for its scripting functionality.  

IDA Pro is largely a win32 disassembler, but it is scriptable, again in Python, and includes a debugging module.

Before I get run off by a screaming mob with pitchforks, flightless birds, members of the family bovidae, etc., I will also mention GDB which has a library in development (libgdb) and can be scripted through macros.

With the exception of GDB which runs on most platforms and has its own macro language, these all share two common problems: Win32 and Python. Matasano is a Ruby shop. We like Ruby. It is good to us. We also wanted a tool for non-Win32 applications. But mostly, we just wanted something in Ruby.

I’m going to stick to the OSX side of Ragweed for this article since I’m most familiar with it and there is still work to be done to unify the (currently) three debugging APIs —- Win32, Linux, and OSX —- inside Ragweed.

Under the hood, Ragweed (on OSX) uses Ruby/DL to perform the various low level system calls necessary to create a debugger. (More about that in my post from last year). These calls are abstracted somewhat to provide a smoother, more Ruby-like interface.

There are two caveats for Ragweed in OSX:

• Due to the changes in Ruby 1.9 to DL, it is currently incompatible with 1.9.

• Also, under OSX, Ragweed wants to run as root due to restrictions on task_for_pid.

That’s it. We just override the signal handlers for the signals we want to know about (or not), attach to a running process, set and install breakpoints, and it’s off to the traces. A simple hit tracer is only a CSV file and read loop away from this.

Want info on a region of memory?

d.region_info(0x0,:basic).dump

What about thread_info?

d.threadinfo(threadid).dump

Break stuff by playing with registers?

regs = d.get_registers(thread_id) regs.eip = 0x420f d.set_registers(thread_id, regs)

Grope through the child’s memory?

Ragweed::Wraposx::vm_read(d.task, address, size) #returns a string of child's memory

There you have it. It’s not pretty but it’s only begun.

CitySec Austin is an informal meetup of information security professionals in Austin. Unlike other meetups, you will not be expected to pay dues, “join up”, or present a zero-day exploit to attend.

Where

Berryhill Baja Grill
3600 N Capital of TX Hwy.
Austin, TX 78746

When

The third Thursday of every month from 5:00pm until we leave.

Why

We know about ISSA, OWASP, and ISACA. Not casual enough. We don’t want to hang out in conference rooms. Just a chance to meet other security folks without sitting through a sales pitch. We also have been known to drink beer.

RC4 is an older stream cipher. AES is a newer block cipher. Block ciphers encrypt in multi-byte chunks (AES’s block size is 16 bytes). Stream ciphers encrypt byte by byte. You might therefore assume RC4 was more convenient to use than AES. RC4 is also simpler than AES, and significantly faster on some benchmarks.

But the practical difference between the two is that you shouldn’t be using RC4. RC4 has significant weaknesses, some of which have contributed to serious breaks in 802.11 security.

The industry “knows” this. So why does RC4 keep popping up? Here’s a theory.

Stream ciphers seem like what you’d want if you were building crypto into a protocol. Most protocols send variable length messages. Block ciphers force you to think about how to fit those messages into fixed-size blocks.

RC4 is a famously simple algorithm. Extremely easy to implement.

There are lots of other stream ciphers. The eSTREAM project has spent the past several years conducting Stream Cipher Thunderdome, in which seven (7) ciphers remain standing despite the best efforts of full-time academic cryptographers. One of them, Trivium, is comparably simple to implement in C.

But nobody knows anything definite about these ciphers. You can still get fired for picking one of them.

It seems simpler to fit RC4 into a protocol than AES. But it’s remarkably easy to make AES act like a stream cipher: run it in Counter mode (CTR).

In CTR, you don’t use AES to encrypt your plaintext. You use it to encrypt a concatenated (say) 96 bit nonce and (say) 48 bit counter to generate a keystream, which you simply XOR into your plaintext. Every time you run out of keystream, you increment the counter and generate more of it. This is secure as long as the counter doesn’t cycle and the nonce is never reused for a new stream.

CTR turns block ciphers into stream ciphers.

CTR actually has practical benefits over RC4; because the keystream is generated by a counter, you can seek to an arbitrary point in the stream and start decrypting.

This is extremely well known to practitioners, and blessed by the 4 cryptographers at NIST. But a friend of mine recently asked a room full of developers how many knew the difference between ECB and CBC (a much more basic question). Almost no hands.

You need to encrypt variable length data. No modern stream cipher escapes the “exotic” stigma. Nobody wants to be the developer who picked FROG or MAGENTA instead of Rijndael before AES. Meanwhile, you don’t “get” block cipher modes. What are you left with?

A bad algorithm that still occassionally gets used in real designs.

I suppose I’d argue that if you see a bespoke crypto design, and it dates from after 2000, and it uses RC4, that’s an audit flag.

If you’ve ever looked under the hood of a Microsoft web app, you’ve seen a ViewState. For the uninitiated, ViewState is Microsoft’s workaround for the pesky fact that HTTP is stateless. It stores the present state of web page controls in a big binary blob.

Whenever I test an ASP.NET application, I either have to pop open a VM and use a viewstate decoding tool or slog through Burp Proxy’s viewstate list tree to see if there’s anything interesting inside. Neither option appeals to me.

Moreover, some sites I work with don’t have the ViewState MAC enabled (to which we normally just say “hey… turn this on”). The MAC is a cryptographic assurance that nobody but the application has tampered with the contents of the controls. If the MAC is disabled, I want to manipulate the viewstate to see if I can make something interesting happen. Unfortunately, there’s no real tool that I could find out there to do this.

So I wrote one.

When you render an ASP.Net page with UI-enabled form controls, ASP.Net serializes the states of those controls.

The resulting set of serialized objects is stored base64-encoded in a hidden form field, which persists between POSTs (as postbacks).

As interesting as that is, that’s not what this post is about.

Have you seen WWMD? You should check it out. It’s my bag of Ruby web pentesting tools.

I spent quite a bit of time in WWMD, and this seemed like the perfect opportunity to add a needed piece of functionality to it and also learn something about how viewstate actually works.

My design goal: a parser that outputs viewstate to YAML (failure, don’t try this at home) or XML (success! thanks Timur!) that could be easily read (or better yet, XPath search), manipulate and re-serialize from XML for fuzzing purposes.

Searching through the state of the game on the web, I started out by trying to reverse viewstate serializations in an attempt to get as much of the structure as I could. I spent about 3 hours doing this and decided that I wanted a working tool.

So, I popped open System.Web.UI.ObjectStateFormatter in .NET Reflector and start porting.

(Have you seen Lutz Roeder’s .NET Reflector? You should check it out.)

After some experimentation, I decided to implement an intermediate set of Ruby objects for each type of ViewState object. Some notes:

• Deserialization takes place in what (functionally speaking) is a big case statement (actually, there’s a lookup table to translate each opcode into a symbol and call self.send(:symbol) with a method defined for each type).

• As bytes are pulled off from the binary string, a new ruby object is created and filled with the correct values for that type, recursively adding objects until we are done.

• The MAC is added to the top level object if it exists (ViewState#mac_enabled? will return boolean).

• Each intermediate ruby object implements #serialize and #to-xml. Reading from XML is
  another recursive walk through a big case statement in #from-xml.

In debug mode, the deserializer will print out a line containing offset [opcode] type: value. Saving this output and grepping through it for string types (-e indexed_string: -e string:) has been quite useful (as opposed to searching through XML).

In about 3 full days of coding (and about another day spent debugging and refactoring), I had a working tool that does exactly what I needed to do. Many thanks to Jeremy (who planted the seed), Timur (for XMLfu), HexFiend and vbindiff (try reversing anything without these two).

1. INT. COFFEE SHOP, MORNING

A “young, cool-people’s” coffee shop on the first floor of an old office building in downtown Chicago. “My band is playing” notices line the wall. A hipster in a tight t-shirt hands a cappucino to MIKE TRACY while THOMAS PTACEK waits impatiently. The coffee shop is loud; Mike and Thomas raise their voices to be heard over the noise.

MIKE TRACY

Did you see that? He worked so hard on my coffee.

THOMAS PTACEK

What? Right. Whatever. Let’s get…

MIKE TRACY

He got all those little beans and put them in the thing and tamped them down and

THOMAS PTACEK

Whatever. Ok. We’ve gotta get ready for this interview

MIKE TRACY (CONT’D)

and he clickity-clack clickity-clacked with the machine and

THOMAS PTACEK

Mike! I get it! He made the shit out of your coffee. What are we going to ask this guy?

Mike walks to a table at the side of the shop, grabbing a lid and a sleeve for his coffee.

MIKE TRACY

(Miffed)

I don’t know. It’s your interview. Single signon cookies?

THOMAS PTACEK

Why SSO?

Mike is maneuvering around people entering the shop through a door leading out to the hallway.

MIKE TRACY

It’s got crypto in it. Everyone always fucks it up.

INT. HALLWAY - CONTINUOUS

Thomas follows Mike, walking towards the elevators.

THOMAS PTACEK

Yeah, that could work. We’ll have two apps. User logged into one of them, needs the other app to do something without making them log in.

MIKE TRACY

Print an invoice.

THOMAS PTACEK

Yeah, this will work. We’ll see if he comes up with the industry standard answer; the cookie both apps honor to let you in, encrypted so users can’t change their account to someone else’s.

MIKE TRACY

So, a base64 blob AES encrypted with a key both servers share? That’s pretty easy, isn’t it? Are we sure this isn’t a layup?

DING. An elevator opens. Thomas and Mike step inside.

THOMAS PTACEK

You’ll be surprised.

2. INT. OFFICE - LATER THAT MORNING

An unadorned off-white office lined with Ikea desks, piled with books, papers, and in one case a pile of random electronics tools (soldering iron, multi, etc). An EASEL PAD stands next to a large window looking out on a brick wall. Thomas and Mike sit office chairs with THE CANDIDATE.

THOMAS PTACEK

So you’d have app ‘A’ set a cookie with your account ID in it, right, but how would you keep the user from switching their account by messing with the cookie?

THE CANDIDATE

Uh, I’d encrypt the cookie?

THOMAS PTACEK

Show us how on the pad?

Thomas hands The Candidate a dry erase marker, as The Candidate walks to the easel pad.

THE CANDIDATE

Does it matter what language I write it in?

MIKE TRACY

Whatever you’re comfortable with.

THE CANDIDATE

(Writing awkwardly, addressing the easel)

Ok, so in C#, I’d use Response.Cookies, and

THOMAS PTACEK

You can just do the part where you encrypt the cookies.

THE CANDIDATE

Oh, ok.

The Candidate writes on the pad, slowly:

THE CANDIDATE

Sorry.

MIKE TRACY

No worries, writing code during interviews sucks.

THOMAS PTACEK

Can you walk us through what that code is doing?

THE CANDIDATE

Sure. So I’m Triple-DES encrypting the cookie, which is the “toEncrypt” function argument.

MIKE TRACY

Triple DES? Seriously?

THE CANDIDATE

Ah, yeah, you’re right, in my last job we had to use Triple DES for campatibility, but I’d use AES now.

The Candidate starts correcting the text on the pad.

THOMAS PTACEK

Don’t worry about it, keep going. But yeah, don’t use Triple DES for anything. It has a bunch of problems, but also an 8 byte block size, which is tiny.

THE CANDIDATE

Ok, so, I take the key and I turn it into an AES key by MD5’ing it.

MIKE TRACY

You know MD5 is broken, right?

THOMAS PTACEK

Yeah, that’s not really the problem there though.

THE CANDIDATE

Oh, I could just use SHA-1.

THOMAS PTACEK

SHA-1 is really fast. Can you see why that’s a problem here?

THE CANDIDATE

(Haltingly)

Um. Not really? Don’t I want this to be fast?

MIKE TRACY

What’s in the cookie you’re encrypting again?

THE CANDIDATE

A string of URL arguments…

The Candidate starts writing on the pad, “userId=39493&role=user&timestamp=1414919”

MIKE TRACY

So what’s to stop me from just running a dictionary through MD5, generating a key, and trying to decrypt the cookie? I’ll know I won when I get clean ASCII.

THE CANDIDATE

And how do I keep that from happening? You should use strong passwords anyways. And I use a salt with the key anyways.

Mike vomits onto the floor.

THOMAS PTACEK

Gross.

MIKE TRACY

(Wiping mouth)

A salt doesn’t do anything here!

THOMAS PTACEK

Just put a “for” loop around SHA-1 and run it 1000 times to generate the key; that’ll at least slow down a brute force attack. SHA-1 is lightning fast. By itself, it’s a crappy way to generate a key.

(To Mike)

Clean that up?

THE CANDIDATE

Well, I guess. Wait, why should we use a password here at all? I could just use a random string of bytes…

The Candidate writes again on the whiteboard

THOMAS PTACEK

That is much better. Sometimes it’s a lot more convenient to use a readable string. If you do, the loop around SHA-1 is similar to what PBKDF does, which is I guess a best practice here. But if you can keep structure out of your crypto keys, that’s much better.

THE CANDIDATE

Ok. Should I keep going?

THOMAS PTACEK

Your encryption function. Do you know what the “ECB” thing there means?

THE CANDIDATE

Oh, fuck! You’re right, that should be CBC.

(Pausing)

Sorry for swearing.

MIKE TRACY

S’okay. You’ll fit right in.

THOMAS PTACEK

You know the difference between ECB and CBC?

THE CANDIDATE

Yeah, like, each block feeds into the next one?

The candidate draws on the easel.

THOMAS PTACEK

Why’s that a win?

THE CANDIDATE

Because if any of the blocks repeat, you can see them repeat?

Mike has opened his laptop and is typing.

MIKE TRACY

(To the laptop)

We have a picture of that somewhere. Oh, here.

Mike raises the laptop up to show The Candidate

MIKE TRACY (CONT’D)

The top part is unencrypted. The bottom part is encrypted ECB. You’re like Jack from Heat Vision and Jack.

THE CANDIDATE

I know EVERYTHING! Right, because one bunch of 16 “black” bytes is the same as the next, so they show up the same in the picture. Neat. Also, in ECB mode you can cut and paste the blocks, right? He could take the “userid” out of your cookie and put it in his own?

THOMAS PTACEK

Sure. That’s a good answer. Let’s move on. Say you’re implementing a web server. What do you think, processes or threads?

3. INT. OFFICE CONFERENCE ROOM - AFTERNOON

A room in the same office, roughly the same size, with an oversized brown kitchen table in the middle, littered with paper and McDonalds wrappers. Thomas and Mike sit at the table, talking to a CONFERENCE PHONE.

CONFERENCE PHONE

So how’d he do?

THOMAS PTACEK

Pretty much aced it.

MIKE TRACY

What? He bombed the cookie part. He used ECB, MD5, and Triple DES!

THOMAS PTACEK

I’m impressed that he could spell ECB, MD5, or Triple DES. And it wouldn’t have mattered if he had used CBC, SHA-256, and AES-256. His code still would have been broken.

CONFERENCE PHONE

How so?

THOMAS PTACEK

He didn’t authenticate the message. Encryption isn’t —-

MIKE TRACY

(Chanting)

Encryption - isn’t - authentication.

CONFERENCE PHONE

Don’t you mean integrity?

THOMAS PTACEK

No, Dave, I mean authentication. They’re called message authentication codes.

CONFERENCE PHONE

Ok, Tom. But he screwed that up?

THOMAS PTACEK

Yeah, but who cares? I’m surprised he even knew what CBC was. But we just asked that to see how he thinks. We’re never going to let him implement crypto code anyways.

CONFERENCE PHONE

I guess we don’t even let you write crypto code.

THOMAS PTACEK

Sure, and when I asked him about processes and threads…

MIKE TRACY

Can I stop you both here for a second?

THOMAS PTACEK

Yeah?

MIKE TRACY

This room is pretty fucking boring. We’re in a screenplay, right?

THOMAS PTACEK

Oh, yeah, you’re right. Let’s fix that.

(Shouting)

Wings of silver!

CONFERENCE PHONE

Nerves of steel!

MIKE TRACY

Thundercats go!

EXT. HURTLING THROUGH SPACE - CONTINUOUS

The office melts away around them, revealing a starfield hurtling past as if moving at awesome speed. Meanwhile, the conference phone transforms into a UNICORN WITH LASER HORN.

DAVE THE LASER UNICORN

It’s “Silverhawks”, jackass.

THOMAS PTACEK

Where were we?

MIKE TRACY

Authentication?

THOMAS PTACEK

Oh yeah. Even if he had done AES-256-CBC. His code is still busted. I can make his messages say whatever I want them to.

DAVE THE LASER UNICORN

How do you do that? Isn’t that the point of CBC mode? Anything you change in the ciphertext randomizes the output. What can an attacker do with that?

THOMAS PTACEK

First of all, sometimes randomizing the output is all you need. If one of the key-value pairs in the cookie is your role, and the default role is “admin”, but the server always generates a “role=user” field…

MIKE TRACY

Yikes. Yeah, that’s bad. Have you ever seen that bug in the wild?

THOMAS PTACEK

Garbling a block to confuse an app? I found a similar problem recently. Login generates an encrypted cookie. Inside the cookie, comma-seperated key-value pairs. If you put a comma in your user name, the server doesn’t want you to inject your own key-value pairs, like “bob comma admin equals yes”. So it quotes the commas. You can mess up a block to eat the quote character.

DAVE THE LASER UNICORN

How do you know what block to mess up?

THOMAS PTACEK

It’s a cookie. You get unlimited tries. Each time, you add another ‘A’ to the login name, or mess with a different block. Eventually you line things up just right so that you’ve garbled the quote character but not the comma. Here, let me show you.

Thomas puts his hand to his forehead, and a beam of light emerges from his forehead, projecting a picture, because it’s my script dammit.

THOMAS PTACEK

Top hexdump. The plaintext of the cookie. Nothing’s been done to it. Second hexdump. The encrypted cookie. Key doesn’t matter. Third hexdump. I’ve flipped a bit in the second AES block.

DAVE THE LASER UNICORN

Convenient how AES blocks and hexdump lines are the same width.

THOMAS PTACEK

Fourth hexdump. The decrypted output, after flipping that bit in the ciphertext. Notice that flipping one bit totally garbled the second block —- and ate my quote character.

MIKE TRACY

Doesn’t the app reject the cookie because of the garbled stuff in the middle of it?

THOMAS PTACEK

Probably not. Why would it? C# and Java and Ruby and Python don’t care what go in your strings. And hey, if it does reject them, flip a different bit. Totally different output. You get 2^128 tries.

MIKE TRACY

Good point. What’s with the red “B” in the decrypted hexdump?

THOMAS PTACEK

Getting to that. Turns out, I can make the cookie say whatever I want. It’s a property of CBC.

The property is this: take a ciphertext block and flip bit 0 (or 2, or N). The resulting plaintext for that block? Garbage. But the next block is normal… except has that bit flipped. Not good!

MIKE TRACY

So you sacrifice one block and flip bits in the second block?

THOMAS PTACEK

Yeah. Although let’s stop calling it “flipping bits” and call it “rewriting”, because that’s what you’re doing.

DAVE THE LASER UNICORN

If you know what bits to flip.

THOMAS PTACEK

You always know what the bits are.

DAVE THE LASER UNICORN

How?

THOMAS PTACEK

Because the bits are always 0x41414141.

DAVE THE LASER UNICORN

Huh?

MIKE TRACY

Because that’s what he stuffed them with. He logged in as bob A-A-A-A-A-A-A-A-A.

THOMAS PTACEK

Right. An SSO cookie is usually, what, 100 bytes? If I stuff 1000 A’s after my login name, and the cookie grows to 1100 bytes? Almost all of those bytes are known to me. Here.

Again with the forehead beam thing.

AES encrypt something that I partially control. Doesn’t matter what the key is. Now XOR that block into the ciphertext. Decrypt it, and somewhere in it you get a random block and “&admin=yes&x=AAAAA”.

MIKE TRACY

Not good.

THOMAS PTACEK

If you’re encrypting something it’s usually somehow user-controlled. I’ll find that by plugging 100 A’s into each form field and waiting for the cookie to grow.

DAVE THE LASER UNICORN

How will you know if the cookie is AES?

THOMAS PTACEK

Same way Chris Eng said to. Add A’s one at a time, see what increments the cookie grows in. 16 bytes at a time? AES.

DAVE THE LASER UNICORN

And CBC?

MIKE TRACY

If you’re encrypting all A’s, the ciphertext blocks will repeat.

DAVE THE LASER UNICORN

And how do you know the format to write into the cookie?

MIKE TRACY

Who cares? Trial and error.

THOMAS PTACEK

Yeah. Point is, you thought encryption protected the contents of the cookie. It doesn’t. Oh look, we’re almost there.

Thomas, Mike, and Dave hurtle towards a star system, a solar system, a planet, powers-of-ten-style, towards the Michigan shore, converging eventually on an office building, and then

INT. OFFICE CONFERENCE ROOM - AFTERNOON

CONFERENCE PHONE

That was really fucking anticlimactic.

4. EXT. PARKING GARAGE - EARLY EVENING

Thomas stands next to his car, a black Volvo 850 held together with duct tape, talking on a cell phone to NATE LAWSON.

NATE LAWSON

You know this scene is a really bad setup for a movie, right?

(Pausing)

And I’m not really OK with you putting words in my mouth.

THOMAS PTACEK

Yeah yeah, whatever. Shut up before I turn you into a claymation character. So yeah, it’s amazing how you can be a top tier vuln researcher for over a decade and not really get how bad it is not to have a MAC.

NATE LAWSON

A MAC doesn’t necessarily save you either.

THOMAS PTACEK

How so?

NATE LAWSON

There’s still a bunch of things you can do wrong. Like I was just saying, Google Keyczar did almost everything right, but compared the MAC using a timeable comparison function. You could tell how many bytes of the MAC matched by watching how long the function took. People make that mistake all the time. An even more common mistake is to generate an error message when your padding is wrong. If you do that, you can decrypt messages.

THOMAS PTACEK

I’ve heard about that. The Bleichenbacher PKCS thing, and the Vaudenay paper.

NATE LAWSON

This was a major TLS finding too.

THOMAS PTACEK

I’ve never really been all that clear on how this works.

NATE LAWSON

Well you know how PKCS 7 padding works, right?

THOMAS PTACEK

Yeah, you have 2 bytes, you need to fill 16 bytes for an AES block, so you fill the remaining 14 bytes with 0xe.

NATE LAWSON

So if you tack a random block onto a CBC message, what happens when the receiver decrypts it?

THOMAS PTACEK

It comes out random.

NATE LAWSON

And the padding?

THOMAS PTACEK

Broken.

NATE LAWSON

Right. And if you send an error when that happens, you know the padding failed. Now if you keep trying different random blocks, what’s eventually going to happen?

THOMAS PTACEK

Uh…

NATE LAWSON

You’ll get a message with valid padding. Valid padding might be 0x3 0x3 0x3. Or it might be 0x4 0x4 0x4 0x4. But if you’re basically generating random blocks, what’s the mostly likely padding you’re going to get that will pass the check?

THOMAS PTACEK

0x1.

NATE LAWSON

Right. And you’re randomizing the output by tacking a random block in front of real ciphertext, which gets XOR’d during decryption. So you know the last byte of your random block…

THOMAS PTACEK

And the 0x1 that you know the padding is, and so that random byte XOR the last byte of the plaintext is 0x1, and so you know the last byte of the plaintext.

(Pausing)

And now that you know the last byte of the plaintext, you can make the padding come out to 0x2 and try randomizing the other 15 bytes to find out the next byte, and so on?

NATE LAWSON

Close enough.

THOMAS PTACEK

That is fucked up. All you did wrong was show me the exception your library generated when you decrypted the block, and I could decrypt a block. You got to reason byte by byte instead of block by block.

NATE LAWSON

You can decrypt whole messages that way. It’s called an error oracle. You can’t show clients discernable errors. You can’t even take different amounts of time to do things! You can watch the system with random inputs and measure how much time things take.

THOMAS PTACEK

There’s no way any programmer is ever going to get this stuff right.

NATE LAWSON

Professional crypto people don’t even get this stuff right. But if you have to encrypt something, you might as well use something that has already been tested.

THOMAS PTACEK

GPG for data at rest. TLS for data in motion.

NATE LAWSON

You can also use Guttman’s cryptlib, which has a sane API. Or Google Keyczar. They both have really simple interfaces, and they try to make it hard to do the wrong thing. What we need are fewer libraries with higher level interfaces. But we also need more testing for those libraries.

THOMAS PTACEK

Like I’ve been saying, if you have to type the letters “A-E-S” into your source code, you’re doing it wrong.

NATE LAWSON

Uh. Ok. Whatever you say, Tom.

5. FADE TO BLACK

Fade in epilogue:

The next day, Thomas’ planet was destroyed. Have you guessed the name of his planet? It was EARTH! DON’T DATE ROBOTS.

The complete change history behind every line on every firewall ruleset — who requested it, why, what were the suggested changes and who approved them — is then available at the click of a button. We think that’s pretty cool.

We are very excited about 2.5 and hope you are too. Check the updated Playbook’s site to see what’s new in 2.5.

Sign up today to learn more about Playbook.

Do you find yourself annoyed,  feeling that you’ve hacked together code to do this many times before, but simply can’t find it?

Do you find yourself hobbling together other tools to do what you need? Do you find yourself wishing you had a Burp for raw TCP connections?

No MORE! Using Matasano’s Port Forwarding Interceptor you have the tool you need right at your fingertips! Lets take a closer look at this exciting new tool shall we? 

Let’s say you are watching your favorite 15 minute ANSI art rendition of Star Wars on telnet://towel.blinkenlights.nl . You think to yourself:

“Man I sure wish I could get in-between my telnet client and the server and begin reversing this Star Wars protocol”.

Then you remember you got Matasano’s PFI off of Github earlier today!

You take a look at the usage and it seems pretty self explanatory…

So then you decide to try it out by running something like this:

(This sets up PFI as a TCP port forward listening on the loopback interface on port 23 and forwarding traffic to towel.blinkenlights.nl on port 23, but you knew that already of course, thats why you ran it…)

You are then greeted by the comforting and familiar PFI GUI windows. And hey, you didn’t even have to install any weird python modules or dependencies!

You take a minute to notice how simple and self-explanatory it all is. One window displays the intercepted text, and allows you to choose whether to intercept. The other window allows you to edit the intercepted data before it is passed on through the tunnel. How easy! It is like a “Burp” for raw TCP!

You then decide to try it out by connecting through the tunnel:

And begin watching your ANSI art show:

So the tunnel works! You look back at your PFI main window and see that data is in fact passing through PFI.

You select the “Intercept” check boxes and begin intercepting and editing data across the tunnel.

And as you begin reversing the complex  ANSI Star Wars protocol you cant help but feel yourself awash with gratitude that Matasano PFI saved you the trouble of having to dig out all your old scripts and programs. You give your monitor a thumbs up and say: “Thanks PFI!”

Then you remember that Matasano Blackbag also had a similar tool (called replug) and then you feel silly, not just about neglecting Blackbag but also that you gave your monitor a thumbs up.

It would look a lot like Google Native Client (NaCl): a mechanism to download a game written in C/C++ from Id Software and run it in your browser, without giving Id Software the ability to take control of your computer.

Google NaCl is, on its face, a crazy-talk idea. It’s a browser plugin that downloads native x86 code from a website and runs it on your machine. If this sounds familiar, it’s because Microsoft tried it over a decade ago with ActiveX. If you’re skeptical about the idea, it’s because ActiveX was a security calamity; O.K. an ActiveX control, and it owns your machine almost completely.

So the primary obstacle between Google and the future of software delivery is security. Google has a lot of interesting ideas about how to overcome that obstacle. But security people unlikely to take Google’s word for it. So Google held a contest: “we’ll publish the source code, you’ll find flaws. The winner gets $0x2000 USD.”

We took the bait. And things were looking great for us.

Then Skynet noticed Google, decided it was a threat, and sent a Mark Dowd unit back through time to terminate it. The contest winner hasn’t been declared yet, but as we aren’t a murderous robot made out of liquid metal, we’re guessing we didn’t take first place.

But we learned lots of stuff in the process, and so we have stuff to say. And when you get right down to it, isn’t that worth a lot more than money? Think about that, while we share some lessons about NaCl and cry into our discount beers.

What is NaCl, and why do I care?

Absent any innovations in browsers or HTML, within 5 years, every application normal people use will be run off a web server and rendered via Javascript, HTML, or Flash. That trend is inexorable. You know it and I know it.

But what if we want to ship a game title? We need a faster runtime, real graphics, and a decent interaction model. Until recently, if we wanted to run “real” programs behind a browser, we had two options: ActiveX and Java.

Consider ActiveX. It’s a really simple idea. In your HTML code, you can specify a URI to a library file. The same kind of libraries your desktop applications use. They’re written in whatever language makes sense to the authors, but they’re delivered as native X86 instructions. They talk to your computer the same way any application does.

This is a powerful concept. Virtually anything a desktop app can do, an ActiveX control can do. Unfortunately, read that last sentence again.

So that’s a problem. Most security-conscious people aren’t willing to trust random native executables running on their computers off web pages.

Consider Java.

Java does something that ups the security ante significantly. Instead of delivering raw X86 opcodes, it’s delivering JVM bytecode. The Java plugin either interprets or compiles that bytecode, but either way, the actual instructions executing on your computer and talking to your OS belong to Java. You only have to trust the one native program.

But wait, there’s more! Because Java programs don’t execute directly on the CPU, there’s an architectural opportunity to improve security: you can put a security layer in between the program and the operating system. Java calls this the “applet sandbox”. The applet sandbox basically says, “applets can’t talk directly to the OS; instead, they have to use a set of interfaces we created specifically for allowing applets to talk directly to the OS”.

You can’t do this with ActiveX, because you’re executing raw X86 instructions. From userland, on an X86 chip, with any reasonable performance, there’s no way to put a layer between Win32 userland (or OS X) and the kernel. If you want that layer there, you have to not execute code off the Internet directly on the CPU.

But wait, there’s still more! Because the JVM designers got to design their own virtual machine and controlled the whole runtime, they were able to make Java bytecode very “regular” and very easy to analyze. Java is so easy to analyze that 99% of compiled Java binaries can be decompiled right back to source code.

X86 is not regular and easy to analyze.

For one thing, X86 instructions come in various shapes and sizes. The shortest X86 instructions (such as “INCR EAX”, or “add 1 to the EAX register”) are one byte long. Because X86 instructions can accept prefixes that change their meanings, the largest X86 instructions are over 10 bytes long.

The irregularity of X86 instruction lengths does more than just make instructions tedious to recognize. It also means that, unlike in a RISC architecture where instructions are always, say, 32 bits long, an X86 instruction is not necessarily at an aligned offset in the file. Byte 1024 of an X86 executable is as likely to be the middle of an instruction as it is the beginning of one.

Why does this matter?

You can’t easily put a layer between a bare-metal X86 program and the OS kernel to keep that program from calling “execve” and running any program it wants. So, instead say you wanted to write an X86 “verifier” that would check programs you ran to make sure they don’t try to call “execve”. You’d have some problems.

For starters, because X86 programs can jump pretty much anywhere in their instruction streams, even if you disassembled the program and checked for calls to execve, a malicious program could make a series of innocuous instructions which, when the program jumped into the middle of them, actually executed execve.

Second, as anyone who’s ever written overflow shellcode knows, X86 programs can execute out of data. So even if you verified the instruction stream perfectly, the program could use innocuous instructions to create malicious instructions in data that the verifier couldn’t reasonable check.

And now we come to the part where we explain why we’re telling you all of this.

Behold, Google NaCl!

Repurposing an idea from the mid-’90s, NaCl employs a very simple trick to make native X86 programs reliably verifiable. And if you can verify an X86 program, you don’t need a layer between the program and the OS: you can just have rules, and refuse programs that break the rules.

The trick is: restrict X86 programs to those that are verifiable.

What are those programs? Well, among other things:

• They must admit to simple disassembly, yielding a stream of recognizable opcodes. This wouldn’t be a strict requirement for an X86 program, but most programs adhere to it anyways.

• Those opcodes must not jump to anything but the beginning of an instruction recognized by that simple disassembly. Easy to say, tricky to implement, but not a huge design change for most X86 code.

• They can’t modify the program text itself.

(Note that we’re butchering this concept a little; in the literature, it makes a big difference whether you patch a candidate program to safety, or whether you detect unsafeness and halt. But we digress.)

With those constraints in place, it turns out NaCl can reliably analyze X86 instructions. The verifier can then add rules:

• You can’t muck with memory management to fool the verifier.

• You can’t talk directly to the operating system. Instead, you can call into trusted code in the first 64k of the binary that will make selected system calls for you, just like with a Java applet.

There is a very important difference between what NaCl is doing and what Java is doing. Java’s security measures are chaperones. They’re always there and always checking your actions. NaCl’s mechanisms are just rules. They’re checked once, and then the program is on its own. NaCl promises to be faster than Java.

More importantly, to build a NaCl program for your customers browsers, you don’t have to port to Java; you just have to use the NaCl build environment (a patched GCC that targets a simple ELF module) on your existing C code.

Google, for instance, ported Quake.

It is worth mentioning here —- and this is to NaCl’s credit —- that this isn’t a new idea. The fundamental approach dates back to Wahbe et al, from SOSP in 1993 (!). The core ideas that make the approach work on X86 (Write/Jump sandboxing vs. full Read/W/J isolation) are in an 05 MIT TR and Usenix article by Stephen McCamant (then a grad student at MIT) and Greg Morrisett at Harvard. And in 2008, Bryan Ford and Russ Cox released a version of the same idea, called Vx32, that runs on Linux and FreeBSD.

Google NaCl would be the first mainstream implementation of the idea.

That’s a bit scary, but if you generalize a little bit, the ideas at play here aren’t really all that different from the ideas VMWare relied on; in fact, Vx32 runs code out of basic block caches just like VMWare (and DynamoRIO before it) and oh my god we need to stop geeking out about this now.

What could possibly go wrong?

First: assume the X86 verification all just worked. You’re still doomed.

That’s because X86 ELF modules are much more complex than most of the file formats browsers already deal with. Any of a million little mistakes in the parser and loader code are going to be game-over security vulnerabilities. Remember Mark Dowd’s crazy Flash vulnerability from last year? The core problem was just a silly parser bug.

We found several of these types of problems. So did Dowd’s team. They were what we focused on.

It’s worth noting that Java had these types of vulnerabilities too. But Java is over a decade old, and this part of Java’s attack surface has been pretty heavily tested. We don’t think it scares security people that much anymore in 2009 (these are famous last words).

Second: assume the loader code is audited several times over, and that it reaches the same level of trustworthiness as the image loading code in your browser already. You’re still doomed.

That’s because NaCl programs still need to be able to talk to the operating system to draw and communicate and manage memory. The NaCl verifier rules keep programs from doing this directly; instead, NaCl programs use virtual system calls through special callgates in a small block of trusted code.

This trusted code base is architecturally similar to the system interfaces in the JVM, but it’s also in many ways more complicated. The JVM needs to provide services to Java programs in terms of Java classes and data types, which is a straightforward prospect. NaCl needs to provide many of those same services in terms of raw memory and state. Screw any of this up, and the contract that keeps NaCl programs bound up in the sandbox can be broken. Programs can escape the sandbox.

Dowd’s team found one of these problems. The NaCl SDK exposes mmap() and munmap(), and allows it to unmap and remap the text segment. But by the time these calls were executed, the program had already been verified. By remapping in code, it was possible to get code into the text segment that wasn’t verified, and could contain real system calls.

Now again, it’s worth noting that Java has these problems too. Most famously, Dino Dai Zovi found a really bad integer mishandling problem in the Java QuickTime extensions that would allow Java programs to directly manipulate raw memory. And unlike the loader bugs, which might be sussed out of the JVM by now, there are probably more problems like this in Java. There’s a lot of action in this part of the attack surface.

Third: assume the loader works and the trusted code base that gates the operating system works. But stop assuming the verifier works right. Because it might not.

In what’s probably the best finding of the contest, Dowd’s team broke the verifier.

As we mentioned earlier, X86 instructions can carry prefixes that alter their operating. The most notable things these prefixes do is change the way the CPU interprets addresses mentioned by the instructions. For instance, the segment override prefixes can tell the CPU to refer to offsets into the data segment (outside the sandbox) instead of the code segment.

So you don’t want to allow those prefixes in NaCl jump instructions. And NaCl didn’t allow them. For jumps with 8 bit relative addressing. But the 16/32 bit addressing variants —- the two byte ones that start with 0Fh? Not so much. Missed that one. In the contest build, there were instruction sequences found that could jailbreak you from the sandbox.

Now if you go through the history of the Java applet sandbox, there are comparably bad flaws —- though none have been found in awhile. But you could also argue that NaCl is at a disadvantage here, because they aren’t implementing the operation of the entire X86 instruction set, but rather a security retrofit of it. When they miss problems like this, they’ll end up with verifier-level jailbreaks.

On the other hand, there’s nothing in the CS literature that suggests those jailbreaks will be that much harder to fix than bugs in the trusted code base. So you could argue that they’re at increased risk of implementation flaws because of this design, but that the design itself isn’t really in any way flawed.

Fourth: Get everything else right, and you still have an architectural flaw: side channels.

Now our lawyers instruct us that we’re required to inform you that Google disclaimed side channel attacks, saying NaCl in its current incarnation was specifically not designed to handle them. Side channel attacks were excluded from the contest rules.

But that doesn’t mean they aren’t a problem. What’s that problem? NaCl programs have access to fine-grained time, and raw access to memory for instructions and data.

The problem is that for the past 6-odd years, crypto-systems researchers (like Dan Boneh, Eran Tromer, Dan Bernstein, Onur Aciicmez and Colin Percival) have been generating research results showing how attackers can extract private keys from systems using fine-grained timers. Some of the most interesting work in this vein shows what attackers that reside on the same hardware as their targets (like in a shared hosting environment) can do by timing microarchitecture features like caches.

Concrete example? Ok. You’re a NaCl program running alongside an SSL implementation being coerced into running over and over again. The SSL code is deciding to jump to different locations in its code based on bits in a secret key. The X86 caches branch targets to implement branch prediction. NaCl programs can’t —- in fact no program can —- directly access the BTB caches. But they can implement code sequences that will time differently depending on what’s in them.

In the canonical exploit, the attacker writes a “spy” program that continuously generates traces based on predicted cache contents. Those traces can be downloaded and used to reconstruct guesses about private keys. They don’t need to be exactly right; they just need to drastically reduce the search space of a brute-force attack.

The contest rules didn’t stop Ralf Philipp Weinmann from submitting a side channel bug (the obvious one, that NaCl exposes the RDTSC instruction for 64 bit fine-grained cycle timings). Google disqualified the finding. We’re looking forward to seeing how that plays out, and we have more to say about this problem.

What does it all mean?

To our knowledge, flaws were found in all the exposed attack surface except for the secure ELF loader. So, that happened.

But what did we expect? The smartest people in the world can’t get software 100% correct, even when security is the key design goal, and the software is tested for over 10 years.

It’s too early to say whether we like the NaCl approach more than the Java approach, or the Flash approach. It might be fair to summarize the approaches as follows:

• Java and Flash have a more resilient architecture, but a lot of moving parts.

• NaCl has fewer moving parts —- most of the work is done by content-controlled x86 code —- but those parts are at greater risk.

What’s hard to argue right now is that NaCl’s code is at an early stage. It’s not secure yet.

So was it smart for Google to hold this contest?

We reserve judgement on the marketing side of things, but from a practical perspective, we’re very glad Google did this. We can all put a bead on where the NaCl implementation is today, and more importantly, the NaCl team has a good idea of what the hotspots are for shoring up security and improving dev practices.

Does NaCl matter? After all, it’s just a beta research project! Well, it has the potential to bring thousands of preexisting C/C++ applications straight to the browser. Think of it this way: all Google has to do is convince one top-tier game developer to release a title on NaCl exclusively for a couple months. It’ll hit critical mass.

Whether it matters or not, wow is it fun stuff. Getting to spend a day on the clock going through SOSP program transformation papers, looking at compilers, and thinking about verification: this is why we got into the business. So as security researchers, we have to thank Google for the opportunity.

The winners of the contest haven’t been announced yet, but unless Dowd disqualifies his team by staging an assault on the Googleplex with automatic weapons in an attempt to defend Skynet from a future Google threat, you’ve gotta assume they’re taking this one. Our congrats to Mark Dowd and Ben Hawkes.

The next Playbook release includes a new feature aimed at addressing both of them. We call it “request and approval workflow”, or just “ticketing” (or “the left fielder”, Because is center field).

Here’s a brief walkthrough of Playbook’s ticketing feature as a preview of the upcoming release:

• Change requests are entered as tickets, accessible through a new Tickets tab.

• Each ticket can have multiple customizable fields, each field can have specific characteristics (e.g., protocol fields can cross reference protocol pages in the wiki, user fields can only contain valid users, etc).

• New requests go into the Unassigned queue, where users can pick them up.

• Engineers working on a ticket can easily search their current rules for firewalls to change, or the wiki for all mentions of a requested protocol.

• Rule changes for a ticket are staged into a ticket branch. These changes do not hit your firewalls until they are approved, allowing for others to continue working on other requests without trouble.

• Tickets submitted for approval include diffs for all suggested rule changes for approvers to review

• When a ticket is approved all its associated changes are merged into the main branch and will be pushed to your firewalls during the next sync. An annotated rule view tells you not only who’s behind each line in your rules but also why is the line there, by linking to the ticket with all the history behind the original request.

Sign up today to test drive Playbook and its new ticketing feature.

You: Awesome at penetration testing and ready to kick ass in either New York or Chicago.  You know travel is a part of the job, and are comfortable testing apps other than web apps.  In your free time, you still like doing this stuff.  Ideally, you can be dropped into an engagement, and hit the ground running.  Great if you have arch/design review and training experience, but by no means a requirement.   We would consider relocation to NY or CHI, but are not currently able to sponsor H1Bs.

Send email to careers.