Regarding The Post On Chargen Earlier Today

Thomas Ptacek | July 21st, 2008 | Filed Under: Uncategorized

Earlier today, a security researcher posted their hypothesis regarding Dan Kaminsky’s DNS finding. Shortly afterwards, when the story began getting traction, a post appeared on our blog about that hypothesis. It was posted in error. We regret that it ran. We removed it from the blog as soon as we saw it. Unfortunately, it takes only seconds for Internet publications to spread.

We dropped the ball here.

Since alerting the Internet earlier in July about the upcoming announcement of his finding, Dan has consistently urged DNS operators to patch their servers. We confirmed the severity of the problem then and, by inadvertantly verifying another researcher’s results today, reconfirm it today. This is a serious problem, it merits immediate attention, and the extra attention it’s receiving today may increase the threat. The Internet needs to patch this problem ASAP.

Dan told me about his finding personally, in order to help ensure widespread patching before further details were announced at the upcoming Black Hat conference. We chose to have a story locked and loaded for that presentation, or for any other confirmed public disclosure. On a personal level, I regret this as well.

Dan did phenomenal work on this research. It was impossible to talk to him today and not know that he was sincere about coordinating a graceful disclosure and fix for the problem. That I helped detract from that work is painful both personally and professionally, and I apologize to Dan for the way this played out.

Thomas Ptacek

Principal, Matasano Security

Jul 21, 2008

101 Comments

What I’ve Been Doing On My Summer Vacation or, “It has to work; Otherwise gdb wouldn’t”

Timur | July 17th, 2008 | Filed Under: Apple, Uncategorized

An intern expects to be given simple projects, like coffee retrieval, or “Hello, World.” So I’ve been sorely disappointed by Matasano. I have been offered coffee retrieval services by senior engineers and my latest project has been anything but “Hello, World.”

In fact, it’s been more like, “Hello, OS X. Tell me your secrets”.

This is the story of one trial-by-fire project handed to an intern that turned out to be more complicated than anyone expected.

1.

It started with Thomas, innocently enough, handing me some debugger code. It was both C and Ruby, and for Solaris and Win32. He said, “I would like you to port this Win32 Ruby code to OS X.”

“Um, okay.”

At that point I’d just finished learning the basics of Ruby via my previous Matasano project, a database backed HTTP proxy. I knew nothing about debuggers, let alone the low level C library calls I’d need and Ruby bindings to make them work. I know, fun, right?

I started simply and dusted the C off in my head so I could begin to read and understand the code Thomas dumped on me, and perhaps learn how a debugger works and gets used. It took a day or two just to read it. I’d ask the office some fairly basic question about debuggers, and receive in return a much longer response than I’d anticipated. Like a tutorial on the workings of x86 assembly. Eventually, I got to a point where I was almost comfortable with how the C debugger worked.

When staring at C code stopped doing me any good, and writing Ruby code started seeming feasible, I moved on to porting the Ruby code. “How hard could it be?”.

2.

Thomas gave me a starting point. Our Ruby code called directly into C libraries using Win32API and Ruby/DL. We have wrapper libraries that make those C calls look like Ruby library functions. So, for instance, in our Wrap32 library, we have:

We had a small piece of this written for OS X as well. I had to build it out. I started with getpid(), a simple system call I could make sure worked before I moved on to something harder. It worked right away. My confidence was high. I was feeling cocky.

Here I should mention that I’d never worked on a decently large coding project before. This was my first.

Throughout this entire project I’ve been trying to write the entire thing far before I actually write even a single function. So, I had many questions:

• What was the script implementing the debugger to look like?

• Was it to be event driven?

• Did we want objects to represent each process, threads, or to make his lunch for him?

I was overzealous. The team was patient. Thomas said simply, “There is no spoon. You’ll need ptrace() and wait() for the breakpoint insertion and signal catching. Just copy the functionality from the Win32 version.”

3.

An brief word from the team about how debuggers work.

The thing you most want to do with a debugger is set and handle breakpoints. On X86, there are two kinds of breakpoints: hardware and software. You mostly use software breakpoints. They way software breakpoints work is, you pick the place in the program you want to break at, and you replace the instruction at that point with “INT 3″ (conveniently enough, this is just the byte “0xCC”). When the program hits the INT instruction, it generates an interrupt. The OS catches the interrupt and kills the program.

Unless you have a debugger attached. If you have a debugger attached, instead of killing the program, the OS tells the debugger. The debugger then swaps the original instruction back in, “rewinds” the prograam back to it, and resumes execution.

Every OS has debugging features. They boil down to the following four capabilities:

• Reading and writing the memory of another process (that’s how you swap INT in for instructions to set breakpoints).

• Catching events from other processes, like breakpoint interrupts.

• Starting, stopping, and pausing threads inside other processes.

• Changing the register state in other processes, for instance by moving the EIP register back 1 byte to rewind the INT 3 instruction that just fired.

The best known Unix debugger interface is ptrace(), and it basically does all four of those things for you, along with the wait() call for detecting events. On Win32, any program can read or write from a process it has the right permissions for, even if it isn’t a debugger; the debugger mostly exists to catch interrupts.

4.

Coding the wrappers for ptrace(), wait(), and waitpid() didn’t take too long. Each just takes a few integers and returns an integer. But ptrace works with request codes, like “PEEK” to read memory or “STEP” to single-step the process. I couldn’t test without knowin all the request codes. So, I started reading man pages, poking at code and trying to get my OS X functions to work.

“To the headers!” I cried. But which one and where are they? As I mentioned, I’m a little new to real — as in non-academic — programming. Google worked OK to get the man pages, but didn’t include the request code numeric values, just the names and what they did. Frustrated, I asked for help.

“find /usr/include | xargs grep ptrace | less” was the response I got from Thomas. You didn’t know he speaks *nix? He does. Hexadecimal too, from what I’ve heard.

A little reading and some copying later I had the constants I needed, and began to test my ptrace and wait functions. The code wasn’t pretty but it seemed to work. I could attach to a process by PID and wait() for it. Now I just needed to get its registers and I’d be almost done.

It didn’t take long to sketch my code based on the Win32 debugger I was given to start with. Soon I had what I thought was the start of a functional debugger in Ruby, along with a handy explanation of the Ruby way of doing things. Up until that point I’d been trying to do things the C way, passing variables by reference, trying to make the Ruby function call an exact match to the C call, and other things I’d picked up from the C/C++/JAVA I learned in college.

I thought I was doing well. Then I tried to find the OSX equivalent of PTRACE_GETREGS to read the registers from other processes, which is kind of important for debuggers.

5.

Here everything starts to get more complicated.

It turns out Apple, in their infinite wisdom, had gutted ptrace(). The OS X man page lists the following request codes:

• PT_ATTACH — to pick a process to debug

• PT_DENY_ATTACH — so processes can stop themselves from being debugged

• PT_TRACE_ME — so debuggers can launch processes that start debugged

• PT_CONTINUE — to restart a program after it’s been stopped

• PT_STEP — to execute just one instruction in the process

• PT_KILL — to kill the process

• PT_DETACH — to release the process

No mention of reading or writing memory or registers. Which would have been discouraging if the man page had not also mentioned PT_GETREGS, PT_SETREGS, PT_GETFPREGS, and PT_SETFPREGS in the error codes section. So, I checked ptrace.h. There I found:

• PT_READ_I — to read instruction words

• PT_READ_D — to read data words

• PT_READ_U — to read U area data if you’re old enough to remember what the U area is

• PT_WRITE_I — and write instructions

• PT_WRITE_D — and data

• PT_WRITE_U — and U

• PT_SIGEXC — and EXC SIGs

• PT_THUPDATE — and update THs

• PT_ATTACHEXC — and attach EXCs

There’s one problem solved. I can read and write memory for breakpoints. But I still can’t get access to registers, and I need to be able to mess with EIP.

That’s when I start hearing “It has to work, otherwise gdb wouldn’t”, rather frequently, from more than one person.

Well, ptrace() won’t work for retrieving registers in OS X.

Matasano Secret Intern X referred me to Nemo’s article at uninformed.org. In it, Nemo lays out the Mach kernel calls that replace some of the lost ptrace() functionality. So, I wrote wrappers for:

• task_for_pid — to find the Mach task of an OS X process

• mach_task_self — to get my debugger’s task

• task_threads — to walk the threads inside a task

• thread_get_state — to get the registers for one of those threads

• thread_set_state — to change those registers

Since I wasn’t using them natively in C I needed to know more about the usage of each function.

“No problem,” I thought, “I’ll just fire up terminal and… Oh, bloit!” No man pages.

I pored over Nemo’s work, what I could find in the headers, and figured out how to call the functions. Now another problem. The Mach functions take pointers to raw C memory.

The way I was told to handle this was, pack the data I needed into Ruby strings or native numeric types with Ruby/DL. After a long, dark period of messing with calls to “strdup” and “DL.malloc”, I found “String#to_ptr”, and at last managed to get the Mach functions working.

I had also found the correct way to get errno through Ruby/DL: DL.last_error. This appears to be documented nowhere in English.

Except for an odd bus error I ran into now and then (but couldn’t duplicate), my Ruby debugger was working and could read and write registers. I’d even checked to make sure they were coming back to me in the correct sequence.

Then, running my get_registers() function repeatedly, I found the registers of a stopped process changing on every call. When I printed them without marshalling they contained the names of some of the functions I’d written occasionally.

“Oh, bloit! I’m really chakked now. I’ve been calling a bloitting buffer overflow a register lookup,” I said to myself. I despaired of my project and my future.

6.

On the train home and all weekend I looked through Apple’s documentation. Google. The header files “It has to work; Otherwise gdb wouldn’t,” another friend said. But he wasn’t able to find the documentation I was looking for. He did find fxr.watson.org and some better explanations of the functions at web.mit.edu/darwin/src/modules/xnu/osfmk/man/. Those turned out to be gold later.

During week one of coding:

• several necessary functions wrapped and working

• DL.txt is really the only Ruby/DL documentation that exists

• Ruby/DL is great for simple C function wrapping but rough around the edges when it comes to more interesting calls.

• Avergage familiarity with Ruby

• Basic understanding of how a debugger works

• A Ruby object that can attach to a process, continue it, detach from it and wait() for it.

• One really convoluted method to read/write random locations in memory

• Average familiarity with system calls in C (now rust free)

7.

Starting the following week, things went a little smoother.

I had my coding flow going. I had better documentation than just header files. I started reading the Mach kernel code.

I wrote a small program in C to test the sequence of system calls I was using in Ruby. If It worked in C, why didn’t it work in Ruby? Then, I found it. I was calling task_threads() wrong, passing an pointer where it expected a pointer-to-pointer. Whee! I vetted the results with gdb’s output.

My code said:

gdb replied:

They agreed! I went home for the day.

8.

Now for wait(), to catch debugger events. wait() was hanging the debugger if I called it more than once. I set it up to use the NOHANG option. I fixed an return value error.

Then, I tested single-stepping with ptrace. Kernel panic.

I put that on the list of broken parts of ptrace to be replaced by a Mach call.

Next up was setting breakpoints. They seemed to install themselves without error but the child wasn’t stopping when ran the command that would hit the breakpoint I’d set. Upon inspection, the breakpoint was replacing an instruction of -1. Which gdb told me was actually 0x55.

I started researching the problem, finding only hints. Did I mention ptrace was gutted in OS X? I read the source for Apple’s version of gdb. Thomas gave me a copy of a DTrace truss and said, “Just do whatever gdb does.”

It took me a while to get the script working. It seems iTunes causes errors in truss (also dtruss) whenever it’s running. I closed iTunes and started using watching gdb for ptrace calls. Rather quickly I noticed an extreme lack of call to ptrace.

Was gdb even using ptrace for reading the process’ memory?

It became apparent ptrace was only really used by gdb to:

• prevent the process from exiting on signals

• passing signals to the child after it processed them.

I then remembered that uninformed.org article. A quick read reminded me that Mach vm_read and vm_write were needed to replace PT_READ and PT_WRITE.

The next day, Thomas was in the office to check on my progress. To move things along he implemented vm_read and vm_write for me while I confirmed a few things with truss and looked for vm_read calls in gdb. I didn’t find any. When he finished the functions, I used them in my breakpoint setting routines. No errors.

No stopping at breakpoints either.

Again the instructions were -1. When I mentioned this Thomas informed me I’d probably need vm_protect as well. Why hadn’t I thought of that? Not too long after that I was able to set and remove breakpoints correctly! I went home for the long weekend.

During week two of coding:

• wrapped and implemented all necessary system calls

• added thread state and breakpoint manipulation to Debuggerx

• gained some knowledge of OS X internals

• found a repeatable kernel panic

• learned basic usage of dtrace and gdb

• learned I tend to overthink my code before writing it

• began to use irb as a scratch pad for testing functions

9.

Now another problem. You can set a breakpoint with the debugger. You can catch the breakpoint. You can resume the process. But you can’t reset the breakpoint without single stepping: to resume the process, you have to clear the breakpoint.

But PT_STEP was panicking the kernel!

I settled on setting the TRAP flag in the EFLAGS register to simulate single-stepping with ptrace. This seemed to work. But now I’m getting bus errors when I resume the process. I verified with Thomas how they were supposed to work. I tried watching gdb for vm_write from truss again, nothing. After some debugging I discovered waitpid() was clearing the trap flag, which Thomas informed me was correct behavior. Some more monkeying around trying to get it working ate up the rest of the day.

The next day, I was able to pass through a breakpoint and reset it. Only problem was, the breakpoint wasn’t being reset fast enough, it wasn’t done immediately one step after it was hit. After clearing some confusion on my part with Thomas, I decided to try PT_STEP again. It worked and didn’t panic the kernel this time. Finally, I had a debugging tool that was complete!

All that remained was to clean up some debug tracing prints and implement a better method to view the registers. Both fairly simple things completed early the next day.

10.

There it is, the story of the birth of DebuggerX. A “simple” porting task handed to an intern to better his understanding of debuggers and Ruby. During the project I’d become quite familiar with Ruby, learned some OS X internals, found a kernel panic in ptrace, and learned better programming technics. I still tend to overthink my code and “have a hard time believing that you’re supposed to ask programs to do the things it looks like they need to do,” according to Thomas, but I have learned it’s quite a bit easier to try something in code than in your head. Since completion of the project as originally stated, I’ve added calls to get information about a thread and began looking into retrieving a list of function symbols from the process’ file. I’ll make another post about that in the future.

43 Comments

Patch Your (non-DJBDNS) Server Now. Dan Was Right. I Was Wrong.

Thomas Ptacek | July 9th, 2008 | Filed Under: Feature, Uncategorized

Thanks to Rich Mogull, Dino and I just got off the phone with Dan Kaminsky. We know what he’s going to say at Black Hat.

What can we say right now?

1. Dan’s got the goods. You know that scene near the end of High Fidelity where Jack Black listens to the skate punk’s electroclash demo? Yep. It’s really f’ing good.

2. This is strong year at Black Hat: Dowd and Lawson in particular have awesome talks linked up. But Dan may have Best of Show here.

3. If you were running DJBDNS before, you’re safe. If you made fun of me for running DJBDNS: sucker.

4. Ryan Russell pointed out earlier on our blog that Dan takes a lot of crap for doing so much public research. You can’t be in the public eye for long without taking fire from people who write shellcode instead of Black Hat talks and press releases. Ryan is right: it’s not fair. I don’t know how you can give Dan crap about his work after this.

I think Dan should come clean on this and publish the details. The 30 days he’s given before Black Hat won’t make much of a difference. But his reason for not doing it is at least plausible. And he did the work. So, it’s his call.

I think I owe Chris Eng $100 now.

104 Comments

Dan Kaminsky could have made hundreds of thousands of dollars with this DNS flaw!

Thomas Ptacek | July 8th, 2008 | Filed Under: Uncategorized

We interrupt this pile-on to bring you this important message: Dan has the goods. Patch now, ask questions later. More to come this evening.

Java JSESSIONID: BB16479A0338D3DCF26D11712F138BC1

.NET ASPESSIONID: HHODHGFDJOJAKDIPPJCKHGOE

SiteMinder SMESSIONID:

su/hxP2nLeaZBdEn8qClOdeCGwG2xfLaBfXQF2QpSCSxKYBLVTF7OfqtVcHxLITpuNa6+1W c2ZJ9MKWInlFlEe5GqZAjobgyzInCwe3JiTebqyJaftWtVht/La0qlvjLF9oaI5y1aIdtUGiTmQI OW28AL0gLJe4pdA0sw2fq4cBG8ZWPMblwX4nGCGXGU8JQ1PtOhm8ohtSQcXZ7lm35t29 P5tcbfDrQs3z4g43zrLRO5M68m91xP7xcHY0uLuSYUSMFIrUbaEVSVVewFY4tskjPYecoWT uLV0deSJilKpfSTVyekbzGXO2ejhIPxsE5cvPVNPt5AoJ6KIdvWMezUHz+KQt3uVuJEHpZkU QhEfLrWAdJ2TwE++na2G3GI8BqlSOB+KRl3rz19/9nqpE87c/IWsscSfOQLemzwd/Z3DZfn ioKB/tFsZWLndqdNq5XmDuRvRN2+EVMT8QFYEq1c+mNhsOIeFCjo8JOOXPG3F+r6h0kXN M4zjRtgN/qSYRAycXluqKozAIMgr5qemW1UItwCyqJu1cDMLuKgkSq9XXA3Cru6PVPF74D1 t8l2IvV2HWmxL2PP4RdIXa5Ofb1sCLc6AUZ9opLGhwYHt7S3PnxXzKoYsMJwoFm7nGqjKp J7S9e0iRTMUqY7fOgSQALLw+hsac7hhNCUtB3/xEhvfJ7Y4b1Xj26jWJAujEnHgF+DUJQHvX hkLl7Rr2dbCPJu/8hDMOKdfz4QJXAQSbCJyA4MrJLXn4UZLpgwMeIVMddvloO4dZatrxQT9m ZQtqvow5jKcpUKhtxqqf7M4MFDMOEvQdIT3U8WRsbjk1lT4UajljxyTa9TSF9sCid1BH/O3Hq YyJtfpDcr7QxqHXr9AZYtHbO93DX/I82bQ3mcCco

DNS XID: 04d8

Getting To File This Week’s Front Page Security Story Before Changing Out Of Your Pajamas: Priceless.

There are some vulnerabilities money can’t buy. For everything else: there’s the DNS.

70 Comments

Hello! A self introduction by Chris Rohlf

Chris | July 7th, 2008 | Filed Under: Feature, Uncategorized

Hello! My name is Chris Rohlf and I just joined the Matasano team. I will be working on various different client projects from the NYC office. I have spent the last 5 years in the security world working on various things from R&D to operations at both government and corporate organizations. Most of my interest and work has revolved around reverse engineering, and vulnerability discovery on different platforms. Some of you may have read my personal blog in the past [EM_386] where I blog about reversing, vulnerabilities, programming and my general security related opinions.

Most recently I worked in an operational role where I got to see first hand the security threats and challenges that a large enterprise faces every day. But now I look forward to putting my prior research experience to work at Matasano.

12 Comments

Ruby for Pentesters #1: Use Modules For Lists Of Constants

Thomas Ptacek | July 3rd, 2008 | Filed Under: Uncategorized

Almost 2 years ago, Dino declared Python to be the “lingua-franca of over-the-hill hackers”, boldly asserting that 5 out of 6 security hackers under the age of 30 preferred Ruby instead. Being 30 at the time, I was an easy psychological target for this argument. I made the switch and haven’t regretted it. You can tell me all you want that “named nested functions are just as good as lambdas”, or that “you can fake Ruby blocks with a for loop and a generator”. Ruby is just nicer to write testing code in, and makes me feel at least 2 years younger and less experienced than I really am. Thanks, Ruby!

I’ve been meaning to write a long post about our house Ruby style, and some of the Ruby tips and tricks we’ve picked up along the way. But every time I sit down to write it, that post starts sounding a lot like work. So instead, I’d like to inaugurate a new series of much easier posts: Ruby for Pen-testers.

Where was I?

1. Use Modules For Lists Of Constants

If you test protocols or C code, you run into lists of magic numbers all the time. For example, here’s a bit of ptrace(2):

This is gross, but it’s C code, so you give them a break. But here’s some code from Pedram’s PyDbg:

Now, Pedram does have the excuse of writing in Python. But here’s Ruby-MySql:

This code has no excuse. (Here’s a rewrite that is much faster). Now, let’s look at net-ssh; if you haven’t read Jamis’ net-ssh code, you shouldn’t write any more packet processing code until you do.

Getting closer. But not there yet. Here’s an even better way:

That’s right: one module per set of constants. In other words, substitute “module” for “enum”. This has many benefits:

1. It’s clean. You can immediately find all the related magic numbers, both from the list, and by looking at code that uses the magic numbers —- you see Ragweed::EFlags::CARRY, you know to look for “EFlags”.

2. Modules come with special bonus features.

For instance:

… which is super nice when you’re printing out the contents of packets.

14 Comments

In Which I Resolve A Titanic Semantic Conflict

Thomas Ptacek | June 22nd, 2008 | Filed Under: Uncategorized

Chris Eng says, there’s a “gut feel” component to penetration testing that can’t be replaced with checklists. It’s an art. Adam Shostack says investigating your “gut feel” is science, if you’re doing it right. Chris and Adam aren’t talking about whether security is an art or a science. They’re talking about whether it’s engineering. I think they’d both agree, it is not.

52 Comments

ChiSec 18 is NEXT WEDNESDAY, with a NEW LOCATION.

Thomas Ptacek | June 20th, 2008 | Filed Under: Uncategorized

ChiSec 18 is Wednesday. This post is a shameless rehash. All the light and heat behind ChiSec has moved, from our blog to CITYSEC.ORG, which is where all the cool kids hang out now.

The details on the web page are up to date, and you can forward that URL to your friends and neighbors. There’s a mailing list; instructions are on the web page. If you join it, you won’t find out about the next ChiSec any faster than our readers, but you will get a say in when and where they’re held.

Some things to know about Chisec:

• No vendors.

• No 45-minute talks.

• No RSVPs.

• No membership.

• No free drinks.

I expect between 20-30 people. You will find:

• Operations people dealing with network security in the trenches

• Security product developers

• Vulnerability researchers

• Consultants and pen-testers

• Management and policy guys

Some of them even have CISSPs, and yet I don’t throw little temper tantrums about it. That’s how cool these meetings are. And all you have to do to participate is show up.

2 Comments

Updates on Drew Yao’s Terrible Ruby Vulnerabilities

Eric Monti | June 20th, 2008 | Filed Under: Uncategorized

[Update: Hello Slashdot and Reddit Readers]

Here’s what we think you should know about the Ruby vulnerabilities:

There are at least three easily exploitable vulnerabilities in MRI, the de facto standard Ruby interpreter. Matasano didn’t discover any of them —- Drew Yao, from Apple’s security team, did the heavy lifting. Reviewing the code for the MRI interpreter is tedious and difficult work, and Drew deserves your thanks for finding these problems and handling them professionally.

All three published vulnerabilities allow normal Ruby code to be coerced into corrupting the memory of the interpreter. They involve integer handling errors in the native code backing Ruby’s Array, String, and Bignum classes. These are core classes in Ruby, and don’t depend on the libraries or extensions that programs load. The vulnerabilities, which are common to all large C codebases, exploit the fact that numbers “wrap” when they hit the largest size allowable by the CPU (usually 32 bits), and the fact that negative numbers and very, very large positive numbers share the same underlying machine representation.

The ability to overwrite memory in the Ruby interpreter is basically the same thing as the ability to upload native machine code into the interpreter. There are thousands of locations in memory that, if overwritten, can redirect code to unsafe libraries or directly into input buffers (such as the contents of web requests). The conditions under which the vulnerabilities are exploitable depend on the Ruby programs you are running. But don’t gamble. Update as soon as you can.

There’s a discussion on the Ruby Forum site about the issues people are having upgrading. We’ll try to track the details of the actual vulnerabilities here, but we’re security people, not Ruby release managers, so you’ll want to direct questions about patches there.

[Resuming regularly scheduled blog post]

Several of us at Matasano felt a chill up our spine seeing this news today. If you do any serious work with ruby, you probably should have too.

Uh… sounds not so good:

With the following vulnerabilities, an attacker can lead to denial of service condition or execute arbitrary code.

• CVE-2008-2662
• CVE-2008-2663
• CVE-2008-2725
• CVE-2008-2726
• CVE-2008-2664

The CVE descriptions are just in “reserved” state, no details at all yet. So, we took a quick stroll over to the ruby subversion tracker and poked around a bit to look for some clues. Clue 1, Drew Yao of Apple Product Security apparently found this stuff. Credit goes to him:

Warning, viewing these patches may not be suitable for young adults and small children. Ominous stuff here.

Revision 17460

* array.c (ary_new, rb_ary_initialize, rb_ary_store, rb_ary_aplice, rb_ary_times): integer overflows should be checked. based on patches from Drew Yao <ayao at apple.com> fixed CVE-2008-2726

* string.c (rb_enc_cr_str_buf_cat): fixed unsafe use of alloca, which led memory corruption. based on a patch from Drew Yao <ayao at apple.com> fixed CVE-2008-2726

… ouch!

A little while later we’re playing around in IRB:

Playing with arrays:

irb(main):001:0> ary = []
=> []
irb(main):002:0> ary[0x7fffffff] = "A"
(irb):2: [BUG] Segmentation fault
ruby 1.8.6 (2007-09-24) [universal-darwin9.0]

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x00500000
0x000c106d in rb_ary_store ()
(gdb) x/i $eip
0xc106d : movl   $0x4,(%edx,%eax,4)
(gdb) p/x $edx
$1 = 0x444340
(gdb) p/x $eax
$2 = 0x2ef30

Playing with strings:

irb(main):001:0> str = "A"*(2**16) ; nil
=> nil
irb(main):002:0> while 1; str << str ; puts str.size; end
131072
...
1073741824
(irb):2: [BUG] Segmentation fault
ruby 1.8.6 (2007-09-24) [universal-darwin9.0]

Abort trap

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x41008000
0xffff132f in __longcopy ()
(gdb) x/i $eip
0xffff132f <__longcopy+303>:    movntdq %xmm0,(%edi,%edx,1)
(gdb) p/x $edi
$1 = 0x41040000
(gdb) p/x $edx
$2 = 0xfffc8000
(gdb) 

Why is this so disturbing? These vulnerabilities are likely to crop up in just about any average ruby web application. And by “crop up” I mean “crop up exploitable from trivial user-specified parameters”. It’s not hard to begin imagining cases where Ruby/Rails programmers use code similar to the samples above to routinely handle user input. Unlike un-handled ruby exceptions getting raised, these bugs aren’t the fault of the programmer as much as the fault of the interpreter. Part of the unwritten “contract” with your interpreted language is that it will prevent you from letting ridiculous things happen by raising an exception.

Weaponizing these for code-execution may or may not be trivial (we’re looking into this too, — keep you posted). But even a class of DoS attacks this trivial would be horrible enough.

We’re still looking into this and will update as new info becomes available.

[Update]

The comments are pretty interesting. Ruby missed one of Drew’s patches, leaving p22 still open to the string case; it’s been fixed since. There’s also a Bignum example case.

41 Comments

De-Obfuscation For the Impatient

Eric Monti | June 2nd, 2008 | Filed Under: Uncategorized

A classic challenge for companies that build products on high-level languages like Perl, Python, PHP, and Ruby —- as well as .NET, and Java —- is that they are shipping their source code to their customers. Companies don’t like disclosing their source code. Language vendors want companies to use their stuff. And so we have a variety of different schemes that try to “protect” source code so that only the language runtimes can use them. Some of them, like Python’s bytecode compilation, are “built in” to the language. Others are add-on packages and extensions.

From what I can tell, they all work about equally regardless of bells and whistles. Not too well.

Take “Eve Online”, an MMORPG. Eve’s client is written, in part, in Python, and shipped in byte-compiled files. Recently, a player named “Abuser” reverse-engineered and decompiled those files, discovered some vulnerabilities, and developed a cheating bot. For MMORPGs, cheating bots are a big deal. You can read online discussions between the game developers and Abuser, posted publicly. “Abuser” went public about his efforts, but he was far from being the first to do this to Eve Online.

Recently, I needed to get access to some obfuscated Perl code. My target was “encrypted” using Perl “source filters”. Unlike Python compiled bytecode, source filters actually obfuscate regular Perl source code by encrypting it then decrypting at runtime. Breaking Perl source filters is pretty easy for anyone who knows how to use a debugger. I’m going to show you how I went about it, using Immunity Debugger and then automated it with Paimei’s PyDbg.

First, a word about Perl source filters. Here’s an excerpt from “man perlfilter”:

NAME
perlfilter - Source Filters

DESCRIPTION

This article is about a little-known feature of Perl called
source filters. Source filters alter the program text of a
module before Perl sees it, much as a C preprocessor
alters the source text of a C program before the compiler
sees it. This article tells you more about what source
filters are, how they work, and how to write your own.

The original purpose of source filters was to let you
encrypt your program source to prevent casual piracy.
This isn't all they can do, as you'll soon learn. But first,
the basics.

...

Decryption Filters

All decryption filters work on the principle of "security
through obscurity." Regardless of how well you write a
decryption filter and how strong your encryption
algorithm, anyone determined enough can retrieve the
original source code. The reason is quite simple - once
the decryption filter has decrypted the source back to its
original form, fragments of it will be stored in the
computer's memory as Perl parses it. The source might
only be in memory for a short period of time, but anyone
possessing a debugger, skill, and lots of patience can
eventually reconstruct your program.

That said, there are a number of steps that can be taken
to make life difficult for the potential cracker. The most
important: Write your decryption filter in C and statically
link the decryption module into the Perl binary. For further
tips to make life difficult for the potential cracker, see
the file decrypt.pm in the source filters module.

At least they’re pretty honest about the limitations, pointing out that a determined attacker is inevitably going to get around any source encryption you can come up with. That last paragraph bugs me, though. They start out up-front about the limitations, they should just leave it at that. Why the caveat about making it difficult when they obviously know it’s a losing battle. Other things that specifically bother me include the statements “make life difficult” and “lots of patience”.

Also, the bit about the decryption filter being written in C and statically linked is, in reality, a relatively minor obstacle. Not to mention, most people are likely to ignore this man-page and use 3rd party builds of Perl for Win32 which are almost always built dynamically linked for most of their functionality.

That said, my target happened to be a customized source filter using proprietary encryption through a dynamically linked library on win32.

So how does one go about de-obfuscation? Better yet, how does one do it equipped with very little patience?

You can approach the problem different ways:

1. You could attempt to identify the encryption algorithm and find where key material is stored. This has the advantage that you can then decrypt code independent of the architecture and/or harness that runs it (i.e. Python, PHP, Perl interpreted languages). My experience is that this requires more “patience”.
2. Use a debugger and find the inevitable place in the harness where decryption occurs and just read the cleartext from a register. This ties you to the architecture and/or harness for the code somewhat more, but it lets you not worry about the encryption and just cut to the chase. In some cases, you may even be able to re-use the same harness for several different variations of a certain type of encryption.

I tend towards the latter approach myself. Side benefit: There are so many crackpot obfuscation encryption schemes. Not having to look too closely at them is better for one’s health.

In the case of Perl, the convention for source filtering is standard enough that my attack stands a good chance of working other filters built similarly with little if any modification.

Refer again to the perlfilter man-page:

The source might only be in memory for a short period of
time, but anyone possessing a debugger, skill, and lots of
patience can eventually reconstruct your program

Just to reiterate… “patience” is not one of my virtues. Friends and family remind me of this frequently. I’ve distilled the steps I took to those that really count below, but all in all, this took me an afternoon or so.

• First I take a look at perl58.dll. I used a freeware tool called dllexp (DLL Explorer) to examine the functions that are exported from this DLL. Some likely candidates pop right out:

• Next I try running an encrypted script through perl.exe with the “-c” flag using a debugger. The “-c” is Perl’s syntax checker, it’ll let me hone in just on the obfuscated code in a specific file. I decide to use Immunity Debugger, since I’ve been meaning to make the switch from OllyDbg for a while now. I’m still a bit of a win32 debugging noob, but have gotten somewhat familiar with OllyDbg. It’s an easy enough transition to Immunity since it is basically a Python version of Olly.

• Run “ImmunityDebugger.exe perl -c obfuscated_file.pl”

• Debugger comes up. Set a breakpoint on Perl_filter_read. By default, Immunity starts debugging at “WinMain()” aka “ModuleEntryPoint()”. At this point, perl58.dll has already been loaded. I can use the symbolic name setting my breakpoint with “bp Perl_filter_read” in the debugger command-line. Yay!

• Tell the debugger to continue execution. My breakpoint on Perl_filter_read hits. Good sign.

• Continue till return. Examine the registers. Nothing jumps out just yet. The first hit on my breakpoint may be too early to rule this function out though.

• Repeat the ‘continue’ and ‘continue till return’ sequence a few times. Lo and behold… I start seeing what looks like a line of “#” characters getting built up at EDX. Like a line of “####…” programmers like to use to make their comments look pretty. The comment header line is apparently being built up as this function gets called several times:

and then…

• I keep this up until I see actual comments and code. The further I go, the more the Perl code makes sense. Since I ran with “-c”, there’s no doubt in my mind this is actual code from the obfuscated script.

At this point, the rest is just refining and automating my steps. I could probably have written an ImmunityDebugger plugin or macro in python, but I’m more familiar doing this using PaiMei’s PyDbg, so I decided to use that instead. Need some more info before I code this, though:

• Fiddle around some more in the debugger examining the different states of EDX and other registers at several breaks. I’m specifically looking for whole lines or chunks if possible. Perl_filter_read appears to be called recursively decrypting in chunks until it has a full line of Perl code to return to its caller.

• I keep continuing in the debugger until it looks like there’s a complete line at EDX at the RETN at end of function. When I find this point, I step back into the caller. Set a breakpoint at the very next instruction *after* the CALL to Perl_filter_read. Take a look at the surrounding code while I’m at it and step a little further down to some interesting looking MOV statements.

• EDX isn’t really where the code is supposed to get returned to the caller. It’s where the code is being built up and might not be consistent enough to rely on when the function returns. I’m better off determining the “proper” return value, I think, and relying on that instead. Plus: It’ll improve my understanding of the code somewhat and may help me port to another source filter or harness sometime later if I ever need to (a long shot, admittedly).

• Take a look at this snippet of code I googled for Perl_filter_read:

Int32
Perl_filter_read(pTHXo_ int idx, SV* buffer, int maxlen)
{
return (
(CPerlObj*)pPerl)->Perl_filter_read(idx, buffer, maxlen)
);
}

• See how EAX gets pushed onto the stack and passed as “buffer” in the last figure? See how ESI got unrolled to back to EAX? See How I’m sitting right at the spot to receive it in EAX? I set a breakpoint right here and disable all my others.

• Now, as I continue and break at the new spot, I’m just seeing the comments and code line by line every time. The code is appearing at several registers, but EAX is the one that I sense I should probably use. As I start combining lines of snagged code in a text editor, it’s definitely functional Perl. This is where I want to break in my PyDbg script.

• Finally, using all this information, I write a quick script for PyDbg. It just automates all the steps I took and snags the value in memory from the EAX register at that last breakpoint. Here’s the code I ended up with. When run, it just dumps out the cleartext line by line.

What have we learned?

Well, obviously, Perl source filters are just not strong protection for your source code. If you know what a CPU register is, and you know how pointers work, it’s a lot easier than the man-page implies to pinpoint the place in the Perl interpreter where cleartext source-code can be found in memory. I’ve even produced a fairly simple script that does just that.

But really, this shouldn’t be news to you. And, how likely are you to run into obfuscated Perl? Not terribly. My point isn’t that Perl is weak. The problem with Perl source filters is an example of a problem that afflicts all high-level language runtimes: if you want your code to run, you have to expose it to the interpreter. On general purpose architectures, you can’t expose something to the interpreter without exposing it to everybody else.

If you’re shipping high-level source code in any form, including bytecode, self-hosted executables, or encrypted bundles, you’re ultimately shipping your source code. Get used to that idea, or go back to writing in C.

16 Comments