Filesystems Fall To Primitive Fuzzing Tools
Thomas Ptacek | October 26th, 2006 | Filed Under: Development, Disclosure, Uncategorized
From DailyDave this week, a discussion about “fsfuzz”, a filesystem fuzzer ostensibly for Linux, supporting most of the major filesystems. I was working on an NTFS fuzzer this week. You care about filesystem fuzzing:
because filesystem bugs mean people can turn $10 thumb drives into “install virtualized rootkit” kernel exploit dongles, and
because SAN-style block-based storage protocols mean that the network controls the filesystem metadata, and you need to survive that attack.
So someone did the work for me. Nice! Let’s check it out.
Hmm. Um. It’s a shell script around mkfs and a C program. The C program doesn’t look big enough to know how to encode NTFS attributes. There must be a pony in here somewhere. No; now I’m confused. All this thing does is spray random bytes over arbitrary files. This works?! Apparently so. From the comment, it broke:
- libmagic
- OS X Preview.app
- xpdf (hang)
- Mach-O loading (“osX 10.3.7, seems to be fixed later”)
- QNX’s ELF loader (“panics almost instantly, yikes!”)
- FreeBSD’s ELF loader
- OpenOffice
- amp
- OS X DMG loader
- binutils libbfd (so, objdump, nm, and gdb)
- libtiff (used tiff2pdf)
- xine (division by 0, took 20 minutes of fuzzing)
- OpenBSD’s ELF loader
- Unixware’s ELF loader
- DragonFlyBSD’s ELF loader
- Solaris 10’s ELF loader
- cistron-radiusd
- linux ext2fs (2.4.29) (“division by 0”)
- linux reiserfs (2.4.29) (“instant panic!!!”)
- linux jfs (2.4.29) (“long (uninteruptable) loop, 2 oopses”)
- linux xfs (2.4.29) (“instant panic”)
- Win32 Flash .SWF
- Quicktime 7.0.1
- totem
- gnumeric
- VLC
- MPlayer
- Python’s bytecode interpreter [!]
- RealPlayer Gold 10.0.6.776
- dvips
So, uh, excuse my language but how fucked up is _this_?
Don’t get me wrong; I’m not criticizing LMH (who wrote fsfuzzer) or Ilja van Sprundel (who wrote mangle, the small C program —- though I am giggling that he opens “urandom” just to generate random fuzzer bytes). The fact that I wasted an afternoon making my tools grok the NTFS MFT means the joke here is very definitely on me. (And yes, the irony of mocking someone for writing shell script instead of C, not lost on me).
No, the problem here is that in 2006 you can apparently rip a large, ugly gash through the whole industry with nothing more than a blind random number generator.
There needs to be a word for this. Our field needs a lot of new words, but we need to start here. What do you call a piece of code that falls to “mangle.c”? Dave says it should be German, proposes “poopensoft”. Dave, that’s not German. “Scheissekraft” captures the spirit, and I feel harsh, but let’s be politic and call it “gruenewerk”. Green. Not ready. Vendors, developers, can you please run the random number generator on your code _before_ you ship?
LMH is going to try to run the “Month of Kernel Bugs” (ala HDM’s MOBB —- GPG key evidently forthcoming). Unfortunately, I think it’s going to be a success for him. Nicely done, LMH. But I feel a bit sick.
October 27th, 2006 at 10:01 am
As a native german speaker: Wtf are those words supposed to mean ?
Poopensoft –> Crap software ?
Scheissekraft translates to “Shit Power”, I am
sure that is not what it was supposed to mean.
Gruenewerk –> Green work ?
October 27th, 2006 at 10:47 am
Halvar is always trying to bog us down with “facts”.
October 27th, 2006 at 10:55 am
“Powered by shit” seems exactly appropriate. Scheissewerk is a bit less euphonious. But your beef is with Dave, for saying the words should be German; if “gruenewerk” means “green work” and not “jelly donut”, I’ve succeeded admirably.
Does “green” not connote “immature” in German culture?
October 27th, 2006 at 12:03 pm
[...] If you just want to find crashers in your OS, download fsfuzzer and mount up filesystems it creates for you. You won’t need to think, and you will probably find bugs. Fsfuzzer is the simplest thing that can possibly works, and that’s clever, and I salute it. [...]
October 28th, 2006 at 1:34 am
it is a symetrically encrypted no key forthcoming
$ gpg –list-packets -v -v fs-bugs-23-10-2006.txt.asc
gpg: armor: BEGIN PGP MESSAGE
gpg: armor header: Version: GnuPG v1.4.5 (GNU/Linux)
:symkey enc packet: version 4, cipher 3, s2k 3, hash 2
salt 3e453cda99dd7b68, count 96
gpg: CAST5 encrypted data
:encrypted data packet:
length: unknown
gpg: encrypted with 1 passphrase
gpg: decryption failed: bad key
October 28th, 2006 at 6:08 pm
Filesystem… but they nailed ELF loaders? Is that a fuzzed ELF file on a valid filesystem, a valid ELF file on a bogus filesystem, or fuzzed both?
October 28th, 2006 at 6:12 pm
The ELF stuff is from “mangle”, the trivial C program which LMH borrowed to do “fsfuzzer”. “fsfuzzer” is itself a Bourne Shell script that mkfs’s filesystems, incurs a bunch of fsops, and then sprays garbage over the resulting image.
October 28th, 2006 at 6:21 pm
Let me rephrase my question. Did OpenBSD fall over for a mangled ELF file on a normal filesystem last week?
October 29th, 2006 at 5:41 pm
Week’s Links…
Passive Application MappingAnalysis of 802.1xFilesystems Fall To Primitive Fuzzing ToolsWTF is NTFS Fuzzing? Matasano ClarifiesRFIDIOt released RFID E-passport skimming PoC10 things you (probably) didnt know about PHP…
October 29th, 2006 at 6:11 pm
Actually, fsfuzzer isn’t just a ’shell script around mkfs’ (thanks to Steve, from Red Hat, it does a good job isolating the test cases. He did a nice usability job, and tested it). The reason behind using a C-based ‘fuzzer’ is performance and portability. Period. Thus, it’s easy to port fsfuzzer to win32, Solaris, BSD and whatnot, in a matter of minutes.
Please read the code in ‘run_test’ file and ‘fsfuzzer’ script itself. Probably you didn’t want to explain how it works technically anyway, but I want to give pointers to people out there that may wonder about how it really works. Besides the ‘oh, it breaks stuff’ thing.
Also, a binfmt-aware fuzzer takes more time to develop and in the end, you either find the same issues or deal with an overcomplicated approach for accomplishing an utterly simple task. Unless the target uses complex validation (not CRC alone, fsfuzzer broke jffs2) or encoded/encrypted streams. In such a case, the advantage of an specific fuzzer is obvious.
You’ve mentioned PROTOS in another post in this blog. PROTOS isn’t ’sophisticated’ at all. It’s the concept that makes it nice. It just delivers a couple test cases using some java code (that could be replaced by netcat and a shell script wrapper
). Of course you need something more complex if the target protocol works over SSL and so on (although you could have binary test cases already anyway).
A side-note: While I have nothing against the now popular ‘blog follow-ups’ (as they reach more people, are certainly more noisy and a bunch more reasons), it would be good if you had sent this through the mailing list at the time I made the announcement. Not everyone reads this blog on a daily basis :).
Regards.
October 29th, 2006 at 10:15 pm
Not everyone keeps up with mailing lists either.
I miss Usenet…
October 29th, 2006 at 10:43 pm
Hey, LMH. Glad you saw the post. Sorry we didn’t cross-post it, but that feels rude (we have a venue, we don’t need to take over Dave’s list at the same time). You should read us more regularly!
Not sure I understand your “portability” point about fsfuzzer:
1. most of the logic is in a Bourne Shell script that uses Linux filesystem commands.
2. “mangle” is not a particularly portable piece of C code (it has a /dev dep and uses mmap).
I’m not advocating doing more or less in C.
Regarding “binfmt-aware” fuzzing: I have some humility about the effectiveness of “blind” fuzzing tools, which you see in my posts. But don’t take it too far. Unless you really believe that, say, a week or two of “fsfuzzer” trials are going to shake out all the ext3 bugs, it seems obvious that “format-awareness” is a necessary next step.
I’m as aware of the simplicity of the PROTOS test-delivery engine as you are of the complexity of the individual test cases, which are enumerated exhaustively from a spec, not generated randomly on the spot.
November 1st, 2006 at 7:44 pm
Tom,
It is quite likely that any new technology or product class will fall down with obvious errors, based on past performance anyway. Didn’t you learn this from all that agent work Matasano’s done?
I’m not overly surprised at this point. A focused effort to identify and fuzz previously-untested product classes would be a boon to researchers and consumers.
November 9th, 2006 at 5:04 am
[...] Many of the flaws found by L.M.H. come from using a data mangling, or fuzzing, tool known as fsfuzzer. Characterized as a rudimentary fuzzer by one researcher, the software creates a file system filled with random data. Subjecting an operating system’s kernel to various permutations of the data in many cases causes crashes, which may be able to be exploited. [...]