crazylazy.info

txt

wishi's Fuzz-Box

A Fuzz-Box for me is a standalone machine. It has to:

• host multiple virtual machines at once (max 2 in my case)
• effectively manage ~4 GB RAM
• be Linux compatible with, stable clean device drivers
• energy efficient and ergonomically able to run 24h/day 7d/week...

Scaling Hardware?

You don't want a performance monster. - Or a gaming machine. And you do not want trash, because you're going to spend valuable time with it.
My old fuzz-box died some weeks ago due... I don't know. Ashes to ashes, trash to trash. But how do I rebuild my low-budget hacking-lab now? If I'd want to spend more money I'd go way over 4 GB of RAM... but it seems a ~350 € Nettop (Acer Aspire Revo R3610 or an ASrock ION 330 e. g.) has an adequate set of hardware to replace the old box. There may be better Nettops. Soon the Intel Atom CPUs used in ION e. g. platforms will support VT-x. And I pretty much guess that'll make setups like this more useable in the future.

Which OS to host all the VMs?

For the host I stick with 64 bit. Furthermore - because I chose it - it needs VMware support. That only left a Debian Linux for me. The Host-OS gets kernel-hardending and all kinds of nifty security stuff because I'll keep it online 24/h a day and it acts like a server.

My security priorities here are:

1. it's not crashing and losing my work
2. it's not a low-hanging fruit because of the confidential nature exploits have for me
3. it remains data-integrity

Extending the Host-OS

The first thing Debian gets is grsec. (paxctl -m for mprotect - but wine may need pactl -smp wine-preloader wine-loader). It gets more common every day because the Linux-kernel developers are sleeping under rocks when it comes to security. The rest is a very minimal setup with Xfce/Xmonad and pretty much anything useable in the backend from ICC to GCC. - You name it.

The CUDA gcc extensions work with gcc ~4.3/2, still unsupported. But that's a hack-pc. I make it work.

VMware offers binary packets. If that doesn't work I fall back to VirtualBox. But I don't want to. Both should run out of the box ;). As it says.

Inviting the Guest-OSes

Do not download any Virtual Appliances. Set them up entirely on your own. Do not download anything hacked. Clean standard setup, no configuration on the targets. If you configure a target, snapshot the VM before you work on. And do not store VMs you use on a USB harddrive. That crashes these cheap beasts in no time.

The choices of your VMs are very much depending on your targets. My example lab:

1. Ubuntu VM(s) - analyzer, configured - 1x
2. Windows XP VMs - analyzer/target - 2x
3. Windows 2000 SP0 VM - target - 1x
4. Windows 7 VMs - targets/analyzer - 2x
5. misc specific stuff (MetaFuzz, HexLive, DAVIX, SIFT, Samurai WTF...)

You can go into "CTF"-ware like DVL, or other stuff. But in case you like this: do some real war-games and enjoy that. - But do not drop a Linux kernel 0day. That simply sucks because a war-game box is supposed to host vulnerable games. That's like stealing sweets from school kids as an adult. Kids are supposed to have sweets as much as these boxes are supposed to be vulnerable: in a balanced way ;). If you want a kernel-exploit on Linux, use your VMs or CTF-ware.

I mainly use the Ubuntu analyzer VM for cumbersome installations, speaking of Valgrind forks/tools, BitBlaze or VINE, DynamoRIO, PIN, metasm...

The Windows XP analyzer VM is a productive reversing VM, too: for using IDA Pro, Boomerang with radare, Bochs, ollyDBG, byakugan, msf, Immunity Debugger, PIN... I pretty much guess everybody has his own favorite tools and systems. Very important for me for example is Paimei, pstalker and pyDBG: it works on Windows, Linux and even MacOS.
Most code-coverage I do is dynamic, block-based with pstalker. It's far away from perfect. Especially on Windows, where you may want to try out IDA Plugins.

It's important to separate the roles:

• the Host-OS generally isn't used to analyze targets. It may run fuzzers, do some file-fuzzing and hosts a bunch of different compilers. But it doesn't directly run any Malware or vulnerable services.
• The target-VMs host vulnerable applications and are heavily targeted and tortured. If they crash you want to have clean logs without any interfering events. Especially on Windows: keep the machines clean.
• The analysis-VMs run dynamic and static analysis of target-software. They may run a target-software; but they aren't supposed to host it forever. There'll be many interfering pieces of software spamming your logs. So before you go after a target in the analysis-VM you should have done your analysis parts ;). Try to switch the VM.

Sharing between VMs

This is a pretty cumbersome point, and kind of complex.

• I do not want clear-text or NTLM obfuscated streams in my (wireless) home-network. So I mainly use sftp/ssh. The OSes each run an OpenSSH server and optionally rsync for larger stuff (-bwlimit=xyz).
• Windows has got a cygwin-SSH server. And there're Windows Services For Unix mainly as a POSIX compliance extension. - Working on cmd.exe remotely with a TCP bind-shell can be of productive use, too ;).
• An important point here is that I can close the SSH services entirely and close all open sockets within the VMs in order to capture packets. Using included features or SMB/Active directory infests my Pcaps and I hate filtering stuff out, that is encrypted. Especially if my friend Bob is after proprietary protocols/software.
• SSH - Clients generally are available for every OS I use: Dokan SSHFS for Windows is pretty kewl for files, sshfs with FUSE for Unices/Linuxes. lftp knows SFTP, rsync works with SSH...
• In many cases a SSH terminal session on the Host-OS with active screened SSH-sessions to the Guest-OSes is fairly enough. htop, watch and lm-sensors are great helpers. BTW: Monitor your temperatures with lm-sensors for example. - I learned that the hard way. And that stinks for real.
• Each VM gets a unique hostname like: analyzer-ubuntu, or target-win2ksp0, debian-guest... so that I can address them without having to remember a static IP.
• Each target gets a case-folder (stored/commited to the Host-OS). Sometimes I tend to use a local git repository. Each case-folder gets mounted via SSH.
• You can script the VM to transfer logfiles into your case-folder when the fuzzing is ready for example. Automation is a crucial point here. You can shutdown the VM automatically, close services before sniffing. Safes a lot of effort.
• On the Host-OS files are stored in a True-Crypt partition. I do not use dm-crypt or any full-disk encryption for performance reasons. Many reads/writes due many VMs.

Peripherals

Many times I don't use X11 forwarding, and just attach a small TFT. Mainly because I still think the TFT is more reactive while typing ;). Seriously: that may be a scheduling issue.

If I want to login remotely I have my OpenVPN network or use Dynamic DNS entry with ssh'ed X11 forwarding (ssh foo -t screen -rd). And the screen session has all the shells. - That's the reason why the Host-OS has to be secure ;).

The (sometimes useful) remote-GUI forwarding has to leave a minimal footprint. Therefore I don't use stuff that bloats up the whole here. Just rdesktop or VNC.

You can attach a USB harddisk, but only temporarily. Most cheap 1 TB disks you may want to use to store fuzzed files onto die after a week or two. But if you find a way to suspend them... please share your idea with me. Unmounting them isn't enough. They need to be turned off, because they get hot if they're just plugged in.

edit: while I was preparing Christmas lights I found the solution: a clock time switch, synched clocks and the at utility to unmount/remount the devices before switching them of every night.

Known issues

• At least you need a DualCore Atom or something similar. The mentioned models (N300 series) for a Nettop have got: MMX, SSE, SSE2, SSE3, SSSE3, Intel 64, XD-Bit, Hyper-Threading. No VT, SSE4, EIST. That scales with max 2 VMs. You get what you pay for. This is no all-in-wonder mini box.
• Sharing external USB storage media directly requires a compatible file format. Or additional utilities.
• The VM file formats should be portable between the Laptop and the Nettop and many other PCs. If you've got a MacOS machine for example: use the same virtualization software.
• Intel Atom CPUs used (N200 or N300 series) nowadays do not support memory remapping and 3,2 GB RAM at max. That's a rough limitation. Consider that you may still want 4 GB, because the onboard graphic uses the RAM, too. I chose some specially cooled RAM with low latency. Arguable of course.
• Furthermore you can activate these new "Memory Boost" BIOS options and do some overlocking if you are willing to take that risk and if your Nettop is actively cooled. Mine is ;).

Performance

Windows XP SP 3 and Ubuntu Xfce 4 running within VMware Workstation 7 64bit on an ASrock ION 330 with ~3,2 GB RAM with Debian Linux 64 bit. The Windows VM runs VisualStudio, Ubuntu has a bunch of Shells open.

Three interesting things:

1. it just shows 4 cores. - In fact that's only a DualCore with Hyper-Threading extension.
2. it runs two VMs and each VM got one "core". I could assign the other cores to do anything else. 2 for Windows 7 e. g.. That's an interesting flexibility.
3. VMWare offers a many features: native VNC (Remote GUI) for example. Furthermore there're some interesting tuning options for the hypervisor; like paravirtualisation. - Getting familiar with this stuff in general can be very helpful for work, because VMWare often gets deployed these days.

There're no real performance problems - as crazy as this might seem. However on Windows 7 x64 Professional (as Host-OS) VMWare Workstation 7 didn't scale and was awfully slow. On the other hand - on Windows - modern games work with an acceptable amount of frames.

Fair enough?

So to sum up: this Nettop thing is fun. While you cannot expect it to scale for a professional environment (I have other louder, more powerful computers ;)) it works surprisingly well with Debian. Very much depending on your needs it might be a good idea or a good excuse to get such a toy for Christmas. It's energy efficient, small and fits next to my external hard-drives, not being much larger.

Have fun,
wishi