If you're like me, your daily in-take of infosec news high-lights new vulns, things broken, and maybe some reverse engineering of some malware, but my day-to-day life is as a software developer in the infosec world. I build software, and I'm going to show you how to quickly stand up something useful using many of the technologies I use in my day-to-day life.

In this series I'm going to show how to quickly stand-up a web app that you can drag and drop files to that will get scanned using some YARA signatures. The value of this series is not so much in the end product as it is in learning how all these tools fit together and how you can build things quickly, with some polish along the way so your projects aren't just hacks that fall apart on the first power failure or when someone else tries to use it. A lot of development today, especially for web-apps, is just gluing different components together: Half the battle is knowing what to use to accomplish your goals, and the other half is just applying the glue. This will teach some of both skills.

To give you a better idea of what you'll be making, here is a screenshot of the final product.

You can see the final code by looking here, but there is a lot of setup needed along the way, maybe I'll write an installer if this gets some attention.

People unfamiliar with software development think it's about knowing certain languages. That's one small part of the discipline. It's more about knowing certain library APIs, OS functionality, how to use tools, how to debug, and lots of other skills.

Skills you'll learn:

• Part 1: Stand-up and webserver
  • Set-up and interact with a Ubuntu VM on VirtualBox through more than the console display, using sshfs.
  • Python's Tornado library for creating a web server
• Part 2: Upload files
  • DropzoneJS for dragging and dropping files to a web app
  • MySQL for a back-end database
• Part 3: YARA signatures
  • Creating YARA signatures
  • Integrating YARA with Python
• Part 4: Scanning files from the web app
  • RabbitMQ for work queuing.
  • Returning data as JSON data from a web app so AJAX calls can be made on it easily.
• Part 5: Finish it
  • Twitter's Bootstrap library for making pretty web pages
  • JQuery Datatables for a pretty table display, with connections for JSON data
  • Using Supervisord for services monitoring and control.

Standing up the environment

I'll be using VirtualBox and a VM for Ubuntu Server 12.04 x64.

1. Install VirtualBox.
2. Create an Ubuntu VM. I downloaded the Ubuntu Server 12.04, 64-bit iso. Create the VM, with all defaults, and make it an SSH server.
3. Snapshot You should always have a clean Ubuntu install ready to play with, so take a snapshot.
4. Add a Host-Only network adapter VirtualBox can be a pain to connect to. The default network setting is NAT, which allows the guest to access the Internet, but you can't connect to the guest from the host. To connect to the guest from the host, you can do one of the following:
   • Set up port-forwarding: Painful because you have to remember the port mapping (8022 on your localhost is 22 on the guest, or something like that).
   • Change the network adapter to Host-only from NAT, so you no longer have access to the Internet from the guest, which means you can't download anything without first downloading it to your host then scp'ing or otherwise copying it over.
   • Add a Host-only network adapter. No negatives.
   Let's add the host-only network adapter. If the following instructions get confusing, look at Host-Only Networking With VirtualBox
   1. In VirtualBox's main menu, go to your preferences. Click Network, and add a Host-only network (click the plus sign and something like "vboxnet0" should appear if you didn't have it already). See this if you have problems.
   2. In your VM, go to it's settings and add a second adapter for Host-Only networking. You'll need to shutdown the VM to do this.
   3. Start the VM up, and if you run ifconfig you'll notice you don't have a second adapter. Run the following to turn it on:

      sudo ifconfig eth1 192.168.56.101 netmask 255.255.255.0 up
      

      That is a temporary fix that will not survive reboot. You can now ssh in though. To make this change permanent edit the file /etc/network/interfaces to add the following:

      # The host-only network interface
      auto eth1
      iface eth1 inet static
      address 192.168.56.101
      netmask 255.255.255.0
      network 192.168.56.0
      broadcast 192.168.56.255

   You can now ssh in. Do that, because working through VitualBox's console display is annoying. You can also edit your /etc/hosts file now if you want so you don't have to remember the IP address.
5. Update it. You're in infosec, so make sure your stuff is patched. Everything should be up-to-date if you just downloaded the latest, but run the following anyway:

   sudo apt-get update
   sudo apt-get dist-upgrade
   

SSHFS: Get yourself a way to edit files

As legit as a console and vim are, it's 2013, and using a GUI text editor like Sublime Text can make life nicer. That means being able to access files as if they were local. You could setup an SMB mount, but let's use sshfs. If you're on Linux, you can follow this, or on Windows try this, but I'm on Mac OSX, so I installed OSXFUSE and then SSHFS from here.
On the guest I ran:

mkdir /var/apps
chmod 777 /var/apps

mkdir /Volumes/apps
sshfs user@192.168.56.101:/ /Volumes/apps

sudo SetFile -a v /Volumes

Tornado: Make a web server

All that work and we still have nothing to show for it. Let's make a web server. We'll use python's Tornado library. On the guest run:

sudo apt-get install python-setuptools
sudo easy_install tornado

Create the directory /var/apps/scanner to work in. Create a file there named webserver.py with the following contents:

#!/usr/bin/python

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello world")

application = tornado.web.Application([
    (r"/", MainHandler),
])

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

Run it with:

python webserver.py

From your host, you should now be able to visit http://192.168.56.101:8000/. You'll see a simple "Hello world" displayed.

Tornado allows you to quickly create web services. You could do something like this with Apache and mod_python, or with Python's twisted library, or gevent. In software development there are often many options to choose from at each step. Tornado I've found is quick and easy, although with that you lose some things like the default logging that Apache has. Also, all python apps are single-threaded, so you have to be more mindful not to do any actions that would cause blocking, like slow database queries.

Reading our code from the bottom up, it is simply saying "Create a web server on port 8000. Anything that tries to get the page '/' should see whatever the MainHandler returns. The MainHandler simply writes 'Hello world'".

Turn it into a static file server

Right now, your webserver can't display any files. It can only respond with "Hello world" and only if the client tries to access "/". Otherwise it gives a 404. Let's turn our server into a static file server by killing the "python webserver.py" process, replacing the webserver.py file with the following and starting the process back up.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

#!/usr/bin/python

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render("static/index.htm") # + Render file


application = tornado.web.Application([
    (r"/", MainHandler),
    (r'/(.*)', tornado.web.StaticFileHandler, {'path': 'static'}), # + File handler
])

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

Now create a directory /var/apps/scanner/static and create a index.htm file there. For example:

echo "<html><body bgcolor=yellow>This is static" > /var/apps/scanner/static/index.htm

After starting the service back up, visit http://192.168.56.101:8000/ and you should see an ugly site with your text.

Tornado caches your files, so any time you edit that static index.htm file or any other file you plan on serving, you'll need to restart webserver.py.

Handlers

To give you a feel for how handlers in Tornado work, let's add two additional ones to our code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#!/usr/bin/python

import tornado.ioloop
import tornado.web
from datetime import datetime                      # +


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render("static/index.htm")

class TimeHandler(tornado.web.RequestHandler):     # +
    def get(self):                                 # +
        self.write("%s" % datetime.now())          # +

class AdditionHandler(tornado.web.RequestHandler): # +
    def get(self, num1, num2):                     # +
        self.write("%d" % (int(num1)+int(num2)))   # +


application = tornado.web.Application([
    (r"/time", TimeHandler),                       # +
    (r"/add/([0-9]+)\+([0-9]+)", AdditionHandler), # +
    (r"/", MainHandler),
    (r'/(.*)', tornado.web.StaticFileHandler, {'path': 'static'}),
])

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

The first new handler (TimeHandler) at http://192.168.56.101:8000/time" shows you how easy it is to get your own python code running whenever the client visits one of our registered handlers.

The other handler (AdditionHandler) at http://192.168.56.101:8000/add/3+15 so you can see how to include parameters in your URL's and how those are interpreted by Tornado. The handlers are registered using regex's, and the "variables" of those regex's get included as additional parameters to the get method.

Play around and make some different handlers!

Next up, Part 2