File scanner web app (Part 2 of 5): Upload files

02 Sep 2013

Part 1, Part 2, Part 3, Part 4, Part 5

So far in this dev training, we stood up an Ubuntu VM and made a basic web service at /var/apps/scanner/webserver.py. Now we'll change it from a simple static web server to something that can upload files.

Upload files using DropzoneJS

We'll use Dropzone.js for uploading files. Get a copy by running:

wget -O /var/apps/scanner/static/dropzone.js \
https://raw.github.com/enyo/dropzone/master/downloads/dropzone.js

Now, let's edit our ugly index.htm to do something.

<html>
<head>
  <script src="/dropzone.js"></script>

  <style>
  .dropzone {
    border-style:dotted;
    border-width:2px;
    min-height: 100px;
    height:100px;
  }
  </style>
</head>

<body>
  <h1>File Scanner</h1>

  <form action="/file-upload"
        class="dropzone"
        id="mydropzone"></form>
</body>

<script>
Dropzone.options.mydropzone = {
  previewTemplate : '<div class="preview file-preview">\
  <div class="dz-details">\
    <b class="dz-filename"><span data-dz-name></span></b>\
    <b class="dz-size" data-dz-size></b>\
  </div>\
  <div class="dz-progress"><span class="dz-upload" data-dz-uploadprogress></span></div>\
  <div class="dz-error-message"><span data-dz-errormessage></span></div>',
  init: function() {
    this.on("complete", function(file) { console.log("Upload complete"); });
  }
};
</script>

Edit webserver.py to contain the following and restart the process.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/usr/bin/python

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render("static/index.htm")


class UploadHandler(tornado.web.RequestHandler):             # +
    def post(self):                                          # +
        file_contents = self.request.files['file'][0].body   # +
        with open("uploads/file", "wb") as f:                # +
            f.write(file_contents)                           # +
        self.finish()                                        # +


application = tornado.web.Application([
    (r"/file-upload", UploadHandler),                        # +
    (r"/", MainHandler),
    (r'/(.*)', tornado.web.StaticFileHandler, {'path': 'static'}),
])

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

Now you can either drag and drop files to your webpage (inside the dotted box) or click inside the dotted box and it will pop-up a file explorer view so you can select a file to upload. The file will be uploaded as /var/apps/scanner/uploads/file. So now we'll need to do something to give unique filenames.

Setting up MySQL

Install MySQL and Python's mysqldb library by running:

sudo apt-get install mysql-server python-mysqldb

For this guide, I'll use the password mypassword. From the command line run:

mysql -u root -p
Provide the password when prompted. Now we'll create a database and our initial table.
create database scanner;
use scanner;
create table files (
  id INT NOT NULL AUTO_INCREMENT,
  submission_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  filename VARCHAR(30) NOT NULL DEFAULT "",
  size INT NOT NULL DEFAULT 0,
  md5 CHAR(32) NOT NULL DEFAULT "",
  PRIMARY KEY(id)
);

Initially, all we'll do in our database is store the filename given to us, and we'll figure out the size and md5 for it. On disk, we'll save the files as uploads/1, uploads/2, etc. Doing this ensures we don't run into naming collisions.

Upload files with unique names

Now we need to modify our webserver.py to use our database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/usr/bin/python

import tornado.ioloop
import tornado.web

import MySQLdb

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render("static/index.htm")


class UploadHandler(tornado.web.RequestHandler):
    def post(self):
        file_name = self.request.files['file'][0].filename
        file_contents = self.request.files['file'][0].body
        file_size = len(file_contents)

        stmt = "INSERT INTO files (filename, size) VALUES (%(filename)s, %(filesize)s)"
        cur.execute(stmt, {'filename': file_name, 'filesize': file_size})
        file_id = cur.lastrowid
        db.commit()

        with open("uploads/%s" % file_id, "wb") as f:
            f.write(file_contents)
        self.finish()


db = MySQLdb.connect(host="localhost",
                     user="root",
                      passwd="mypassword",
                      db="scanner")
cur = db.cursor()

application = tornado.web.Application([
    (r"/file-upload", UploadHandler),
    (r"/", MainHandler),
    (r'/(.*)', tornado.web.StaticFileHandler, {'path': 'static'}),
])

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

We'll fill in the MD5 hash later. For now, try uploading some files, and ensure they should up with ID's for names in uploads, and you can check the mysql using select * from files.

Conclusion

You can now upload files to your webserver and have some info about them stored in the database. In order to have something interesting happen with the file, you'll need to complete the rest of the series.

Next Part 3