Things I've learned using skulpt for in-browser Python code

29 Jan 2013

This post summarizes some core concepts in getting skulpt (Python in the browser) to work.

My IceBuddha project uses skulpt for in-browser Python code. I wanted the ability to allow users to write their own parsing scripts that they could test immediately. The solution I initially chose was to have them write JavaScript with the help of some features of a domain specific language that was parsed with PEG.js. Few security professionals code in JavaScript regularly though, and they were my target audience so I wanted to use Python.

Options:

  1. Upload the Python code and binary to be parsed to my server, and have the Python code execute there with the binary as input and produce a result to send back to the user's browser.
    • Positives
      • Easiest solution to implement
    • Negatives
      • Security concerns of user provided data being executed on my server.
      • Potentially large amounts of data being uploaded to my server (the binary files especially could be large).
  2. Leave the binary in the user's browser, and upload the Python code to my server, compile it to JavaScript using pyjs (formerly called Pyjamas) or some other option, and send the JavaScript back to the user's browser.
    • Positives
      • Possibly a more featureful implementation of Python than what Skulpt supports
    • Negatives
      • I wanted as few dependencies on my server as possible, and for privacy concerns of users I didn't want the user to upload anything to my server.
  3. Leave the binary and the Python code on the users browser, and compile the Python code to JavaScript locally using Skulpt.
    • Positives
      • Completely in-browser solution.
    • Negatives
      • Using Skulpt requires the user to download between 242K-440K of JavaScript.
      • Imported modules have to be compiled to JavaScript in advance, or you could do a hack of concat'ing files together.
      • Some Python functionality is not supported.
      • Debugging Python code that has been compiled to JavaScript can be tricky, especially when the problem may simply be that the feature you want to use has not been added to Skulpt yet.

For my needs, I decided to go with Skulpt.

Hello World with Skulpt

There is a great example on using skulpt on the homepage of skulpt.org, but it does graphics and thus a little extra than is really needed, so I'm going to simplify this further.

  1. Download skulpt.js (242KB)
  2. Add the following to a new file called test.html

    <html> 
<head>
<script src="skulpt.js" type="text/javascript"></script> 
</head> 

<body> 
<textarea id="pythonCode">
for i in range(10):
    print i
</textarea><br /> 
<pre id="output"></pre> 

<script type="text/javascript"> 
function outf(text) { 
    var mypre = document.getElementById("output"); 
    mypre.innerHTML = mypre.innerHTML + text; 
} 

var code = document.getElementById("pythonCode").value; 
Sk.configure({output:outf}); 
eval(Sk.importMainWithBody("<stdin>",false,code)); 
</script> 

</body> 
</html>

  3. Open test.html in your browser, and it should display the Python code and then the numbers 0 through 9. Hurray!
The JavaScript at the bottom finds the code we want to run (in the textarea), tells skulpt to use outf as our stdout when Python calls print, and then runs the code.

Loading Python code from files on the server

For me, the first thing that bothered me when working with skulpt, is I didn't want to have my Python code in my html or JavaScript files. I wanted just Python in .py files. This way all my syntax high-lighting and PEP8 checking would work. My solution is in my JavaScript code, I use jquery to grab my file:

cacheBreaker = "?"+new Date().getTime();
$.get("./test.py"+cacheBreaker, function(response) {
    runSkulpt(response);
});

Passing data to the Python code

To pass in data, I make my Python code a class, and then do things slightly differently. Here is an example:

Javascript code

var module = Sk.importMainWithBody("<stdin>", false, code);
var obj = module.tp$getattr('MyClass');
var runMethod = obj.tp$getattr('run');

var arrayForSkulpt = new Array();
for (var i=0; i<5; i++) {
  arrayForSkulpt[i] = i;
}

// Run parse script
var ret = Sk.misceval.callsim(runMethod, Sk.builtin.list(arrayForSkulpt));

Python code

class MyClass:
    def run(self, data):
        for i in data:
            print i

Getting data back from the Python code

I had trouble returning anything other than arrays, strings, numbers, and arrays of arrays back from the Python. So in my Python code I just return some arrays of data from the run method, and then in my JavaScript code, I spent a lot of time with the JavaScript debugger to figure out how to get the data back I wanted. The trick is that the data you want will be in ret.v in that JavaScript code from the last example. So if you change the Python code to:

class MyClass:
    def run(self, data):
        sum = 0
        for i in data:
            sum += i
        return sum

Then in your JavaScript you'll have:

// Run parse script
var ret = Sk.misceval.callsim(runMethod, Sk.builtin.list(arrayForSkulpt));
var sum = ret.v;

Adding modules to skulpt

Depending on your needs, you may want to just concatenate files together and run that blob through Skulpt. If you want to do more legit modules, then you'll need to compile those in advance, and this will require you to use an additional file (which you will build) called builtin.js that will be about 200K and will need to be sent to users of your site.

  1. Download the Skulpt code: git clone https://github.com/bnmnetp/skulpt.git
  2. Copy your module directory to ./skulpt/src/lib/. For example: 
    cd skulpt
mkdir ./src/lib/MyModule
cat >./src/lib/MyModule/__init__.py <<EOL
class MessagePrinter:
    def printMessage(self):
        print "Hello World"
EOL
  3. Compile skulpt and copy the new files to your server: 
    ./m dist
cp dist/* yourWebServer/.
  4. At the top of your HTML file, you'll need to add a reference to the file builtin.js after skulpt.js, so it looks like: 
    <html> 
<head>
<script src="skulpt.js" type="text/javascript"></script> 
<script src="builtin.js" type="text/javascript"></script> 
</head>
  5. In your JavaScript code, you'll need to add in the function builtinRead and add it to your Sk.configure call: 
    function builtinRead(x) {
    if (Sk.builtinFiles === undefined || Sk.builtinFiles["files"][x] === undefined)
            throw "File not found: '" + x + "'";
    return Sk.builtinFiles["files"][x];
}

Sk.configure({output:outf, read:builtinRead});
  6. Change your Python to use the new module: 
    import MyModule

class MyClass:
    def run(self, data):
        mp = MyModule.MessagePrinter()
        mp.printMessage()