How To Run Sandboxed Django Sites in Production

Recently, I moved all my sites from a dedicated server to a great VPS over at SliceHost. I took the opportunity of “the big move” to fix an ongoing problem I’d had, which was that all four of my Django sites were using the same libraries.

Ouch! That meant that I couldn’t update any of my sites without updating all of them, or at least retesting all of them.

This article will explain how I sandboxed my Django sites, so that all of them have their own versions of the appropriate library, and how I have them all running on the new production server.

The Tools

• Lighttpd – I am convinced that Lighty is the easiest, fastest (or close enough not to matter), most reliable way to serve Django. Whenever possible, I use it for my own sites and for clients. I am not an Apache fan, I find it bloated, opaque, hard to debug and hard to configure.
• Supervisor – A fantastic, easily configured Daemon starter and monitor for Linux. It doesn’t replace the normal /etc/init.d/ system on Linux, instead it supplements it with the ability to run certain things – in this case my Django servers – very easily. It automatically restarts failed processes, monitors uptime, and is generally my favorite new find in sysadmin world.
• Superlance – A plugin for Supervisor that does HTTP web “pings”, emails on failure and restarts the failed process.
• Virtualenv – The tool that does the actual creation of the sandboxes. Absolutely invaluable.

Setting up your VirtualEnvironments

There are plenty of good tutorials about using virtualenv. SaltyCrane has a particularly good one with links to his sources.

I set it up pretty much as he describes, with the exception that any packages that I’m going to use from source (including the site itself) get put in a “src” directory at the root of the virtual environment.

I then symlink src/site-i-am-working-on to the root as well. Makes it easier to get to work on the site, but keeps everything in the same logical places.

Lastly, I create a file called “production”, in the bin directory. It is just a shell script that gives details about how to run this particular environment in production. Here’s an example:

SLUG=invisible
PID_FILE=/var/venv/$SLUG/$SLUG.pid
HOST=127.0.0.1
PORT=16667
MANAGE_COMMAND="runfcgi host=$HOST port=$PORT pidfile=$PID_FILE daemonize=false maxchildren=10  maxspare=5 minspare=2 method=threaded outlog=/var/venv/$SLUG/logs/${SLUG}.stdout errlog=/var/venv/$SLUG/logs/${SLUG}.stderr"

This shell script is going to be used by a convenient little script named “start_venv.sh” to make the fastcgi process start inside its virtual environment. That file is included in my example supervisor.tgz archive.

#! /bin/sh
set -e

if [ -z $1 ]
then
  echo "USAGE: start_venv.sh venv [workdir]"
  exit 2
fi

VENV=/var/venv/$1

if [ -z $2 ]
then
  STARTDIR=$VENV/$1
else
  STARTDIR=$VENV/$2
fi

if [ ! -f $VENV/bin/production ]
then
  echo "Could not find $VENV/bin/production"
  exit 2
fi

source $VENV/bin/activate
cd $STARTDIR
source $VENV/bin/production
python manage.py $MANAGE_COMMAND

You’ll have to adjust a path or two to your preference. I put everything in /var/venv/[project], and I almost always arrange things so that the project itself is named the same, at the root of the virtual environment. For example, “invisible castle” is at /var/venv/invisible/invisible. That way, I can start the app with a simple “start_venv.sh invisible”

Setting up Supervisor

A simple “pip install superlance” should do the trick. It will install both Supervisor and the Superlance plugin.

I’m including my init script for supervisor in my example supervisor.tgz archive. It is Debian specific, modify to your needs on a RedHat system.

Configuring Supervisor to start and monitor your sites is remarkably simple and short. Here are the key lines from my supervisord.conf file:

[program:invisible]
command=/usr/bin/pidproxy /var/venv/invisible/invisible.pid /var/venv/start_venv.sh invisible
autostart=true
autorestart=true
startsecs=10

[eventlistener:invisible_httpok]
command=python -u /usr/bin/httpok -p invisible -m myaddress@example.com http://invisiblecastle.com/statusping/
events=TICK_60

The first starts invisible castle, the second runs an HTTP check every 60 seconds, emailing me and restarting if it didn’t work.

For the “statusping”, I just use the simplest thing possible, a “direct to template” entry in my urls.

urlpatterns += patterns('django.views.generic.simple',
 (r'^statusping/$', 'direct_to_template', {'template': 'statusping.html'})

My “statusping.html” file simply contains the word “OK”, it doesn’t extend any base templates, it just says “OK” when hit. Easy, low resource use.

After supervisor is set up, you can control it by simply typing “supervisorctl”, which will show you a list of all monitored processes and their uptime. Start, stop or restart by typing for example “restart invisible”, you get the idea. It could hardly be easier, and you can make supervisorctl available to non-root users, which is very nice.

Setting up Lighttpd

Here’s an example section from my lighttpd.conf file:

fastcgi.server = (
  "/invisible.fcgi" =>
  ((
      "check-local" => "disable",
      "host" => "127.0.0.1",
      "port" => 16667,
      "min-proces" => 4,
      "max-load-per-proc" => 3,
  )),
  # more .fcgi definitions here
)

# redirect www hits
$HTTP["host"] =~ "www\.invisiblecastle\.com(.*)" {
	url.redirect = ( "^/(.*)" => "http://invisiblecastle.com/$1" )
}

$HTTP["host"] == "invisiblecastle.com" {
    alias.url = (
       "/media/" => "/var/venv/invisible/lib/src/django/contrib/admin/media/",
       "/static/" => "/var/venv/invisible/lib/src/invisible/static/",
    )

    url.rewrite-once = (
        "^(/media.*)$" => "$1",
        "^(/static.*)$" => "$1",
        "^/favicon\.ico$" => "/static/favicon.ico",
        "^(/.*)$" => "/invisible.fcgi$1",
    )

    server.errorlog = "/var/venv/invisible/logs/invisible.error.log"
    accesslog.filename = "/var/venv/invisible/logs/invisible.access.log"
}

Technorati Tags: Django, Lighttpd, supervisord, virtualenv

Related posts:

1. Django and Lighttpd configuration for smooth SSL I use and prefer Lighttpd for serving my Django applications....
2. Django and Lighttpd init script and config for SSL. I've gotten a lot of interest in my posting about...
3. Lighttpd 1.5 prerelease doesn't like Django Whew. I spent quite a bit more time debugging this...
4. Zyons Forum for Django This summer, I've been converting my biggest site, Invisible Castle,...
5. Howto Reset The Admin Password in Django I keep needing to do this, darn it! I leave...