Django on DreamHost VPS

I recently successfully migrated my first Django app from DreamHost shared hosting to a DreamHost VPS (virtual private server). I hope this info will be useful to others. The following two blog posts were absolutely key to me getting this working, especially the first one. Thanks Preston and Graham!

• Setting up Pinax on DreamHost Private Server
• An Improved WSGI Script for use with Django

Disable Passenger if Migrating from Shared Hosting

If you set up your shared site for Django recently, you likely enabled Phusion Passenger to serve your Django projects. While it works great on shared hosting, you’re better off using the more conventional mod_wsgi approach on a PS. If you don’t disable it, it will interfere with mod_wsgi AND use up a lot of the memory you are paying for on your PS. In the graph below, memory use had been steady at about 250 MB. After disabling Passenger, memory use leveled off at around 50 MB. This chart is from the DreamHost control panel.

Disable DreamHost Management of Web Server

To enable mod_wsgi and make a few directories accessible by Apache, you will need to modify httpd.conf. The DreamHost control panel does a great job of managing httpd.conf for you. But, unless you want to be constantly merging in your changes, you need to manage httpd.conf yourself.

Under Private Servers, select Configure Server. Then in the Web Server Configuration section, uncheck DreamHost Managed and click the Save psNNNNN settings… button.

To edit the config and restart Apache, you need to setup a user with sudo privileges. I created a special user account solely for this purpose. After creating the new user account through the control panel, give that user sudo privileges via the Manage Admin Servers entry in the Private Servers section of the DreamHost control panel.

Important Directories

In addition to knowing about /usr/local/dh/apache2, there are a couple of other important directories, some of which need to be added. Replace YOU with your username. This directory layout isn’t required, but you’ll have to adjust the rest of the instructions if you save things elsewhere.

• /home/YOU/src – Convenient place for downloading libraries for building and installing
• /home/YOU/wsgi-scripts – Will contain a wsgi-script for each Django project
• /home/YOU/projects – Will contain all Django projects

Since Apache will be accessing these directories as a different user, make sure the directories are accessible (i.e., chmod 755). I got bit by this as part of my shared hosting migration. On shared hosting, scripts get run under your user, so my project didn’t need to be group or other executable.

Inside of a Django project, I use the following layout. In this example, I’m serving up static files with the same web server. In the instructions below, I’ll be setting up an alias to the media directory from Apache.

__init.py__
manage.py
settings.py
templates
  |---- app1
     |---- base.html
     |---- index.html
     |---- ...
  |---- app2
     ...
media
  |---- css
  |---- js
  ...
...

To keep people from accessing directory listings of your media directory, you can add a .htaccess file to that directory with the following contents.

Options -Indexes

Setup Python

I ran into issues with Python 2.6 (No module named _md5), so I recommend sticking with Python 2.5. In case the default Python version on your server is 2.4, make 2.5 the default:

$ sudo rm /usr/bin/python
$ sudo ln -s /usr/bin/python2.5 /usr/bin/python

Create a directory in which to download and install libraries.

$ mkdir ~/src
$ cd ~/src

Download and install setuptools for Python 2.5.

wget http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c11-py2.5.egg#md5=64c94f3bf7a72a13ec83e0b24f2749b2
sudo sh setuptools-0.6c11-py2.5.egg

Upgrade Django, if desired.

$ sudo easy_install pip
$ pip install -U django

Download and build MySQLdb library. I prefer oursql, but it is easier to get started with MySQLdb.

$ tar xzf MySQL-python-1.2.3c1.tar.gz
$ cd MySQL-python-1.2.3c1
$ sudo python setup.py install

Preston’s instructions also cover the Python Imaging Library and virtualenv. I’ve skipped them here to keep the instructions shorter. You can easily install them later.

Before preceding with mod_wsgi, make sure your Django app runs on the Django dev server. It’s a lot easier to sort out issues with settings.py at this stage.

Create your Django project with django-admin.py or copy an existing project into the projects directory mentioned above.
In settings.py, add a variable that determines the location of your project directory. You’ll also need to add import os at the top of the file.

PROJECT_ROOT = os.path.realpath(os.path.dirname(__file__))

And then set up your media and templates:

MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'media')
TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, 'templates'),)

In the instructions below, I set up aliases for Apache to the project media and admin media directoris, so I set both media URLs to /media/.

MEDIA_URL = '/media/'
ADMIN_MEDIA_PREFIX = '/media/'

Running with the Devserver

From your project directory, create the database if your app needs one (this assumes you’ve already set up a MySQL database through the control panel or manually).

$ chmod u+x manage.py
$ ./manage.py syncdb

Start the dev server and make sure your app loads.

./manage.py runserver 0.0.0.0:8000

If not, check the output that is being logged to the console. You may also want to temporarily change settings.py so that DEBUG = True. Kill the dev server with ctrl-c once you’ve got things sorted out.

Install mod_wsgi

Download, build and install mod_wsgi.

$ cd ~/src
$ wget http://modwsgi.googlecode.com/files/mod_wsgi-3.0c5.tar.gz
$ tar xzf mod_wsgi-3.0c5.tar.gz
$ cd mod_wsgi-3.0c5
$ ./configure --with-apxs=/usr/local/dh/apache2/template/sbin/apxs --with-python=/usr/bin/python
$ make
$ sudo make install

Create a WSGI Script

In the /home/YOU/wsgi-scripts directory create a PROJECT.wsgi file, where you replace PROJECT with your project’s name. Below are the contents of mine. Replace YOU and PROJECT with appropriate values. See the comments for this post for an explanation of why you should use /var/tmp instead of /tmp for PYTHON_EGG_CACHE.

import os
import sys

# setup_tools will try to cache egg files in surprising locations where the
# apache user likely doesn't have write access unless you do this
os.environ['PYTHON_EGG_CACHE'] = '/var/tmp'

# Ensure that parent directory of project is on PYTHONPATH
sys.path.insert(0,'/home/YOU/projects')

os.environ['DJANGO_SETTINGS_MODULE'] = 'PROJECT.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

def test_wsgi(environ, start_response):
    status = '200 OK'
    output = 'Hello World! wsgi py \n' + sys.version + '\n' + '\n'.join(sys.path)
    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)
    return [output]

# Uncomment next line to test that basic mod_wsgi setup works
# and to see the contents of PYTHONPATH
application = test_wsgi

Note, that the last line is uncommented. That will allow us to first make sure that mod_wsgi is working, in general. Later, we’ll comment it out.

Test mod_wsgi

Next you need to edit the Apache config file. Change psNNNNN to your actual private server ID in each of the following steps.

To keep some of the following commands short, cd into the apache2 directory:

$ cd /usr/local/dh/apache2

To edit the Apache config (of course, you can use another editor besides vi):

$ sudo vi apache2-psNNNNN/etc/httpd.conf

To restart Apache:

$ sudo /etc/init.d/httpd2 restart apache2-psNNNNN

To view the Apache error log:

$ sudo less apache2-psNNNNN/logs/error.log

Open httpd.conf in an editor, search for the modules section and add the following line:

LoadModule wsgi_module /dh/apache2/template/lib/modules/mod_wsgi.so

Then, search for the VirtualHost section for your domain. At the end before , add the following while replacing YOU and PROJECT as appropriate:

# wsgi
<Directory /home/YOU/wsgi-scripts>
Order allow,deny
Allow from all
</Directory>

# absolute path
WSGIScriptAlias / /home/YOU/wsgi-scripts/PROJECT.wsgi

<Directory "/home/YOU/projects/PROJECT/media">
Allow from all
Order allow,deny
</Directory>

<Directory "/usr/lib/python2.5/site-packages/django/contrib/admin/media">
Allow from all
Order allow,deny
</Directory>

Alias "/media/" "/home/YOU/projects/PROJECT/media/"

# note some use hyphen - others _
Alias "/media/" "/usr/lib/python2.5/site-packages/django/contrib/admin/media/"

Only the first two sections are needed for a basic mod_wsgi test. If you run into any problems, try it with just the part up to and including the WSGIScriptAlias.

Restart Apache and access what would be the normal URL for your Django app. You should see something like:

Hello World! wsgi py
2.5 (release25-maint, Jan 24 2010, 14:37:54)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)]
/home/YOU/projects
/usr/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg
/usr/lib/python2.5/site-packages/MySQL_python-1.2.3c1-py2.5-linux-x86_64.egg
/usr/lib/python2.5/site-packages/pip-0.7.1-py2.5.egg
/usr/lib/python25.zip
/usr/lib/python2.5
/usr/lib/python2.5/plat-linux2
/usr/lib/python2.5/lib-tk
/usr/lib/python2.5/lib-dynload
/usr/local/lib/python2.5/site-packages
/usr/lib/python2.5/site-packages
/usr/lib/python2.5/site-packages/PIL
/usr/lib/site-python

Test Django with mod_wsgi

Once that’s working, comment out the last line in your WSGI script and hopefully your app will load.

Troubleshooting

I found it helpful to open a couple of console windows where I could:

• Log in as admin user and tail the Apache error log (sudo tail -f apache2-psNNNNN/logs/error.log)
• Log in as admin user and restart Apache as necessary
• Log in as regular user and tail the HTTP error log (tail -f /home/YOU/DOMAIN/http/error.log)
• Log in as regular user and edit my wsgi script, settings.py, etc.

The output in the error logs was key to solving most of the problems I ran into. The two web pages I mentioned at the beginning of this post were also crucial. Graham’s post includes a lot of useful detail on WSGI. Reading it enabled me to better understand what the error messages I was seeing really meant.