Notes on using pip and virtualenv with Django

Date: 2009-05-06 | Modified: 2009-09-27 | Tags: django, install_setup, python, ubuntu | 4 Comments

I have been using a symlinking method to install Python packages up to this point. To better handle dependencies and multiple versions I have wanted to switch over to pip and virtualenv. Pip is a better alternative to Easy Install and virtualenv is a tool to create isolated Python environments. I have wanted to use pip and virtualenv for a long time now. Finally, today, I took my first steps and created an environment with the Python packages required for this blog. My notes are below. (I am running Ubuntu Intrepid and Python 2.5.)

A lot of my notes on virtualenv are taken from Arthur Koziel's excellent tutorial. Also, below are some links to some essential information on pip, virtualenv, and Python packaging (mostly from the creator of pip and virtualenv, Ian Bicking):

Workingenv is dead, long live Virtualenv! by Ian Bicking (2007-10-10)
mailing list post explaining a number of package management technologies by Kevin Teague (2008-09-14)
pyinstall: A New Hope by Ian Bicking (2008-09-24)
pyinstall pybundles by Ian Bicking (2008-10-01)
pyinstall is dead, long live pip! by Ian Bicking (2008-10-28)
On packaging by James Bennett (2008-12-14)
A Few Corrections To "On Packaging" by Ian Bicking (2008-12-14)
Why I like pip by James Bennett (2008-12-15)
Using pip Requirements by Ian Bicking (2008-12-16)

Install Easy Install

If you don't already have Easy Install, it can be installed as follows:

sudo apt-get install python-setuptools python-dev build-essential

Install pip

sudo easy_install -U pip

Install virtualenv

sudo pip install -U virtualenv

Output:

Downloading/unpacking virtualenv
  Downloading virtualenv-1.3.3.tar.gz (1.0Mb): 1.0Mb downloaded
  Running setup.py egg_info for package virtualenv
Installing collected packages: virtualenv
  Running setup.py install for virtualenv
    Installing virtualenv script to /usr/bin
Successfully installed virtualenv

Create virtual environment

mkdir -p /srv/python-environments
cd /srv/python-environments
virtualenv --no-site-packages saltycrane

Output:

New python executable in saltycrane/bin/python
Installing setuptools............done.

Clear the PYTHONPATH variable

I don't know if this is necessary, but I had a problem with the akismet module when this was set. Maybe I did something wrong, but when I cleared PYTHONPATH, it worked.

export PYTHONPATH=

Install a package (Yolk) to the new virtual environment

Yolk is a tool that can list Python packages.

cd /srv/python-environments
pip install -E saltycrane/ yolk

Output:

Downloading/unpacking yolk
  Downloading yolk-0.4.1.tar.gz (80Kb): 80Kb downloaded
  Running setup.py egg_info for package yolk
Requirement already satisfied: setuptools in /usr/lib/python2.5/site-packages (from yolk)
Installing collected packages: setuptools, yolk
  Running setup.py install for yolk
    Installing yolk script to /srv/python-environments/saltycrane/bin
Successfully installed yolk

Activate a virtual environment and use it

Activate the saltycrane virtual environment

cd /srv/python-environments
source saltycrane/bin/activate

Run yolk to list your installed packages:

yolk -l

Python          - 2.5.2        - active development (/usr/lib/python2.5/lib-dynload)
setuptools      - 0.6c9        - active 
wsgiref         - 0.1.2        - active development (/usr/lib/python2.5)
yolk            - 0.4.1        - active

Deactivate environment
```
deactivate
```
Try running yolk now:
```
yolk
```
```
-bash: yolk: command not found
```

Install Django with pip and virtualenv

cd ~/lib/python-environments
pip install -E saltycrane/ Django

Downloading/unpacking Django
  Downloading Django-1.0.2-final.tar.gz (4.6Mb): 4.6Mb downloaded
  Running setup.py egg_info for package Django
Installing collected packages: Django
  Running setup.py install for Django
    changing mode of build/scripts-2.5/django-admin.py from 644 to 755
    changing mode of /srv/python-environments/saltycrane/bin/django-admin.py to 755
Successfully installed Django

Create a requirements file from existing package versions using pip freeze

cd /srv/python-environments
pip freeze -E saltycrane/ requirements.txt

This creates /srv/python-environments/requirements.txt:

Django==1.0.2-final
wsgiref==0.1.2
yolk==0.4.1

Install packages based on a requirements file

Here's my requirements file for this blog, /srv/SaltyCrane/pip-requirements.txt:

Django==1.0.2-final
Markdown==2.0
BeautifulSoup==3.0.7a
Pygments==1.0

-e svn+http://django-tagging.googlecode.com/svn/trunk/@154#egg=django-tagging
-e hg+http://bitbucket.org/ubernostrum/django-contact-form/#egg=django-contact-form
-e hg+http://bitbucket.org/jezdez/akismet/#egg=akismet

Install:

cd /srv/python-environments
pip install -E saltycrane/ -r /srv/SaltyCrane/pip-requirements.txt

Output:

Downloading/unpacking Markdown==2.0 (from -r /srv/SaltyCrane/pip-requirements.txt (line 3))
  Could not fetch URL http://pypi.python.org/simple/Markdown/2.0: HTTP Error 404: Not Found
  Will skip URL http://pypi.python.org/simple/Markdown/2.0 when looking for download links for Markdown==2.0 (from -r /srv/SaltyCrane/pip-requirements.txt (line 3))
  Downloading Markdown-2.0.tar.gz (71Kb): 71Kb downloaded
  Running setup.py egg_info for package Markdown
Checking out django-tagging from svn+http://django-tagging.googlecode.com/svn/trunk/@154#egg=django-tagging checkout from svn+http://django-tagging.googlecode.com/svn/trunk/@154#egg=django-tagging (from -r /srv/SaltyCrane/pip-requirements.txt (line 7))
  Checking out http://django-tagging.googlecode.com/svn/trunk/ (to revision 154) to ./saltycrane/src/django-tagging
  Running setup.py egg_info for package django-tagging
Downloading/unpacking BeautifulSoup==3.0.7a (from -r /srv/SaltyCrane/pip-requirements.txt (line 4))
  Could not fetch URL http://pypi.python.org/simple/BeautifulSoup/3.0.7a: HTTP Error 404: Not Found
  Will skip URL http://pypi.python.org/simple/BeautifulSoup/3.0.7a when looking for download links for BeautifulSoup==3.0.7a (from -r /srv/SaltyCrane/pip-requirements.txt (line 4))
  Downloading BeautifulSoup-3.0.7a.tar.gz
  Running setup.py egg_info for package BeautifulSoup
Checking out django-contact-form from hg+http://bitbucket.org/ubernostrum/django-contact-form/#egg=django-contact-form checkout from hg+http://bitbucket.org/ubernostrum/django-contact-form/#egg=django-contact-form (from -r /srv/SaltyCrane/pip-requirements.txt (line 8))
  Cloning hg http://bitbucket.org/ubernostrum/django-contact-form/ to ./saltycrane/src/django-contact-form
  Running setup.py egg_info for package django-contact-form
Requirement already satisfied: Django==1.0.2-final in ./saltycrane/lib/python2.5/site-packages (from -r /srv/SaltyCrane/pip-requirements.txt (line 2))
Downloading/unpacking Pygments==1.0 (from -r /srv/SaltyCrane/pip-requirements.txt (line 5))
  Could not fetch URL http://pypi.python.org/simple/Pygments/1.0: HTTP Error 404: Not Found
  Will skip URL http://pypi.python.org/simple/Pygments/1.0 when looking for download links for Pygments==1.0 (from -r /srv/SaltyCrane/pip-requirements.txt (line 5))
  Downloading Pygments-1.0.tar.gz (930Kb): 930Kb downloaded
  Running setup.py egg_info for package Pygments
Checking out akismet from hg+http://bitbucket.org/jezdez/akismet/#egg=akismet checkout from hg+http://bitbucket.org/jezdez/akismet/#egg=akismet (from -r /srv/SaltyCrane/pip-requirements.txt (line 9))
  Cloning hg http://bitbucket.org/jezdez/akismet/ to ./saltycrane/src/akismet
  Running setup.py egg_info for package akismet
Installing collected packages: akismet, BeautifulSoup, Django, django-contact-form, django-tagging, Markdown, Pygments
  Running setup.py install for Markdown
    changing mode of build/scripts-2.5/markdown.py from 644 to 755
    changing mode of /srv/python-environments/saltycrane/bin/markdown.py to 755
  Running setup.py develop for django-tagging
    Creating /srv/python-environments/saltycrane/lib/python2.5/site-packages/tagging.egg-link (link to .)
    Adding tagging 0.3-pre to easy-install.pth file
    
    Installed /srv/python-environments/saltycrane/src/django-tagging
  Running setup.py install for BeautifulSoup
  Running setup.py develop for django-contact-form
    Creating /srv/python-environments/saltycrane/lib/python2.5/site-packages/django-contact-form.egg-link (link to .)
    Adding django-contact-form 0.3 to easy-install.pth file
    
    Installed /srv/python-environments/saltycrane/src/django-contact-form
  Running setup.py install for Pygments
    Installing pygmentize script to /srv/python-environments/saltycrane/bin
  Running setup.py develop for akismet
    Creating /srv/python-environments/saltycrane/lib/python2.5/site-packages/akismet.egg-link (link to .)
    Adding akismet 0.1.5 to easy-install.pth file
    
    Installed /srv/python-environments/saltycrane/src/akismet
Successfully installed

Use virtualenv with django and mod_python

I got the following from the Django, mod_python and virtualenv article at http://mydjangoblog.com/.

Create a file /srv/SaltyCrane/myvirtualdjango.py:

activate_this = "/srv/python-environments/saltycrane/bin/activate_this.py"
execfile(activate_this, dict(__file__=activate_this))

from django.core.handlers.modpython import handler

Edit your httpd.conf

    <Location "/">
        SetHandler python-program
        PythonHandler myvirtualdjango
        SetEnv DJANGO_SETTINGS_MODULE iwiwdsmi.settings
        PythonPath "['/srv/SaltyCrane',] + sys.path"
        PythonDebug Off
    </Location>

Use virtualenv with django and mod_wsgi

Added 2009-09-27: To use the the packages in my virtualenv, I used site.addsitedir at the top of my .wsgi application file. You may also want to set the WSGIPythonHome variable in your httpd.conf file (outside of any VirtualHost sections). For detailed information on using mod_wsgi with virtualenv, see the Virtual Environments section of the modwsgi project documentation.

/srv/SaltyCrane/saltycrane.wsgi:

import os
import sys
import site

site.addsitedir('/srv/python-environments/saltycrane/lib/python2.5/site-packages')

os.environ['DJANGO_SETTINGS_MODULE'] = 'iwiwdsmi.settings'

sys.path.append('/srv/SaltyCrane')

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

httpd.conf:

    WSGIScriptAlias / /srv/SaltyCrane/saltycrane.wsgi

4 Comments

#1 Alexander Artemenko commented on 2009-05-06:

By the way, if you'll need something more complex, than just installing applications in the virtual environment, I suggest to look at zc.buildout.

I'm already moving in that direction, I use buildout to deploy my django sites.

#2 Eliot commented on 2009-05-07:

Hi Alexander, thanks for the tip. I had noticed Jacob Kaplan-Moss' post about zc.buildout, but didn't read it carefully. I think I had already made up my mind to use pip+virtualenv. I will keep my eye on zc.buildout.

#3 Skylar commented on 2009-12-22:

Nice article, I love virtualenv. I'm looking for a way to install Django==dev from a requirements file but so far no luck. I guess I could point the requirements file at the github mirror or directly at svn trunk. I thought there was an easy way to denote this ... </ramble>

#4 Tim commented on 2009-12-26:

Skylar:

-e svn+http://code.djangoproject.com/svn/django/trunk#egg=Django

That what you're after?


Name:	Required
Email:	Required, but not displayed
Website:	Optional
Comment:	Format using Markdown. (No HTML.) Code blocks: prefix each line by at least 4 spaces or 1 tab (and a blank line before and after) Code span: surround with backticks Blockquotes: prefix lines to be quoted with > Links: <URL> Links w/ description: [description](URL)

SaltyCrane Blog