SaltyCrane Blog — Notes on JavaScript and web development

Django Blog project #3: Using CSS and Template Inheritance

Version 0.0.1 of my new blog had no style. For 0.0.2, I added a little CSS to my Django template to make it look a little better.

Create new Django templates

I created two new Django templates, base.html and frontpage.html. base.html contains all the HTML and CSS that is common to all the web pages that will be on this site. frontpage.html is the template for the front page of my new blog. It uses the object-oriented technique of extending my base.html template. It uses the boilerplate code in base.html and then fills in sections with content specific to the front page. It is similar to server side includes, but more powerful because specific content can be inserted anywhere instead of just at the beginning or end. Django does have an include mechanism which acts like server side includes, but the extension method (also called Template Inheritance) is the preferred way of doing things.


Here is the code for frontpage.html:
{% extends "base.html" %}

{% block main %}
  {% for post in post_list %}
    {{ post }}
    <hr />
  {% endfor %}

{% endblock %}
Here is the code for base.html:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
body {
 margin: 0;
 padding: 0;
 font-family: Georgia, Times, "Times New Roman", serif;
 color: black;
 width: 600px;
 border-right: 1px solid black;
}
#header {
 background-color: #666;
 border-bottom: 1px solid #333;
}
#sidebarLeft {
 float: left;
 width: 160px;
 margin-left: 10px;
 padding-top: 1em;
}
#mainRight {
 padding-top: 1em;
 margin: 0 2em 0 200px;
}
#footer {
 clear: both;
 background-color: #ccc;
 padding-bottom: 1em;
 border-top: 1px solid #333;
 padding-left: 200px;
}
h2.sidebar {
  font-size: 120%;
}
</style>
<title>{% block title %}Sofeng's Blog Version 0.0.2{% endblock %}</title>
</head>
<body>

<div id="header">
  <h1>Sofeng's Blog Version 0.0.2</h1>
</div>
<div id="sidebarLeft">
  {% block sidebar %}
  <h2 class="sidebar">ABOUT</h2>
  <p>This is my new blog created using <a href="http://www.djangoproject.com">Django</a>,
    a <a href="http://www.python.org">Python</a> web framework. This site is under 
    construction. My current blog is located at: 
    <a href="http://iwiwdsmi.blogspot.com">http://iwiwdsmi.blogspot.com</a>.
  </p>
  {% endblock %}
</div>
<div id="mainRight">
  {% block main %}{% endblock %}
</div>
<div id="footer">
  A <a href="http://www.djangoproject.com/">Django</a> site.
</div> 

</body>
</html>

Modify view to use new template

To use the new template, I created a new view called frontpage in ~/src/django/myblogsite/myblogapp/views.py:

from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post

def frontpage(request):
    posts = Post.objects.all()
    post_body_list = [post.body for post in posts]
    return render_to_response('frontpage.html', 
                              {'post_list': post_body_list})
Correction 7/6/2008: I previously had from myblogapp.models import Post on the second line. This works, but is inconsistent with my urls.py below and can (and did for me) cause subtle errors in the future. I corrected the line to read: from myblogsite.myblogapp.models import Post.

Map an URL to the new view

To complete the change, I mapped the url /blog to the new frontpage view in ~/src/django/myblogsite/urls.py:

from django.conf.urls.defaults import *
from myblogsite.myblogapp.views import *

urlpatterns = patterns(
    '',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^myview1/$', myview1),
    (r'^blog/$', frontpage),
)

Upload project to Webfaction server

I updated my previous post to include a section on installing Mercurial. Here are my steps for pushing my changes to the server. It's a little more complicated than it should be because I was mucking with the repository, but future pushes should be much easier.

$ hg pull -u --remotecmd /home/sofeng/bin/hg ssh://[email protected]/webapps/django/myblogsite
[email protected]'s password:
pulling from ssh://[email protected]/webapps/django/myblogsite
searching for changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 4 changes to 5 files (+1 heads)
not updating, since new heads added
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg merge
 local changed templates/base.html which remote deleted
use (c)hanged version or (d)elete? c
3 files updated, 0 files merged, 3 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg commit -m 'commit after merge'
$ hg push --remotecmd /home/sofeng/bin/hg ssh://[email protected]/webapps/django/myblogsite
[email protected]'s password:
pushing to ssh://[email protected]/webapps/django/myblogsite
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 3 changesets with 5 changes to 6 files

Deploy

With my cleanup of the mercurial repository, I didn't have to make any file changes. I just updated the working directory of my project using hg update -C and restarted the Apache web server.

$ cd ~/webapps/django/myblogsite
$ hg update -C
5 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ ~/webapps/django/apache2/bin/restart

Now, pointing my browser at http://saltycrane.com/blog/, shows new version 0.0.2 of my blog!

Here is a snapshot screenshot of my blog version 0.0.2:


Related posts:
  Django Blog Project #1: Creating a basic blog
  Django Blog Project #2: Deploying at Webfaction
  Django Blog Project #4: Adding post metadata
  Django Blog Project #5: YUI CSS and serving static media

Mount drives manually on Ubuntu Linux

Since I'm running the wmii window manager instead of Gnome or KDE, devices such as USB drives, external hard drives, and SD card readers don't automatically mount. This can be good or bad depending on your preference. Here are my notes on mounting devices manually. I am no expert at this stuff, so use this at your own risk. There is definitely the possibility of losing data when dealing with this stuff. Read the man pages and use this just as a refresher. As an example, I will mount a SD card.


To see what devices are available:
$ sudo fdisk -l
This gives me the following output:
Disk /dev/sda: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x38000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1           7       56196   de  Dell Utility
/dev/sda2               8         660     5245222+   b  W95 FAT32
/dev/sda3   *         661       30017   235810102+  83  Linux
/dev/sda4           30018       30394     3028252+   5  Extended
/dev/sda5           30018       30394     3028221   82  Linux swap / Solaris

Disk /dev/sdf: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0008bc3c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1       38913   312568641   83  Linux

Disk /dev/sdd: 512 MB, 512229376 bytes
9 heads, 8 sectors/track, 13895 cylinders
Units = cylinders of 72 * 512 = 36864 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               4       13896      500107+   6  FAT16
/dev/sda3 is my main hard disk partition and /dev/sdf1 is an external hard drive formatted for Linux. The last device, /dev/sdd1 is my SD card.

Create a mount point
$ sudo mkdir /media/sdcard

Mount the device
$ sudo mount /dev/sdd1 /media/sdcard
Now you can access the files at /media/sdcard.

List mounted devices
$ mount -l
You should see things such as:
/dev/sda3 on / type ext3 (rw,errors=remount-ro) []
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
/dev/sdf1 on /media/disk type ext3 (rw,nosuid,nodev,uhelper=hal) []
tmpfs on /lib/modules/2.6.24-18-generic/volatile type tmpfs (rw,mode=0755)
/dev/sdd1 on /media/sdcard type vfat (rw) []
The last line shows device /dev/sdd1 mounted at /media/sdcard.

Unmount the device
$ sudo umount /media/sdcard
or
$ sudo umount /dev/sdd1

Django Blog Project #2: Django Webfaction deployment

I mentioned in my previous Django post that I'd write about hosting my new Django blog site on Webfaction. As promised, here it is.

I had some problems deploying my first site at Webfaction. First I got an error about pysqlite. I tried using Postgresql, and then I got an error about something else when trying to do a python2.5 manage.py syncdb. I think that they didn't set me up with a current Django installation. Luckily the support was prompt and helpful. I ended up reinstalling the Django application from the Webfaction Control Panel. Here are the steps I took to deploy my less than fledgling site.


Sign up for a shared hosting plan at Webfaction

I signed up for Webfaction's lowest plan at $9.50/month with no commitment. Here is their pricing plans.


Play with new account

I got a welcome email within 24 hours with the details about my account. sofeng.webfactional.com was a domain name which they provided. Visiting this address in my browser showed my the It worked! Django new project page. I also ssh'd into my shell account by using ssh sofeng.webfactional.com. The OS is CentOS5 which, I think, is based on Red Hat. It was a little different than the Ubuntu that I'm used to. I copied some of my configuration files over to make me feel more at home. On my home machine:

$ scp .bashrc [email protected]:
$ scp .screenrc [email protected]:
$ scp .emacs [email protected]:


Reinstall Django

Note, if your Webfaction account is set up properly with Django, you shouldn't need to take these steps. Like I mentioned earlier, I had some problems with my Django installation. So I ended up reinstalling Django. It's not as hard as it sounds-- just a few clicks in the Control Panel.

  1. In my browser, I went to the Webfaction Control Panel at https://panel.webfaction.com/.
  2. Under "> Domains / websites", I selected "Applications".
  3. I removed my current Django installation by clicking the icon with the minus sign.
  4. I clicked the icon with the plus sign to add a new application.
  5. I filled in the following information:
    "Name:": django
    "App type:": Django 0.96.2/mod_python 3.3.1/Python 2.5
    Then I clicked the "Create" button.
  6. Finally, I needed to specify which URL path I want to use with my Django application.
    Under "> Domains / websites", I selected "Websites".
  7. I selected the icon with the pencil to edit the website settings.
  8. In the "Site apps:" section, I selected the icon with the plus sign to add a new App and path. I selected "django" as my "App:" and entered "/" (without the quotes) in the "URL path" field. Then I clicked "Update".
  9. After a couple minutes, I was able to view the Django It worked! page at http://sofeng.webfactional.com.

Upload my project

  1. The welcome message recommends using sftp, but I used rsync instead because I'm more familiar with it:
    sofeng@home:~ $ rsync -avz ~/src/django/myblogsite [email protected]:webapps/django

Deploy
  1. I ssh'd into my webfaction shell account:
    sofeng@home:~ $ ssh [email protected]
  2. I set the PYTHONPATH. (I actually put this in my .bash_profile)
    sofeng@web36:~ $ export PYTHONPATH=$HOME/webapps/django
  3. sofeng@web36:~ $ cd ~/webapps/django/myblogsite
  4. I tried to do a python2.5 manage.py syncdb, but got a database error:
    Traceback (most recent call last):
      File "manage.py", line 11, in 
        execute_manager(settings)
      File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 1672, in execute_manager
        execute_from_command_line(action_mapping, argv)
      File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 1571, in execute_from_command_line
        action_mapping[action](int(options.verbosity), options.interactive)
      File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 504, in syncdb
        cursor = connection.cursor()
      File "/home/sofeng/webapps/django/lib/python2.5/django/db/backends/sqlite3/base.py", line 58, in cursor
        self.connection = Database.connect(**kwargs)
    sqlite3.OperationalError: unable to open database file
  5. I needed to edit my settings.py file. I changed the following lines:
    DATABASE_NAME = '/home/sofeng/webapps/django/myblogsite/mydatabase.sqlite3'
    TEMPLATE_DIRS = (
        '/home/sofeng/webapps/django/myblogsite/templates',
    )
  6. I tried python2.5 manage.py syncdb again:
    $ python2.5 manage.py syncdb
    Loading 'initial_data' fixtures...
    No fixtures found.
    It worked.
  7. The next step was to set up the Apache configuration.
    $ cd ~/webapps/django/apache2/conf
  8. I edited the following line in httpd.conf:
        SetEnv DJANGO_SETTINGS_MODULE myblogsite.settings
  9. Then I restarted the Apache server:
    $ ~/webapps/django/apache2/bin/restart
  10. In my browser, I went to http://sofeng.webfactional.com/myview1, but got an error:
    ImportError at /myview1/
    No module named blog.models
    Request Method:  GET
    Request URL:  http://sofeng.webfactional.com/myview1/
    Exception Type:  ImportError
    Exception Value:  No module named blog.models
    Exception Location:  /home/sofeng/webapps/django/lib/python2.5/django/core/urlresolvers.py in _get_urlconf_module, line 177
  11. I guess I need my project on the Python path. I edited the following line in httpd.conf:
        PythonPath "['/home/sofeng/webapps/django/myblogsite', '/home/sofeng/webapps/django', '/home/sofeng/webapps/django/lib/python2.5'] + sys.path"
  12. Then I went back to http://sofeng.webfactional.com/myview1 and saw my page. Success!

Install Mercurial

Update 6/7/2008: By default Webfaction doesn't come with Mercurial installed, but they do allow you to do a user installation. I decided it would be useful to have Mercurial so I could pull and merge changes with greater control. Here are my steps.This will do a user-wide (as opposed to system-wide) installation of Mercurial in ~/bin and ~/lib.

  1. sofeng@home:~/incoming $ wget http://www.selenic.com/mercurial/release/mercurial-1.0.1.tar.gz
  2. sofeng@home:~/incoming $ scp mercurial-1.0.1.tar.gz [email protected]:
  3. sofeng@web36:~ $ mv mercurial-1.0.1.tar.gz ~/tmp
  4. $ cd tmp
  5. $ tar zxvf mercurial-1.0.1.tar.gz
  6. $ cd mercurial-1.0.1
  7. $ make install-home
  8. $ export PYTHONPATH=$HOME/lib/python:$PYTHONPATH
    Note: you will need to put this in your .bashrc instead of your .bash_profile because Mercurial only executes .bashrc on a remote hg push by default.
  9. $ hg version
    Mercurial Distributed SCM (version 1.0.1)
    
    Copyright (C) 2005-2008 Matt Mackall  and others
    This is free software; see the source for copying conditions. There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Other Errors
  • No module name django.core.management
    $ python manage.py syncdb
    Traceback (most recent call last):
      File "manage.py", line 2, in ?
        from django.core.management import execute_manager
    ImportError: No module named django.core.management
    You need to set the PYTHONPATH:
    $ export PYTHONPATH=$HOME/webapps/django
    You may get this if you use python manage.py syncdb instead of python2.5 manage.py syncdb.

Related posts:
  Django Blog Project #1: Creating a basic blog
  Django Blog Project #3: Using CSS and Template Inheritance
  Django Blog Project #4: Adding post metadata
  Django Blog Project #5: YUI CSS and serving static media

Django Blog Project #1: Creating a basic blog

It's been a while since my last post on Django. I became very busy but also found the Django tutorial to be somewhat dryand also somewhat lazy*. Luckily, the official Django Book was published during this time and it is more interesting to read. This post will be a very brief summary of the first 6 chapters of the Django Book as I apply it towards the creation of a new blog website. I highly recommend reading the book (I forgot to mention it is available free online). Then after reading the first six chapters, I hope this post can serve as kind of a refresher on how to put everything together.

As I mentioned, I decided to create my own blog site as my first Django project. I know it is not the most original idea in the world, but I thought it would be useful and a good learning experience. The following steps are my first cut at my new blog (dubbed 0.0.1) and basically just document the basic concepts of Django without providing much actual functionality.

I develop a model, a template, and a view in accordance with Django's MTV (see also MVC) development pattern. The model is of a blog post (aptly named Post) and contains only one attribute, the post body data (i.e. the actual text of the post). I should add in other data such as the title, date, tags, etc. But in order to keep things simple this first time around, it just has the post body. The model is connected to a SQLite database and is updated using Django's excellent admin interface. Finally, the model data is combined with a very basic template which just displays a title (My New Blog Version 0.0.1), and all the blog post bodies separated by a <hr>. Like I said, it's not very useful at this point, but I think I understand the basic concepts and how to put everything together much better now. The next step in my Django development will be to create some more interesting templates and views and add more useful data like titles and dates.

I also have a couple of related plans:

  • Set up hosting: I've decided to use WebFaction for my hosting but I need to set up and upload my new, almost-website there. This will probably be the subject of my next Django post.
  • Copy my Blogger posts over to my new site. I've already figured out how to use Beautiful Soup to screen scrape my Blogger posts and import them into SQLite. Likely I will do this further on in the process.

Here are the steps I took for my first cut at my new blog. Note, I'm running on Ubuntu so that's why I have the $ bash prompt and use /home/sofeng paths in my examples.

*Update 2008-10-09: I realize my assessment of the tutorial might have sounded critical-- in actuality, since I was job searching at the time, I was just trying to avoid sounding lazy. For the record, I find the Django documentation to be excellent and one of the highlights of the project.

Create a new project

The first thing to do after installing Django is to create a new project. Luckily, it takes just one command to create the project.

  1. Create a new project
    $ cd ~/src/django
    $ django-admin.py startproject myblogsite
  2. Take a look at the site using the development server
    $ python manage.py runserver
    Then go to http://127.0.0.1:8000

At first I thought the admin interface was kind of boring. However, for my blog site, I will use the admin interface to enter new blog posts.

  1. Edit myblogsite/settings.py to add the admin application to the list of installed apps:
    INSTALLED_APPS = (
       'django.contrib.auth',
       'django.contrib.contenttypes',
       'django.contrib.sessions',
       'django.contrib.sites',
       'django.contrib.admin',
    )
  2. Install database tables for the admin interface:
    $ python manage.py syncdb
    At this point I was asked to create a superuser to log into the admin interface. I answered "yes" and filled in the appropriate information.
    Creating table auth_message
    Creating table auth_group
    Creating table auth_user
    Creating table auth_permission
    Creating table django_content_type
    Creating table django_session
    Creating table django_site
    Creating table django_admin_log
    
    You just installed Django's auth system, which means you don't have any superusers defined.
    Would you like to create one now? (yes/no): yes
    Username (Leave blank to use 'sofeng'): sofeng
    E-mail address: [email protected]
    Password:
    Password (again):
    Superuser created successfully.
    Installing index for auth.Message model
    Installing index for auth.Permission model
    Installing index for admin.LogEntry model
    Loading 'initial_data' fixtures...
    No fixtures found.
  3. Edit myblogsite/urls.py to include the admin url.
    from django.conf.urls.defaults import *
    
    urlpatterns = patterns('',
       (r'^admin/', include('django.contrib.admin.urls')),
    )
  4. Run the development server:
    $ python manage.py runserver
    Then go to http://127.0.0.1:8000/admin Log in and take a look around.
Set up the SQLite3 database

I chose SQLite because it is a lightweight, simple alternative to MySQL or PostgreSQL. This makes it great for a development website.

  1. Edit the following section in the myblogsite/settings.py file:
    DATABASE_ENGINE = 'sqlite3'
    DATABASE_NAME = '/home/sofeng/src/django/myblogsite/mydatabase.sqlite3'
    The rest of the DATABASE_ variables are not used with SQLite.
  2. Test out the database configuration: Run the shell:
    $ python manage.py shell
    Then type these commands:
    >>> from django.db import connection
    >>> cursor = connection.cursor()
    If nothing happens, all is good. See Table 5-2 in Chapter 5 of the Django Book common database configuration error messages.
Create an App

I think of an "app" as a piece of specific functionality of a website, whereas a project corresponds to a particular website. There can be many apps in a project. Also, apps can be used in more than one project. For more information about the differences between projects and apps see Chapter 5 of the Django Book.

  1. Create an app
    $ cd ~/src/django/myblogsite
    $ python manage.py startapp myblogapp
Create a Model

I created one model, the Post model. A model roughly corresponds to a SQL table. And each attribute in that model corresponds to a table row. I added the class Admin: so that my Post model would show up in the Admin interface (where I can insert the data).

  1. Edit myblogsite/myblogapp/models.py to look like the following:
    from django.db import models
    
    class Post(models.Model):
       body = models.TextField()
    
       # in the future I will add these other attributes
    #    title = models.CharField(maxlength=500)
    #    timestamp = models.CharField(maxlength=50)
    #    tags = models.CharField(maxlength=200)
    
       class Admin:
           pass
    Correction 7/6/2008: For the Post's body field, I previously used the line: body = models.CharField(maxlength=999999). However, thanks to Myles's comment in my post #4, I've changed this to use the more appropriate TextField.
Install the Model

After writing the Python model code, I needed to create the actual tables in the SQLite database. The following steps include a couple of checks, then I create the tables in the last step.

  1. Edit myblogsite/settings.py file again and add the blog app to the list of installed apps:
    INSTALLED_APPS = (
       'django.contrib.auth',
       'django.contrib.contenttypes',
       'django.contrib.sessions',
       'django.contrib.sites',
       'myblogsite.myblogapp',
    )
  2. Try validating the model:
    $ python manage.py validate
    Which gives the following message:
    0 errors found.
  3. Check the CREATE TABLE statements that Django will generate. Note, the database won't be modified.
    $ python manage.py sqlall myblogapp
    Which yields the following:
    BEGIN;
    CREATE TABLE "myblogapp_post" (
       "id" integer NOT NULL PRIMARY KEY,
       "body" text NOT NULL
    );
    COMMIT;
    Correction 7/6/2008: I've updated the results here to reflect the correction I made to the model above.
  4. Now, actually create the tables in SQLite:
    $ python manage.py syncdb
    Which yields something like this:
    Creating table blog_post
    Loading 'initial_data' fixtures...
    No fixtures found.
Create some new data using the admin interface

Now that I created the models and tied them to the admin interface, I can start adding data using the admin interface.

  1. Start the development server again:
    $ python manage.py runserver
    Go to http://127.0.0.1:8000/admin and log in.
  2. Under the "Blog" heading, click "Posts", then add some new posts using "Add post" and the "Save" links. This will add data to the SQLite database.
Create a template

Now I will display the data I just created using a template and a view. The template holds all the HTML code and some simple Django template code which the view's Python code uses to customize the page.

  1. Create the file myblogsite/templates/mytemplate.html and put the following inside:
    <html>
     <head><title>Post</title></head>
     <body>
       <h1>My New Blog Version 0.0.1</h1>
    
       {% for post in post_list %}
       {{ post }}
       <hr />
       {% endfor %}
    
     </body>
    </html>
  2. Edit myblogsite/settings.py again to instruct Django where to find the template files.
    TEMPLATE_DIRS = (
       '/home/sofeng/src/django/myblogsite/templates',
    )
    Be sure to include the comma at the end.
Create a view

The view is where I will grab the data from my model and insert it into my template.

  1. Create a new file myblogsite/myblogapp/views.py and put the following inside:
    from django.shortcuts import render_to_response
    from myblogsite.myblogapp.models import Post
    
    def myview(request):
       posts = Post.objects.all()
       post_body_list = [post.body for post in posts]
       return render_to_response('mytemplate.html',
                                 {'post_list': post_body_list})
    Correction 7/6/2008: I previously had from myblogapp.models import Post on the second line. This works, but is inconsistent with my urls.py below and can (and did for me) cause subtle errors in the future. I corrected the line to read: from myblogsite.myblogapp.models import Post.
Map an URL to the new view

Finally, I map an URL to my newly created view.

  1. Edit myblogsite/urls.py so that it looks like:
    from django.conf.urls.defaults import *
    from myblogsite.myblogapp.views import myview
    
    urlpatterns = patterns('',
       (r'^admin/', include('django.contrib.admin.urls')),
       (r'^myview/$', myview),
    )
  2. Take a look at the new page: Run the server:
    $ python manage.py runserver
    Then go to http://127.0.0.1:8000/myview Visiting the url shows all the posts I entered through the admin interface. Nice. Here is a snapshot screenshot of my new blog:

That's it for now. I tried to map out the basic steps for using Django's MTV development pattern. Hopefully, in the future, I'll be able to add more useful features to my new Django-powered blog.


Related posts:
  Install Django on Ubuntu
  Django Blog Project #2: Deploying at Webfaction
  Django Blog Project #3: Using CSS and Template Inheritance
  Django Blog Project #4: Adding post metadata
  Django Blog Project #5: YUI CSS and serving static media
  Django Blog Project #6: Creating standard blog views
  Django Blog Project #7: Adding a simple Atom feed
  Django Blog Project #8: Adding basic comment functionality

How to paste in Cygwin bash using CTRL-V

I come from a Windows background where Cut/Copy/Paste are almost always CTRL+X/C/V respectively. I like this consistency, so adjusting to Cygwin's (or other *nix environments') use of SHIFT+INSERT or the middle mouse button for pasting in terminals is a little annoying. After a bit of googling, I found a mailing list thread that solved my problem.

This method uses a modification to readline's ~/.inputrc configuration file, so it should work in rxvt, xterm, or even the default cmd.exe shell. Here is what to do:

Add the following line to your ~/.bashrc:
stty lnext ^q stop undef start undef
And add the following line to your ~/.inputrc:
"\C-v": paste-from-clipboard

These modifications will unmap stty's lnext (literal next-character) and readline's quoted-insert settings which are normally mapped to CTRL-V. For more details, see this cygwin mailing list thread.

Note, this appears to be a Cygwin-specific solution. Linux users could use custom key bindings in gnome-terminal or konsole, or translation tables with xterm.


Useful commands for reference:
$ bind -p
$ stty -a

How to use Python's enumerate and zip to iterate over two lists and their indices.

enumerate- Iterate over indices and items of a list

The Python Cookbook (Recipe 4.4) describes how to iterate over items and indices in a list using enumerate. For example:

alist = ['a1', 'a2', 'a3']

for i, a in enumerate(alist):
    print i, a

Results:

0 a1
1 a2
2 a3

zip- Iterate over two lists in parallel

I previously wrote about using zip to iterate over two lists in parallel. Example:

alist = ['a1', 'a2', 'a3']
blist = ['b1', 'b2', 'b3']

for a, b in zip(alist, blist):
    print a, b

Results:

a1 b1
a2 b2
a3 b3

enumerate with zip

Here is how to iterate over two lists and their indices using enumerate together with zip:

alist = ['a1', 'a2', 'a3']
blist = ['b1', 'b2', 'b3']

for i, (a, b) in enumerate(zip(alist, blist)):
    print i, a, b

Results:

0 a1 b1
1 a2 b2
2 a3 b3

Recommended books

I love having a subscription to Safari Books Online. Currently my company provides a free subscription, but if I get a new job, I might consider subscribing myself. Since I get to browse a number of books at no cost, I thought I'd note which books are my favorites. (Note, I am not being paid by Safari Books Online.)

General Software

  • Structure and Interpretation of Computer Programs, Second Edition, Harold Abelson and Gerald Jay Sussman, MIT Press, ?year?
    I learned about this book through a job posting. It might bring you to tears if you get it. I'm only in the second chapter. It is used in an introductory Computer Science course at MIT. It uses Scheme (Lisp) to demonstrate concepts.
    Available free online at: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html
    A free video lecture series is also available.

C

  • The C Programming Language, Second Edition, Brian W. Kernighan and Dennis M. Ritchie, Prentice Hall, 1988
    The definitive C book.

Python

  • Core Python Programming, Second Edition, Wesley J. Chun, Prentice Hall, September 18, 2006
    Usually I like O'Reilly books best, but I slightly prefer Chun's text to Learning Python.
    Available at Safari Books Online
  • The Django Book, Apress, December 2007
    I think this is the first official Django book.

SQLite

  • The Definitive Guide to SQLite, Mike Owens, Apress, May 2006
    I browsed a few SQL books but liked this one better than most. It has a good theory section.
    Available at Apress.com

Linux or related

  • X Power Tools, Chris Tyler, O'Reilly, December 15, 2007
    Lots of good information on the X Window System and more; easy to understand. I wish the basic Ubuntu or Linux books had some of this information.
    Available at Safari Books Online
  • SSH, The Secure Shell: The Definitive Guide, Second Edition, Daniel J. Barrett; Richard E. Silverman; Robert G. Byrnes, O'Reilly, May 10, 2005
    Available at Safari Books Online

Ruby

Javascript

  • Javascript: The Good Parts, Douglas Crockford, O'Reilly, May 2008
    This is the first Javascript book I read as I switched to Javascript and Frontend development. I was debating whether to read it since it was so old but I found it to be good and recommend it.
  • You Don't Know JS: ES6 & Beyond, Kyle Simpson, O'Reilly, December 2015
    This is the primary way I learned ES6. It is very detail oriented.
    Available free online on github
  • JavaScript Allongé, the "Six" Edition, Reg "raganwald" Braithwaite, Leanpub, 2016
    This was the third Javascript book I read and it was the most fun. It teaches functional programming concepts such as closures, shadowing, higher order functions, combinators, decorators, etc using ES6.
    Available free online at: https://leanpub.com/javascriptallongesix/read

Non technical

  • The Hitchhiker's Guide to the Galaxy by Douglas Adams
  • Calvin and Hobbes by Bill Watterson
  • Crime and Punishment by Fyodor Dostoyevsky
  • Mere Christianity by C. S. Lewis
  • Screwtape Letters by C. S. Lewis

See also

How to install Mercurial 1.0 on Cygwin

Mercurial 1.0 has been released and I just posted, How to install Mercurial 1.0 on Ubuntu Gutsy (or Hardy). Here are instructions for installing Mercurial 1.0 on Cygwin.

Install Mercurial 1.0
  1. Install Python Easy Install
    $ cd /tmp
    $ wget http://peak.telecommunity.com/dist/ez_setup.py
    $ python ez_setup.py
  2. Install Mercurial 1.0
    $ easy_install -U mercurial
  3. Trying hg version, gives me a ImportError: Permission denied error. Apparently, there is a Eggs/Cygwin problem. To fix it, make the dll files executable:
    $ chmod +x /usr/lib/python2.5/site-packages/mercurial-1.0-py2.5-cygwin-1.5.25-i686.egg/mercurial/*.dll
Try hg version again:
Mercurial Distributed SCM (version 1.0)

Copyright (C) 2005-2008 Matt Mackall  and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Install hgk extension
I'm not sure how to get hg view working in Cygwin. If anyone knows, please let me know.

How to install Mercurial 1.0 on Ubuntu Hardy

Mercurial 1.0 is out! Mercurial is a next generation "fast, lightweight source control management system" boasting conversions by notable projects NetBeans, OpenJDK, Globulation2, Xine, Mozilla, grml, and OpenSolaris.

From the release notes, version 1.0 has improvements for copying, renaming, removing, and merging. I want to use the most robust version possible, so I decided to upgrade. Ubuntu Gutsy's Mercurial version is 0.9.4, and even Hardy will only have 0.9.5. Luckily, Mercurial 1.0 is easy-installable. That makes it easy to install:


Install Mercurial 1.0
  1. If you don't already have Mercurial 0.9.4 installed, I'd suggest installing it because it will setup /etc/mercurial for using Mercurial extenstions.
    sudo apt-get install mercurial
  2. Install Python Easy Install, build tools, and Python header files.
    sudo apt-get install python-setuptools python-dev build-essential
  3. Install Mercurial 1.0
    sudo easy_install -U mercurial
That was easy! Test it out by typing hg version.
Mercurial Distributed SCM (version 1.0)

Copyright (C) 2005-2008 Matt Mackall  and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Upgrade to 1.0.1
Update 6/2/2008: Mercurial 1.0.1 has been released as a bugfix version. You can upgrade from 1.0 to 1.0.1 using Easy Install:
sudo easy_install -U mercurial

Install hgk extension
If you try the hg view command, you will get a sh: hgk: not found error. To fix this, I copied the hgk executable from the contrib directory in the source package to a directory in my path:
  1. Download and unpack the Mercurial source package.
    cd /tmp
    wget http://www.selenic.com/mercurial/release/mercurial-1.0.tar.gz
    tar zxvf mercurial-1.0.tar.gz
  2. Copy hgk somewhere on your path. E.g.:
    cp mercurial-1.0/contrib/hgk ~/bin
Note, if you get a /usr/bin/env: wish: No such file or directory error, it means you need to install the Tk package:
sudo apt-get install tk8.5

Other Notes
  • hbisect warning
    Per the Upgrade Notes, bisect is now a built-in command and the hbisect extension should not be used. If you get a extension 'hgext/hbisect' overrides commands: bisect warning, remove the hbisect= line from your configuration file (located at /etc/mercurial/hgrc.d/hgext.rc or ~/.hgrc).

  • Where are the files located?
    The hgext and mercurial files should be located at /usr/lib/python2.5/site-packages/mercurial-1.0-py2.5-linux-*.egg

  • stdio.h error during easy_install
    If you got the following error while trying to run easy_install, it probably means you don't have the "build-essential" package installed. $ sudo apt-get install build-essential.
    In file included from /usr/lib/gcc/i486-linux-gnu/4.2.3/include/syslimits.h:7,
                     from /usr/lib/gcc/i486-linux-gnu/4.2.3/include/limits.h:11,
                     from /usr/include/python2.5/Python.h:18,
                     from mercurial/mpatch.c:23:
    /usr/lib/gcc/i486-linux-gnu/4.2.3/include/limits.h:122:61: error: limits.h: No such file or directory
    In file included from mercurial/mpatch.c:23:
    /usr/include/python2.5/Python.h:32:19: error: stdio.h: No such file or directory
    /usr/include/python2.5/Python.h:34:5: error: #error "Python.h requires that stdio.h define NULL."
    /usr/include/python2.5/Python.h:37:20: error: string.h: No such file or directory
    /usr/include/python2.5/Python.h:39:19: error: errno.h: No such file or directory
    /usr/include/python2.5/Python.h:41:20: error: stdlib.h: No such file or directory
    /usr/include/python2.5/Python.h:43:20: error: unistd.h: No such file or directory
    /usr/include/python2.5/Python.h:55:20: error: assert.h: No such file or directory
    In file included from /usr/include/python2.5/Python.h:57,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyport.h:7:20: error: stdint.h: No such file or directory
    In file included from /usr/include/python2.5/Python.h:57,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyport.h:89: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_uintptr_t’
    /usr/include/python2.5/pyport.h:90: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_intptr_t’
    /usr/include/python2.5/pyport.h:113: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_ssize_t’
    /usr/include/python2.5/pyport.h:231:76: error: math.h: No such file or directory
    /usr/include/python2.5/pyport.h:238:22: error: sys/time.h: No such file or directory
    /usr/include/python2.5/pyport.h:239:18: error: time.h: No such file or directory
    /usr/include/python2.5/pyport.h:257:24: error: sys/select.h: No such file or directory
    /usr/include/python2.5/pyport.h:296:22: error: sys/stat.h: No such file or directory
    In file included from /usr/include/python2.5/Python.h:76,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pymem.h:50: warning: parameter names (without types) in function declaration
    /usr/include/python2.5/pymem.h:51: error: expected declaration specifiers or ‘...’ before ‘size_t’
    In file included from /usr/include/python2.5/Python.h:78,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/object.h:104: error: expected specifier-qualifier-list before ‘Py_ssize_t’
    /usr/include/python2.5/object.h:108: error: expected specifier-qualifier-list before ‘Py_ssize_t’
    /usr/include/python2.5/object.h:131: error: expected declaration specifiers or ‘...’ before ‘*’ token
    /usr/include/python2.5/object.h:131: warning: type defaults to ‘int’ in declaration of ‘Py_ssize_t’
    /usr/include/python2.5/object.h:131: error: ‘Py_ssize_t’ declared as function returning a function
    /usr/include/python2.5/object.h:131: warning: function declaration isn’t a prototype
    /usr/include/python2.5/object.h:149: error: ‘readbufferproc’ declared as function returning a function
    /usr/include/python2.5/object.h:150: error: ‘writebufferproc’ declared as function returning a function
    /usr/include/python2.5/object.h:151: error: ‘segcountproc’ declared as function returning a function
    /usr/include/python2.5/object.h:152: error: ‘charbufferproc’ declared as function returning a function
    /usr/include/python2.5/object.h:215: error: expected specifier-qualifier-list before ‘lenfunc’
    /usr/include/python2.5/object.h:229: error: expected specifier-qualifier-list before ‘lenfunc’
    /usr/include/python2.5/object.h:244: error: expected declaration specifiers or ‘...’ before ‘FILE’
    /usr/include/python2.5/object.h:258: warning: ‘struct _typeobject’ declared inside parameter list
    /usr/include/python2.5/object.h:258: warning: its scope is only this definition or declaration, which is probably not what you want
    /usr/include/python2.5/object.h:259: warning: ‘struct _typeobject’ declared inside parameter list
    /usr/include/python2.5/object.h:262: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/object.h:262: error: field ‘ob_size’ declared as a function
    /usr/include/python2.5/object.h:264: error: field ‘tp_basicsize’ declared as a function
    /usr/include/python2.5/object.h:264: error: field ‘tp_itemsize’ declared as a function
    /usr/include/python2.5/object.h:309: error: field ‘tp_weaklistoffset’ declared as a function
    /usr/include/python2.5/object.h:324: error: field ‘tp_dictoffset’ declared as a function
    /usr/include/python2.5/object.h:389: error: expected declaration specifiers or ‘...’ before ‘FILE’
    In file included from /usr/include/python2.5/Python.h:79,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/objimpl.h:97: warning: parameter names (without types) in function declaration
    /usr/include/python2.5/objimpl.h:98: error: expected declaration specifiers or ‘...’ before ‘size_t’
    /usr/include/python2.5/objimpl.h:228: error: ‘PyGC_Collect’ declared as function returning a function
    /usr/include/python2.5/objimpl.h:249: error: field ‘gc_refs’ declared as a function
    /usr/include/python2.5/objimpl.h:288: warning: parameter names (without types) in function declaration
    In file included from /usr/include/python2.5/Python.h:83,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/unicodeobject.h:55:19: error: ctype.h: No such file or directory
    /usr/include/python2.5/unicodeobject.h:118:21: error: wchar.h: No such file or directory
    In file included from /usr/include/python2.5/Python.h:83,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/unicodeobject.h:384: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/unicodeobject.h:385: error: field ‘length’ declared as a function
    /usr/include/python2.5/unicodeobject.h:447: error: ‘PyUnicodeUCS4_GetSize’ declared as function returning a function
    /usr/include/python2.5/unicodeobject.h:521: warning: type defaults to ‘int’ in declaration of ‘wchar_t’
    /usr/include/python2.5/unicodeobject.h:521: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
    /usr/include/python2.5/unicodeobject.h:539: warning: type defaults to ‘int’ in declaration of ‘wchar_t’
    /usr/include/python2.5/unicodeobject.h:539: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
    /usr/include/python2.5/unicodeobject.h:1102: error: ‘PyUnicodeUCS4_Tailmatch’ declared as function returning a function
    /usr/include/python2.5/unicodeobject.h:1114: error: ‘PyUnicodeUCS4_Find’ declared as function returning a function
    /usr/include/python2.5/unicodeobject.h:1123: error: ‘PyUnicodeUCS4_Count’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:84,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/intobject.h:24: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/intobject.h:38: warning: parameter names (without types) in function declaration
    /usr/include/python2.5/intobject.h:41: error: ‘PyInt_AsSsize_t’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:86,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/longobject.h:25: error: ‘_PyLong_AsSsize_t’ declared as function returning a function
    /usr/include/python2.5/longobject.h:26: warning: parameter names (without types) in function declaration
    /usr/include/python2.5/longobject.h:69: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘_PyLong_NumBits’
    /usr/include/python2.5/longobject.h:85: error: expected declaration specifiers or ‘...’ before ‘size_t’
    /usr/include/python2.5/longobject.h:108: error: expected declaration specifiers or ‘...’ before ‘size_t’
    In file included from /usr/include/python2.5/Python.h:87,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/floatobject.h:15: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:89,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/complexobject.h:39: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:92,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/stringobject.h:36: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/stringobject.h:36: error: field ‘ob_size’ declared as a function
    /usr/include/python2.5/stringobject.h:67: error: ‘PyString_Size’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:94,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/tupleobject.h:25: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/tupleobject.h:25: error: field ‘ob_size’ declared as a function
    /usr/include/python2.5/tupleobject.h:40: error: ‘PyTuple_Size’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:95,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/listobject.h:23: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/listobject.h:23: error: field ‘ob_size’ declared as a function
    /usr/include/python2.5/listobject.h:38: error: field ‘allocated’ declared as a function
    /usr/include/python2.5/listobject.h:47: error: ‘PyList_Size’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:96,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/dictobject.h:55: error: field ‘me_hash’ declared as a function
    /usr/include/python2.5/dictobject.h:71: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/dictobject.h:72: error: field ‘ma_fill’ declared as a function
    /usr/include/python2.5/dictobject.h:73: error: field ‘ma_used’ declared as a function
    /usr/include/python2.5/dictobject.h:79: error: field ‘ma_mask’ declared as a function
    /usr/include/python2.5/dictobject.h:108: error: ‘PyDict_Size’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:98,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/setobject.h:36: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/setobject.h:38: error: field ‘fill’ declared as a function
    /usr/include/python2.5/setobject.h:39: error: field ‘used’ declared as a function
    /usr/include/python2.5/setobject.h:45: error: field ‘mask’ declared as a function
    /usr/include/python2.5/setobject.h:79: error: ‘PySet_Size’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:99,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/methodobject.h:82: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:101,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/funcobject.h:22: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:102,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/classobject.h:13: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/classobject.h:24: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/classobject.h:31: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:103,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/fileobject.h:11: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/fileobject.h:12: error: expected specifier-qualifier-list before ‘FILE’
    /usr/include/python2.5/fileobject.h:38: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/fileobject.h:40: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
    /usr/include/python2.5/fileobject.h:57: error: expected declaration specifiers or ‘...’ before ‘FILE’
    /usr/include/python2.5/fileobject.h:58: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_UniversalNewlineFread’
    In file included from /usr/include/python2.5/Python.h:105,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/traceback.h:13: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:106,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/sliceobject.h:23: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:107,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/cellobject.h:10: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:109,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/genobject.h:13: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:110,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/descrobject.h:46: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/descrobject.h:50: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/descrobject.h:55: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/descrobject.h:60: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/descrobject.h:65: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:111,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/weakrefobject.h:16: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/weakrefobject.h:65: error: ‘_PyWeakref_GetWeakrefCount’ declared as function returning a function
    In file included from /usr/include/python2.5/Python.h:114,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyerrors.h:10: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/pyerrors.h:17: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/pyerrors.h:31: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/pyerrors.h:44: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/pyerrors.h:52: error: field ‘ob_refcnt’ declared as a function
    /usr/include/python2.5/pyerrors.h:329: error: expected declaration specifiers or ‘...’ before ‘size_t’
    /usr/include/python2.5/pyerrors.h:330: error: format string argument not a string type
    /usr/include/python2.5/pyerrors.h:331: error: expected declaration specifiers or ‘...’ before ‘size_t’
    In file included from /usr/include/python2.5/Python.h:118,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyarena.h:50: error: expected declaration specifiers or ‘...’ before ‘size_t’
    In file included from /usr/include/python2.5/Python.h:120,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pythonrun.h:34: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:35: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:37: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:38: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:39: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:44: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:54: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:60: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:77: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:142: error: expected ‘)’ before ‘*’ token
    /usr/include/python2.5/pythonrun.h:144: error: expected ‘)’ before ‘*’ token
    In file included from /usr/include/python2.5/Python.h:122,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/sysmodule.h:12: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
    In file included from /usr/include/python2.5/Python.h:124,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/import.h:33: error: expected declaration specifiers or ‘...’ before ‘size_t’
    /usr/include/python2.5/import.h:33: error: expected declaration specifiers or ‘...’ before ‘FILE’
    In file included from /usr/include/python2.5/Python.h:126,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/abstract.h:421: error: ‘PyObject_Size’ declared as function returning a function
    /usr/include/python2.5/abstract.h:433: error: ‘PyObject_Length’ declared as function returning a function
    /usr/include/python2.5/abstract.h:436: error: ‘_PyObject_LengthHint’ declared as function returning a function
    /usr/include/python2.5/abstract.h:774: error: ‘PyNumber_AsSsize_t’ declared as function returning a function
    /usr/include/python2.5/abstract.h:947: error: ‘PySequence_Size’ declared as function returning a function
    /usr/include/python2.5/abstract.h:956: error: ‘PySequence_Length’ declared as function returning a function
    /usr/include/python2.5/abstract.h:1078: error: ‘PySequence_Count’ declared as function returning a function
    /usr/include/python2.5/abstract.h:1097: error: ‘_PySequence_IterSearch’ declared as function returning a function
    /usr/include/python2.5/abstract.h:1122: error: ‘PySequence_Index’ declared as function returning a function
    /usr/include/python2.5/abstract.h:1161: error: ‘PyMapping_Size’ declared as function returning a function
    /usr/include/python2.5/abstract.h:1171: error: ‘PyMapping_Length’ declared as function returning a function
    In file included from /usr/include/python2.5/compile.h:5,
                     from /usr/include/python2.5/Python.h:128,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/code.h:11: error: field ‘ob_refcnt’ declared as a function
    In file included from /usr/include/python2.5/Python.h:131,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pystrtod.h:11: error: expected declaration specifiers or ‘...’ before ‘size_t’
    In file included from /usr/include/python2.5/Python.h:151,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyfpe.h:129:20: error: signal.h: No such file or directory
    /usr/include/python2.5/pyfpe.h:130:20: error: setjmp.h: No such file or directory
    In file included from /usr/include/python2.5/Python.h:151,
                     from mercurial/mpatch.c:23:
    /usr/include/python2.5/pyfpe.h:132: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PyFPE_jbuf’
    mercurial/mpatch.c:54:24: error: sys/types.h: No such file or directory
    mercurial/mpatch.c:58:25: error: arpa/inet.h: No such file or directory
    mercurial/mpatch.c:60:23: error: inttypes.h: No such file or directory
    mercurial/mpatch.c: In function ‘lalloc’:
    mercurial/mpatch.c:77: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:77: error: (Each undeclared identifier is reported only once
    mercurial/mpatch.c:77: error: for each function it appears in.)
    mercurial/mpatch.c:82: warning: implicit declaration of function ‘malloc’
    mercurial/mpatch.c:82: warning: incompatible implicit declaration of built-in function ‘malloc’
    mercurial/mpatch.c:89: warning: implicit declaration of function ‘free’
    mercurial/mpatch.c: In function ‘combine’:
    mercurial/mpatch.c:197: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:224: warning: implicit declaration of function ‘memcpy’
    mercurial/mpatch.c:224: warning: incompatible implicit declaration of built-in function ‘memcpy’
    mercurial/mpatch.c: In function ‘decode’:
    mercurial/mpatch.c:244: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:249: warning: incompatible implicit declaration of built-in function ‘memcpy’
    mercurial/mpatch.c:250: warning: implicit declaration of function ‘ntohl’
    mercurial/mpatch.c:250: error: ‘uint32_t’ undeclared (first use in this function)
    mercurial/mpatch.c:250: error: expected expression before ‘)’ token
    mercurial/mpatch.c:251: error: expected expression before ‘)’ token
    mercurial/mpatch.c:252: error: expected expression before ‘)’ token
    mercurial/mpatch.c: In function ‘apply’:
    mercurial/mpatch.c:310: warning: incompatible implicit declaration of built-in function ‘memcpy’
    mercurial/mpatch.c:317: warning: incompatible implicit declaration of built-in function ‘memcpy’
    mercurial/mpatch.c: In function ‘fold’:
    mercurial/mpatch.c:330: warning: passing argument 2 of ‘PyList_GetItem’ makes pointer from integer without a cast
    mercurial/mpatch.c:332: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:335: warning: passing argument 2 of ‘decode’ makes integer from pointer without a cast
    mercurial/mpatch.c: In function ‘patches’:
    mercurial/mpatch.c:355: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:360: error: ‘PyObject’ has no member named ‘ob_refcnt’
    mercurial/mpatch.c:371: warning: passing argument 1 of ‘calcsize’ makes integer from pointer without a cast
    mercurial/mpatch.c:376: warning: passing argument 2 of ‘PyString_FromStringAndSize’ makes pointer from integer without a cast
    mercurial/mpatch.c:382: warning: passing argument 3 of ‘apply’ makes integer from pointer without a cast
    mercurial/mpatch.c:383: error: ‘PyObject’ has no member named ‘ob_refcnt’
    mercurial/mpatch.c:383: error: ‘PyObject’ has no member named ‘ob_type’
    mercurial/mpatch.c: In function ‘patchedsize’:
    mercurial/mpatch.c:401: error: ‘NULL’ undeclared (first use in this function)
    mercurial/mpatch.c:407: warning: incompatible implicit declaration of built-in function ‘memcpy’
    mercurial/mpatch.c:408: error: ‘uint32_t’ undeclared (first use in this function)
    mercurial/mpatch.c:408: error: expected expression before ‘)’ token
    mercurial/mpatch.c:409: error: expected expression before ‘)’ token
    mercurial/mpatch.c:410: error: expected expression before ‘)’ token
    mercurial/mpatch.c: At top level:
    mercurial/mpatch.c:435: error: ‘NULL’ undeclared here (not in a function)
    mercurial/mpatch.c: In function ‘initmpatch’:
    mercurial/mpatch.c:442: warning: passing argument 2 of ‘PyErr_NewException’ from incompatible pointer type
    mercurial/mpatch.c:442: warning: passing argument 3 of ‘PyErr_NewException’ from incompatible pointer type
    error: Setup script exited with error: command 'gcc' failed with exit status 1
    
  • Python.h error during easy_install
    If you got the following error while trying to run easy_install, it probably means you don't have the "python-dev" package installed. $ sudo apt-get install python-dev.
    Searching for mercurial
    Reading http://cheeseshop.python.org/pypi/mercurial/
    Couldn't find index page for 'mercurial' (maybe misspelled?)
    Scanning index of all packages (this may take a while)
    Reading http://cheeseshop.python.org/pypi/
    Reading http://cheeseshop.python.org/pypi/Mercurial/1.0
    Reading http://www.selenic.com/mercurial
    Best match: mercurial 1.0
    Downloading http://www.selenic.com/mercurial/release/mercurial-1.0.tar.gz
    Processing mercurial-1.0.tar.gz
    Running mercurial-1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-MfmZhU/mercurial-1.0/egg-dist-tmp-JDYH6V
    mercurial/mpatch.c:23:20: error: Python.h: No such file or directory
    mercurial/mpatch.c:64: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
    mercurial/mpatch.c: In function 'lalloc':
    mercurial/mpatch.c:92: warning: implicit declaration of function 'PyErr_Occurred'
    mercurial/mpatch.c:93: warning: implicit declaration of function 'PyErr_NoMemory'
    mercurial/mpatch.c: In function 'decode':
    mercurial/mpatch.c:265: warning: implicit declaration of function 'PyErr_SetString'
    mercurial/mpatch.c:265: error: 'mpatch_Error' undeclared (first use in this function)
    mercurial/mpatch.c:265: error: (Each undeclared identifier is reported only once
    mercurial/mpatch.c:265: error: for each function it appears in.)
    mercurial/mpatch.c: In function 'calcsize':
    mercurial/mpatch.c:283: error: 'mpatch_Error' undeclared (first use in this function)
    mercurial/mpatch.c: In function 'apply':
    mercurial/mpatch.c:306: error: 'mpatch_Error' undeclared (first use in this function)
    mercurial/mpatch.c: At top level:
    mercurial/mpatch.c:322: error: expected ')' before '*' token
    mercurial/mpatch.c:344: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
    mercurial/mpatch.c:392: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
    mercurial/mpatch.c:432: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'methods'
    mercurial/mpatch.c:439: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'initmpatch'
    error: Setup script exited with error: command 'gcc' failed with exit status 1
  • Couldn't find a setup script Error:
    If you get the following error when trying to run sudo easy_install -U mercurial, it could mean that there is a "mercurial" directory in your current working directory. Change to a different directory and run the command again.
    Processing mercurial
    error: Couldn't find a setup script in mercurial
  • error: can't create or remove files in install directory
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/lib/python2.5/site-packages/test-easy-install-28328.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/lib/python2.5/site-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
      
      http://peak.telecommunity.com/EasyInstall.html
    
    Please make the appropriate changes for your system and try again.
    You will get this error if you don't use the sudo command. Use
    $ sudo easy_install -U mercurial

Working with files and directories in Python

I often have a difficult time finding the appropriate functions to work with files and directories in Python. I think one reason is because the Library Reference puts seemingly related functions in two different places: 11 File and Directory Access and 14.1.4 Files and Directories.

Section 11, File and Directory Access contains documentation for useful functions such as os.path.exists which checks if a path exists, glob.glob which is useful for matching filenames using the Unix-style * and ? wildcards, and shutil.copy which is similary to the Unix cp command.

This section includes a total of 11 subsections for modules related to file and directory access, but it does not contain basic commands such as os.chdir, os.listdir, or os.rename. These functions are documented in Section 14.1.4, Files and Directories, instead.

Here is a summary of some of the functions I find useful. Check the documentation for more complete and detailed information.

11 File and Directory Access
  • os.path module:
    • exists - checks if a path or file exists
      Example:
      import os.path
      print os.path.exists("c:/Windows")
      Results:
      True
    • isfile and isdir - test if the path is a file or directory, respectively.
      Example:
      import os.path
      print os.path.isfile("c:/Windows")
      print os.path.isdir("c:/Windows")
      Results:
      False
      True
    • getmtime - returns the modification time of a path
      Example:
      import os.path
      import time
      mtime = os.path.getmtime("c:/Windows")
      print time.gmtime(mtime)
      Results:
      (2008, 4, 2, 15, 58, 39, 2, 93, 0)
  • glob module:
    • glob: returns a list of paths matching a Unix-style glob pattern.
      Example:
      import glob
      print glob.glob("c:/windows/*.bmp")
      Results:
      ['c:/windows\\Blue Lace 16.bmp', 'c:/windows\\Coffee Bean.bmp', 'c:/windows\\default.bmp', 'c:/windows\\FeatherTexture.bmp', 'c:/windows\\Gone Fishing.bmp', 'c:/windows\\Greenstone.bmp', 'c:/windows\\Prairie Wind.bmp', 'c:/windows\\Rhododendron.bmp', 'c:/windows\\River Sumida.bmp', 'c:/windows\\Santa Fe Stucco.bmp', 'c:/windows\\Soap Bubbles.bmp', 'c:/windows\\winnt.bmp', 'c:/windows\\winnt256.bmp', 'c:/windows\\Zapotec.bmp']
  • shutil module:
    • copy - similar to Unix cp
    • copy2 - similar to Unix cp -p
    • copytree - similar to Unix cp -r
    • rmtree - similar to Unix rm -r
14 Generic Operating System Services - > 14.1 os --- Miscellaneous operating system interfaces -> 14.1.4 Files and Directories
  • chdir - change the current working directory
  • getcwd - return a string representing the current working directory
  • listdir - return a list of the names of the entries in the directory.
  • makedir - create a directory
  • remove - remove a file (this is identical to unlink)
  • rename - rename the file or directory
  • walk - walks a directory tree (see my os.walk example)