Django Blog Project #12: Adding Pygments syntax highlighting

Date: 2008-08-20 | Tags: django | 2 Comments

I've finally added automatic code highlighting to my blog. It uses Pygments to do the syntax highlighting and Beautiful Soup to find all the <pre> blocks to highlight. I still write my blog posts in HTML, but now add a class attribute to my <pre> tags to specify the Pygments lexer to use. For example, for python code, I use:

<pre class="python">
import this
def demo():
    pass</pre>

Which turns into:

import this
def demo():
    pass

I bought James Bennett's book, Practical Django Projects about a month ago and it has good information about creating a blog with Django. It also documented techniques for syntax highlighting which I used here. To summarize, I added a new attribute, called body_highlighted to my Post model. Then, I added a custom save() method which parses my original HTML with Beautiful Soup and highlights it with Pygments.

Model changes

Diff for ~/src/django/myblogsite/myblogapp/models.py:

--- a/myblogapp/models.py       Fri Aug 08 17:08:57 2008 -0700
+++ b/myblogapp/models.py       Wed Aug 20 15:39:20 2008 -0700
@@ -2,6 +2,8 @@
 from django.db import models
 from django.contrib.auth.models import User
 from django.contrib import admin
+from BeautifulSoup import BeautifulSoup
+from pygments import formatters, lexers, highlight
 
 class Post(models.Model):
     author = models.ForeignKey(User)
@@ -13,7 +15,28 @@
     tags = models.CharField(max_length=200, help_text="Space separated.")
     body = models.TextField()
     body_html = models.TextField(editable=False, blank=True)
+    body_highlighted = models.TextField(editable=False, blank=True)
     lc_count = models.IntegerField(default=0, editable=False)
+
+    def save(self):
+        self.body_highlighted = self.highlight_code(self.body)
+        super(Post, self).save()
+
+    def highlight_code(self, html):
+        soup = BeautifulSoup(html)
+        preblocks = soup.findAll('pre')
+        for pre in preblocks:
+            if pre.has_key('class'):
+                try:
+                    code = ''.join([unicode(item) for item in pre.contents])
+                    lexer = lexers.get_lexer_by_name(pre['class'])
+                    formatter = formatters.HtmlFormatter()
+                    code_hl = highlight(code, lexer, formatter)
+                    pre.contents = [BeautifulSoup(code_hl)]
+                    pre.name = 'div'
+                except:
+                    pass
+        return unicode(soup)
 
     def get_tag_list(self):
         return re.split(" ", self.tags)

Update the database

List the SQL commands Django would use:

$ cd ~/src/django/myblogsite/
$ python manage.py sqlall myblogapp

BEGIN;
CREATE TABLE "myblogapp_post" (
    "id" integer NOT NULL PRIMARY KEY,
    "author_id" integer NOT NULL REFERENCES "auth_user" ("id"),
    "title" varchar(200) NOT NULL,
    "slug" varchar(200) NOT NULL,
    "date_created" datetime NOT NULL,
    "date_modified" datetime NOT NULL,
    "tags" varchar(200) NOT NULL,
    "body" text NOT NULL,
    "body_html" text NOT NULL,
    "body_highlighted" text NOT NULL,
    "lc_count" integer NOT NULL
)
;
CREATE INDEX "myblogapp_post_author_id" ON "myblogapp_post" ("author_id");
CREATE INDEX "myblogapp_post_slug" ON "myblogapp_post" ("slug");
COMMIT;

Enter the sqlite3 shell:

$ sqlite3 mydatabase.sqlite3

and enter the following statements:

sqlite> ALTER TABLE myblogapp_post ADD COLUMN body_highlighted text;
sqlite> .exit

Update the template

Diff for ~/src/django/myblogsite/templates/singlepost.html:

--- a/templates/singlepost.html Fri Aug 08 17:08:57 2008 -0700
+++ b/templates/singlepost.html Wed Aug 20 14:41:15 2008 -0700
@@ -16,7 +16,11 @@
 {% block main %}
   <br />
   <h3 id="highlight">{{ post.title }}</h3>
-  {{ post.body|safe }}
+  {% if post.body_highlighted %}
+    {{ post.body_highlighted|safe }}
+  {% else %}
+    {{ post.body|safe }}
+  {% endif %}
   <div class="post_footer">
     {% if post.author.first_name %}
       Author: {{ post.author.first_name }}<br /></div>

Add CSS for Pygments

One last step is to add the CSS for Pygments. Here is an excerpt from my ~/src/django/myblogsite/media/css/mystyle.css:

/* PYGMENTS STYLE */
/* customized */
.c  { color: #008040; font-style: italic } /* Comment */
.cm { color: #008040; font-style: italic } /* Comment.Multiline */
.cp { color: #BC7A00 } /* Comment.Preproc */
.c1 { color: #008040; font-style: italic } /* Comment.Single */
.cs { color: #008040; font-style: italic } /* Comment.Special */
.gd { color: grey; text-decoration: line-through } /* Generic.Deleted */
.gi { color: red; } /* Generic.Inserted */
.k  { color: #000080; font-weight: bold } /* Keyword */
.kc { color: #000000; font-weight: bold } /* Keyword.Constant */
.kd { color: #000000; font-weight: bold } /* Keyword.Declaration */
.kp { color: #000000 } /* Keyword.Pseudo */
.kr { color: #000000; font-weight: bold } /* Keyword.Reserved */
.kt { color: #000000; font-weight: bold } /* Keyword.Type */

/* original settings */
.err { border: 1px solid #FF0000 } /* Error */
.o { color: #666666 } /* Operator */
.ge { font-style: italic } /* Generic.Emph */
.gr { color: #FF0000 } /* Generic.Error */
.gh { color: #000080; font-weight: bold } /* Generic.Heading */
.go { color: #808080 } /* Generic.Output */
.gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.gs { font-weight: bold } /* Generic.Strong */
.gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.gt { color: #0040D0 } /* Generic.Traceback */
.m { color: #666666 } /* Literal.Number */
.s { color: #BA2121 } /* Literal.String */
.na { color: #7D9029 } /* Name.Attribute */
.nb { color: #008000 } /* Name.Builtin */
.nc { color: #0000FF; font-weight: bold } /* Name.Class */
.no { color: #880000 } /* Name.Constant */
.nd { color: #AA22FF } /* Name.Decorator */
.ni { color: #999999; font-weight: bold } /* Name.Entity */
.ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.nf { color: #0000FF } /* Name.Function */
.nl { color: #A0A000 } /* Name.Label */
.nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.nt { color: #008000; font-weight: bold } /* Name.Tag */
.nv { color: #19177C } /* Name.Variable */
.ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.w { color: #bbbbbb } /* Text.Whitespace */
.mf { color: #666666 } /* Literal.Number.Float */
.mh { color: #666666 } /* Literal.Number.Hex */
.mi { color: #666666 } /* Literal.Number.Integer */
.mo { color: #666666 } /* Literal.Number.Oct */
.sb { color: #BA2121 } /* Literal.String.Backtick */
.sc { color: #BA2121 } /* Literal.String.Char */
.sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.s2 { color: #BA2121 } /* Literal.String.Double */
.se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.sh { color: #BA2121 } /* Literal.String.Heredoc */
.si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.sx { color: #008000 } /* Literal.String.Other */
.sr { color: #BB6688 } /* Literal.String.Regex */
.s1 { color: #BA2121 } /* Literal.String.Single */
.ss { color: #19177C } /* Literal.String.Symbol */
.bp { color: #008000 } /* Name.Builtin.Pseudo */
.vc { color: #19177C } /* Name.Variable.Class */
.vg { color: #19177C } /* Name.Variable.Global */
.vi { color: #19177C } /* Name.Variable.Instance */
.il { color: #666666 } /* Literal.Number.Integer.Long */

All pau. Now we should have pretty syntax highlighted code! (For those keeping track, this is now version 0.1.3 of my blog.)

2 Comments

#1 mapleoin commented on 2008-09-28:

This post was excellent. Thanks!

There's one problem in the view (mentioned in the django docs here. I found that modifying the save method to look like this works:

def save(self, force_insert=False, force_update=False):
    self.body_highlighted = self.highlight_code(self.body)
    super(Post, self).save()

#2 Kenny Meyer commented on 2010-01-23:

Thanks for this great post. I'm trying to implement Syntax Highlighting with pygments and you have provided a good base. I have a very similar set-up, though I guess this is written for an older Django version, because you're still using the unicode() wrapper.

Keep it up!


Name:	Required
Email:	Required, but not displayed
Website:	Optional
Comment:	Format using Markdown. (No HTML.) Code blocks: prefix each line by at least 4 spaces or 1 tab (and a blank line before and after) Code span: surround with backticks Blockquotes: prefix lines to be quoted with > Links: <URL> Links w/ description: [description](URL)

SaltyCrane Blog