SaltyCrane Blog — Notes on JavaScript and web development

Python datetime / time conversions

from datetime import datetime
import time

#-------------------------------------------------
# conversions to strings
#-------------------------------------------------
# datetime object to string
dt_obj = datetime(2008, 11, 10, 17, 53, 59)
date_str = dt_obj.strftime("%Y-%m-%d %H:%M:%S")
print date_str

# time tuple to string
time_tuple = (2008, 11, 12, 13, 51, 18, 2, 317, 0)
date_str = time.strftime("%Y-%m-%d %H:%M:%S", time_tuple)
print date_str

#-------------------------------------------------
# conversions to datetime objects
#-------------------------------------------------
# time tuple to datetime object
time_tuple = (2008, 11, 12, 13, 51, 18, 2, 317, 0)
dt_obj = datetime(*time_tuple[0:6])
print repr(dt_obj)

# date string to datetime object
date_str = "2008-11-10 17:53:59"
dt_obj = datetime.strptime(date_str, "%Y-%m-%d %H:%M:%S")
print repr(dt_obj)

# timestamp to datetime object in local time
timestamp = 1226527167.595983
dt_obj = datetime.fromtimestamp(timestamp)
print repr(dt_obj)

# timestamp to datetime object in UTC
timestamp = 1226527167.595983
dt_obj = datetime.utcfromtimestamp(timestamp)
print repr(dt_obj)

#-------------------------------------------------
# conversions to time tuples
#-------------------------------------------------
# datetime object to time tuple
dt_obj = datetime(2008, 11, 10, 17, 53, 59)
time_tuple = dt_obj.timetuple()
print repr(time_tuple)

# string to time tuple
date_str = "2008-11-10 17:53:59"
time_tuple = time.strptime(date_str, "%Y-%m-%d %H:%M:%S")
print repr(time_tuple)

# timestamp to time tuple in UTC
timestamp = 1226527167.595983
time_tuple = time.gmtime(timestamp)
print repr(time_tuple)

# timestamp to time tuple in local time
timestamp = 1226527167.595983
time_tuple = time.localtime(timestamp)
print repr(time_tuple)

#-------------------------------------------------
# conversions to timestamps
#-------------------------------------------------
# time tuple in local time to timestamp
time_tuple = (2008, 11, 12, 13, 59, 27, 2, 317, 0)
timestamp = time.mktime(time_tuple)
print repr(timestamp)

# time tuple in utc time to timestamp
time_tuple_utc = (2008, 11, 12, 13, 59, 27, 2, 317, 0)
timestamp_utc = calendar.timegm(time_tuple_utc)
print repr(timestamp_utc)

#-------------------------------------------------
# results
#-------------------------------------------------
# 2008-11-10 17:53:59
# 2008-11-12 13:51:18
# datetime.datetime(2008, 11, 12, 13, 51, 18)
# datetime.datetime(2008, 11, 10, 17, 53, 59)
# datetime.datetime(2008, 11, 12, 13, 59, 27, 595983)
# datetime.datetime(2008, 11, 12, 21, 59, 27, 595983)
# (2008, 11, 10, 17, 53, 59, 0, 315, -1)
# (2008, 11, 10, 17, 53, 59, 0, 315, -1)
# (2008, 11, 12, 21, 59, 27, 2, 317, 0)
# (2008, 11, 12, 13, 59, 27, 2, 317, 0)
# 1226527167.0
# 1226498367

How to escape (percent-encode) a URL with Python

import urllib

print urllib.quote_plus("http://www.yahoo.com/")
print urllib.quote_plus("Kruder & Dorfmeister")

Results:

http%3A%2F%2Fwww.yahoo.com%2F
Kruder+%26+Dorfmeister

It is easy to be drawn to the urlencode function in the Python urllib module documentation. But for simple escaping, only quote_plus, or possibly quote is needed. I believe this is the appropriate solution to Python urlencode annoyance and O'Reilly's Amazon Hack #92.

For reference: Percent-encoding on Wikipedia

How to set the font for new frames in Emacs 23

I had been using the following elisp to set my font in Emacs 23. However, it did not work when creating new frames with C-x 5 2.

(if (>= emacs-major-version 23)
    (set-default-font "Monospace-11"))

Here is how I set the font for all frames:

(if (>= emacs-major-version 23)
    (modify-all-frames-parameters
     '((font . "Monospace-11"))))

See the documentation for modify-all-frames-parameters for more information.

Running a Twisted Perspective Broker example with twistd

I've been using Twisted's Perspective Broker to manage networking for my Python program. Perspective Broker allows me to run a Python program on a remote machine and perform remote method calls on an object in the Python program. It also allows me to serialize objects and transfer them over TCP.

Once I got a Perspective Broker server and client running, I wanted to create a "Twisted Application" and run it using twistd, the Twisted Daemon. Two major options are: creating a .tac file and creating a twistd plugin. Below, I show the steps I took to create a .tac application script and a twistd plugin using a simple Perspective Broker example.

Basic Perspective Broker server and client

Here is a simple Perspective Broker server and client example which I adapted from Twisted's examples page. The PB server code lives in pbsimpleserver.py:

from twisted.spread import pb
from twisted.internet import reactor

class Echoer(pb.Root):
    def remote_echo(self, st):
        print 'echoing:', st
        return st

if __name__ == '__main__':
    serverfactory = pb.PBServerFactory(Echoer())
    reactor.listenTCP(8789, serverfactory)
    reactor.run()

Here is a simple PB client, pbsimpleclient.py:

from twisted.spread import pb
from twisted.internet import reactor

class EchoClient(object):
    def connect(self):
        clientfactory = pb.PBClientFactory()
        reactor.connectTCP("localhost", 8789, clientfactory)
        d = clientfactory.getRootObject()
        d.addCallback(self.send_msg)
        
    def send_msg(self, result):
        d = result.callRemote("echo", "hello network")
        d.addCallback(self.get_msg)

    def get_msg(self, result):
        print "server echoed: ", result
        
if __name__ == '__main__':
    EchoClient().connect()
    reactor.run()

This code connects the client to the server using port 8789. It sends a message by calling the server's remote_echo method with an argument of "hello network". The server prints the message and returns the same message to the client. The client then prints the message.

To test the code, I ran both client and server on my local machine. I ran python pbsimpleserver.py in one terminal shell, then ran python pbsimpleclient.py in another shell. In the server shell, I got:

echoing: hello network

and in the client shell I got:

server echoed:  hello network

To stop the client and server, I hit CTRL-C in both shells

Converting to a Twisted Application (.tac script)

To create a Twisted application, I used the twisted.application.service.Application object. When converting the example code above to an Application, I replaced twisted.internet.reactor.listenTCP with twisted.application.internet.TCPServer and twisted.internet.reactor.connectTCP with twisted.application.internet.TCPClient. Below is my server application, pbsimpleserver_app.py. (Note: the two files below are considered .tac files even though the filename doesn't end in .tac. From the documentation, .tac files are Python files which configure an Application object and assign this object to the top-level variable "application".)

from twisted.spread import pb
from twisted.application.internet import TCPServer
from twisted.application.service import Application

class Echoer(pb.Root):
    def remote_echo(self, st):
        print 'echoing:', st
        return st

serverfactory = pb.PBServerFactory(Echoer())
application = Application("echo")
echoServerService = TCPServer(8789, serverfactory)
echoServerService.setServiceParent(application)

Here is my client application, pbsimpleclient_app.py:

from twisted.spread import pb
from twisted.application.internet import TCPClient
from twisted.application.service import Application

class EchoClient(object):
    def send_msg(self, result):
        d = result.callRemote("echo", "hello network")
        d.addCallback(self.get_msg)

    def get_msg(self, result):
        print "server echoed: ", result

e = EchoClient()
clientfactory = pb.PBClientFactory()
d = clientfactory.getRootObject()
d.addCallback(e.send_msg)        

application = Application("echo")
echoClientService = TCPClient("localhost", 8789, clientfactory)
echoClientService.setServiceParent(application)

To run these as daemons with twistd, I executed:

twistd -l server.log --pidfile server.pid -y pbsimpleserver_app.py

then:

twistd -l client.log --pidfile client.pid -y pbsimpleclient_app.py

This created the log file, server.log:

2008-10-27 00:08:35-0700 [-] Log opened.
2008-10-27 00:08:35-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up
2008-10-27 00:08:35-0700 [-] reactor class: 
2008-10-27 00:08:35-0700 [-] twisted.spread.pb.PBServerFactory starting on 8789
2008-10-27 00:08:35-0700 [-] Starting factory 
2008-10-27 00:08:53-0700 [Broker,0,127.0.0.1] echoing: hello network

and client.log:

2008-10-27 00:08:53-0700 [-] Log opened.
2008-10-27 00:08:53-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up
2008-10-27 00:08:53-0700 [-] reactor class: 
2008-10-27 00:08:53-0700 [-] Starting factory 
2008-10-27 00:08:53-0700 [Broker,client] server echoed:  hello network

Creating a twistd plugin

In order to pass command line arguments to my Twisted application daemon, I need to create a twistd plugin. The following is how I implemented my plugin after reading the Writing a twistd plugin documentation. Here is my directory structure. EchoProj is located in ~/Projects.

EchoProj
|-- echoproj
|   |-- __init__.py
|   |-- pbsimpleclient.py
|   `-- pbsimpleserver.py
`-- twisted
    `-- plugins
        |-- echoclient_plugin.py
        `-- echoserver_plugin.py

pbsimpleserver.py:

from twisted.spread import pb

class EchoServer(pb.Root):
    def remote_echo(self, st):
        print 'echoing:', st
        return st

pbsimpleclient.py:

class EchoClient(object):
    def send_msg(self, result):
        d = result.callRemote("echo", "hello network")
        d.addCallback(self.get_msg)

    def get_msg(self, result):
        print "server echoed: ", result

echoserver_plugin.py:

from zope.interface import implements
from twisted.python import usage
from twisted.plugin import IPlugin
from twisted.application.service import IServiceMaker
from twisted.application.internet import TCPServer
from twisted.spread import pb
from echoproj.pbsimpleserver import EchoServer

class Options(usage.Options):
    optParameters = [["port", "p", 8789, "The port number to listen on."]]

class MyServiceMaker(object):
    implements(IServiceMaker, IPlugin)
    tapname = "echoserver"
    description = "Echo Server"
    options = Options

    def makeService(self, options):
        serverfactory = pb.PBServerFactory(EchoServer())
        return TCPServer(int(options["port"]), serverfactory)

serviceMaker = MyServiceMaker()

echoclient_plugin.py:

from zope.interface import implements
from twisted.python import usage
from twisted.plugin import IPlugin
from twisted.application.service import IServiceMaker
from twisted.application.internet import TCPClient
from twisted.spread import pb
from echoproj.pbsimpleclient import EchoClient

class Options(usage.Options):
    optParameters = [["port", "p", 8789, "The port number to connect to."],
                     ["host", "h", "localhost", "The host machine to connect to."]
                     ]

class MyServiceMaker(object):
    implements(IServiceMaker, IPlugin)
    tapname = "echoclient"
    description = "Echo Client"
    options = Options

    def makeService(self, options):
        e = EchoClient()
        clientfactory = pb.PBClientFactory()
        d = clientfactory.getRootObject()
        d.addCallback(e.send_msg)
        return TCPClient(options["host"], int(options["port"]), clientfactory)

serviceMaker = MyServiceMaker()

I set the PYTHONPATH to include the top-level project directory:

export PYTHONPATH="$HOME/Projects/EchoProj:$PYTHONPATH"

Running twistd --help now showed "echoserver" and "echoclient" in the list of commands. To run my server and client as daemons using port 8790 on my local machine, I executed:

twistd -l server.log --pidfile server.pid echoserver -p 8790

and

twistd -l client.log --pidfile client.pid echoclient -p 8790

This produced the logfiles, server.log:

2008-10-27 11:49:12-0700 [-] Log opened.
2008-10-27 11:49:12-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up
2008-10-27 11:49:12-0700 [-] reactor class: 
2008-10-27 11:49:12-0700 [-] twisted.spread.pb.PBServerFactory starting on 8790
2008-10-27 11:49:12-0700 [-] Starting factory 
2008-10-27 11:49:17-0700 [Broker,0,127.0.0.1] echoing: hello network

and client.log:

2008-10-27 11:49:17-0700 [-] Log opened.
2008-10-27 11:49:17-0700 [-] twistd 8.1.0 (/usr/bin/python 2.5.2) starting up
2008-10-27 11:49:17-0700 [-] reactor class: 
2008-10-27 11:49:17-0700 [-] Starting factory 
2008-10-27 11:49:17-0700 [Broker,client] server echoed:  hello network

Twisted links

Twisted tutorials

Twisted philosophy

Can't block for a Deferred in Twisted

Despite the existence of the promising waitForDeferred/deferredGenerator and the newer inlineCallbacks, it appears there is no way to block while waiting for a Deferred. Brian Granger described the problem on the Twisted mailing list:

I have a function that returns a Deferred. I need to have the result of this Deferred returned in a (apparently) blocking/synchronous manner:
def myfuncBlocking():
  d = myfuncReturnsDeferred()
  ...
  result =

  return result
I need to be able to call this function like:
result = myfuncBlocking()
The question is how to get the result out of the Deferred() and make it *look* like myfuncBlocking() has blocked.
glyph provided the succinct answer (as well as an interesting commentary on using Twisted the wrong way).
This issue has been discussed repeatedly - long story short, it's just a bad idea.

Hmmm, maybe learning Twisted will be harder than I thought.

Update 2008-10-20: Marcin Kasperski wrote a good example comparing raw deferreds, deferred generators, and inline callbacks.

Running functions periodically using Twisted's LoopingCall

Twisted is pretty cool-- it is very powerful, but I haven't had the easiest time learning it. Here is a simple example that runs a couple functions periodically (at different rates) using LoopingCall.

For more information, here are the Twisted docs for LoopingCall.

from datetime import datetime
from twisted.internet.task import LoopingCall
from twisted.internet import reactor

def hyper_task():
    print "I like to run fast", datetime.now()

def tired_task():
    print "I want to run slowly", datetime.now()

lc = LoopingCall(hyper_task)
lc.start(0.1)

lc2 = LoopingCall(tired_task)
lc2.start(0.5)

reactor.run()

Results:

I like to run fast 2008-10-14 15:51:02.449537
I want to run slowly 2008-10-14 15:51:02.449915
I like to run fast 2008-10-14 15:51:02.551972
I like to run fast 2008-10-14 15:51:02.652013
I like to run fast 2008-10-14 15:51:02.752006
I like to run fast 2008-10-14 15:51:02.852008
I like to run fast 2008-10-14 15:51:02.952487
I want to run slowly 2008-10-14 15:51:02.952681
I like to run fast 2008-10-14 15:51:03.052012
I like to run fast 2008-10-14 15:51:03.152012
I like to run fast 2008-10-14 15:51:03.252010
I like to run fast 2008-10-14 15:51:03.352009
I like to run fast 2008-10-14 15:51:03.452008
I want to run slowly 2008-10-14 15:51:03.452206
I like to run fast 2008-10-14 15:51:03.552009
I like to run fast 2008-10-14 15:51:03.652013

Using TimerService with twistd

To create a daemon with twistd that achieves the same effect, use TimerService. TimerService runs LoopingCall under the hood. It is meant to be used with the Twisted Application infrastructure. See also the documentation on TimerService.

timerservice_ex.py

from datetime import datetime
from twisted.application import service
from twisted.application.internet import TimerService

def tired_task():
    print "I want to run slowly", datetime.now()

application = service.Application("myapp")
ts = TimerService(0.5, tired_task)
ts.setServiceParent(application)

Run it:

$ twistd -y timerservice_ex.py 

Console output is stored in twistd.log:

2010-09-20 18:53:50-0700 [-] Log opened.
2010-09-20 18:53:50-0700 [-] using set_wakeup_fd
2010-09-20 18:53:50-0700 [-] twistd 10.1.0 (/home/saltycrane/.virtualenvs/default/bin/python 2.6.5) starting up.
2010-09-20 18:53:50-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-09-20 18:53:50-0700 [-] I want to run slowly 2010-09-20 18:53:50.896477
2010-09-20 18:53:51-0700 [-] I want to run slowly 2010-09-20 18:53:51.397043
2010-09-20 18:53:51-0700 [-] I want to run slowly 2010-09-20 18:53:51.897087
2010-09-20 18:53:52-0700 [-] I want to run slowly 2010-09-20 18:53:52.397047
2010-09-20 18:53:52-0700 [-] I want to run slowly 2010-09-20 18:53:52.897068
2010-09-20 18:53:53-0700 [-] I want to run slowly 2010-09-20 18:53:53.397073
2010-09-20 18:53:53-0700 [-] I want to run slowly 2010-09-20 18:53:53.897032
2010-09-20 18:53:54-0700 [-] I want to run slowly 2010-09-20 18:53:54.397083

Installing beanstalkd and pybeanstalk on Ubuntu

Nikolay pointed me at beanstalkd, a lightweight, message queue service partially inspired by the popular caching system, memcached. It features a blocking "reserve" call so workers don't need to poll for new jobs. However, some might miss the data persistence since the queue is stored in memory. beanstalkd has client libraries in Erlang, Perl, PHP, Python, and Ruby. Of course, I will use the Python version, pybeanstalk. Simon Willison also found beanstalkd interesting. (Credit to him for some of the words above.)

Update 2008-10-13: For a helpful example on using beanstalkd and pybeanstalk see Parand's beanstalkd tutorial.

Here is how I installed the beanstalkd server and pybeanstalk client on Ubuntu. I found no Ubuntu package for beanstalkd, so I installed from source.

Install beanstalkd
  • Install prerequistes
    $ sudo apt-get install libevent1 libevent-dev
  • Download
    $ cd ~/lib
    $ wget http://xph.us/software/beanstalkd/rel/beanstalkd-1.0.tar.gz
  • Unpack
    $ tar zxvf beanstalkd-1.0.tar.gz
  • Make
    $ cd beanstalkd-1.0
    $ make
  • Print help
    $ ./beanstalkd -h
Install PyYAML
  • Download
    $ wget http://pyyaml.org/download/pyyaml/PyYAML-3.06.tar.gz
  • Unpack
    $ tar zxvf PyYAML-3.06.tar.gz
  • Put PyYAML-3.06/lib/yaml somewhere on your Python path or run python setup.py.
Install pybeanstalk
  • Download
    $ wget http://pybeanstalk.googlecode.com/files/pybeanstalk-0.11.1.tar.gz
  • Unpack
    $ tar zxvf pybeanstalk-0.11.1.tar.gz
  • Put pybeanstalk-0.11.1/beanstalk somewhere on your Python path or run python setup.py
Run beanstalkd server
  • $ ~/lib/beanstalkd-1.0/beanstalkd -d -l 127.0.0.5 -p 11300
Run test client
  • Create a file and run it:
    from beanstalk import serverconn
    from beanstalk import job
    
    SERVER = '127.0.0.5'
    PORT = 11300
    
    # setup connection
    connection = serverconn.ServerConn(SERVER, PORT)
    connection.job = job.Job
    
    # produce data
    for i in range(5):
        print 'put data: %d' % i
        data = job.Job(data=str(i), conn=connection)
        data.Queue()
    
    # consume data
    while True:
        j = connection.reserve()
        print 'got data: %s' % j.data
        j.Finish()
    
    Results:
    put data: 0
    put data: 1
    put data: 2
    put data: 3
    put data: 4
    got data: 0
    got data: 1
    got data: 2
    got data: 3
    got data: 4

Installing Python 2.6 from source on Ubuntu Hardy

Python 2.6 was released yesterday! This version aims to smooth the transition from Python 2.5 to Python 3.0 which is planned for release soon (currently available as a release candidate). Python 3.0 will be break backwards compatibility with the 2.x series. Python 2.6 is backwards compatible with 2.5. All the backwards compatible features of 3.0 have been backported to 2.6.

One of the new 2.6 features I'm particularly intersted in is the new multiprocessing module which has a similar interface to the threading module, but it uses processes instead of threads. This avoids the limitations imposed by the Global Interpreter Lock in a multi-threaded Python program. Yet it still has the nice communications and management features like Pipe()s, Queues, Pools, etc. I didn't plan to focus so much on the multiprocessing module in this post-- I just want to document my install notes on Ubuntu Linux. For all the new features in 2.6 see What's New in Python 2.6. (It is a long list).

  • Download the Python 2.6 compressed source tarball
    $ cd incoming
    $ wget http://www.python.org/ftp/python/2.6/Python-2.6.tgz
  • Unpack
    $ tar zxvf Python-2.6.tgz
  • Read the README at ~/incoming/Python-2.6/README

  • Install prerequisites (Disclaimer: I know nothing about libraries, packages, dependencies, etc. This is what I did-- I am not sure if I grabbed the correct prerequisites or not.)
    $ sudo apt-get install build-essential
    $ sudo apt-get install libncursesw5-dev
    $ sudo apt-get install libreadline5-dev
    $ sudo apt-get install libssl-dev
    $ sudo apt-get install libgdbm-dev
    $ sudo apt-get install libbz2-dev
    $ sudo apt-get install libc6-dev
    $ sudo apt-get install libsqlite3-dev
    $ sudo apt-get install tk-dev
  • Configure. I am installing to ~/lib/python2.6.
    $ cd Python-2.6
    $ ./configure --prefix=/home/sofeng/lib/python2.6
  • Make
    $ make
    Note I got the following message:
    Failed to find the necessary bits to build these modules:
    bsddb185           sunaudiodev   
    To find the necessary bits, look in setup.py in detect_modules() for the module's name.
    I was not able to find the Ubuntu packages for these.

  • Try it out (Optional)
    $ ./python
    Python 2.6 (r26:66714, Oct  2 2008, 15:32:46) 
    [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 
  • Test (Optional)
    $ make test
    I got the following status:
    327 tests OK.
    33 tests skipped:
        test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
        test_cd test_cl test_codecmaps_cn test_codecmaps_hk
        test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
        test_gl test_imgfile test_kqueue test_linuxaudiodev test_macos
        test_macostools test_normalization test_ossaudiodev test_pep277
        test_py3kwarn test_scriptpackages test_socketserver test_startfile
        test_sunaudiodev test_timeout test_urllib2net test_urllibnet
        test_winreg test_winsound test_zipfile64
    Those skips are all expected on linux2.
  • Install
    $ make install
  • Finally, I added ~/lib/python/bin/python2.6 to my PATH

Notes

The first time I ran, I got this message:

Failed to find the necessary bits to build these modules:
_hashlib           _ssl               bsddb185
bz2                gdbm               readline
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

I think I found Ubuntu packages for some of them:

  • _hashlib and _ssl in libssl-dev
  • bz2 in libbz2-dev
  • gdbm in libgdbm-dev
  • readline in libreadline5-dev

Installing Emacs 23 from CVSBazaar source on Ubuntu HardyKarmic

I have been using the emacs-snapshot package in Ubuntu Hardy. However, when I tried to use Tramp, I got an error message: Variable binding depth exceeds max-specpdl-size. I couldn't find out how to fix this, and I didn't want to use Emacs 22.3 because it doesn't have Xft (anti-aliased fonts), so I decided to live on the bleeding edge and install Emacs 23 from CVS. Besides the INSTALL and INSTALL.CVS files, I also used theBlackDragon's article for reference.

Update 2008-10-09: I just found that Romain Francoise maintains an emacs-snapshot Debian package and it has been adapted for Ubuntu. This is an alternative to installing from source.

Update 2010-01-09: Since Emacs has switched from CVS to Bazaar, I've updated these instructions to use Bazaar. Also, I'm now running on Ubuntu 9.10 Karmic Koala instead of Ubuntu Hardy.

Update 2012-09-19: Here are some additional Ubuntu 12.04 packages I needed to install Emacs 24.2.

$ sudo apt-get install libgif-dev
$ sudo apt-get install libtiff4-dev
$ sudo apt-get install xaw3dg-dev
$ sudo apt-get install librsvg2-dev
$ sudo apt-get install libmagick++-dev
$ sudo apt-get install libgpm-dev
$ sudo apt-get install libgconf2-dev
$ sudo apt-get install libselinux1-dev
$ sudo apt-get install libm17n-dev
$ sudo apt-get install libotf-dev 
  • Install Bazaar
    $ sudo apt-get install bzr
    
  • Get the source code from the Bazaar repository
    $ cd ~/incoming
    $ bzr branch http://bzr.savannah.gnu.org/r/emacs/trunk emacs_trunk
    
  • Read the INSTALL and INSTALL.BZR files in ~/incoming/emacs_trunk

  • Install prerequisites:
    $ sudo apt-get install build-essential
    $ sudo apt-get build-dep emacs23
    

    To see what is installed by build-dep, see the emacs23 karmic package page

  • Configure. The argument --prefix=/home/saltycrane/lib/emacs-bzr-20100210 means I am installing Emacs in /home/saltycrane/lib/emacs-bzr-20100210.
    $ cd ~/incoming/emacs_trunk
    $ ./configure --prefix=/home/saltycrane/lib/emacs-bzr-20100210
    
  • Per the INSTALL.BZR file, I needed to do a make bootstrap instead of make because some files, such as byte-compiled lisp files are not stored in Bazaar. Note, this takes a long time (over 10 min for me).
    $ make bootstrap
    
  • Make (Optional)
    $ make
    
  • Test it (Optional)
    $ src/emacs -q
    
  • Install
    $ make install
    
  • Create symlinks (~/bin is already on my PATH)
    $ ln -s ~/lib/emacs-bzr-20100210 ~/lib/emacs
    $ cd ~/bin
    $ ln -s ../lib/emacs/bin/* .
    

    Alternatively, I could add ~/lib/emacs-bzr-20100210/bin to my PATH.

Now I have "Pretty Emacs" with working Tramp for remote file access. It also has multi-tty support which is supposed to be very cool (but I haven't tried it yet) is very cool (e.g. for displaying my running desktop emacs process on my Android phone.)

Error messages
  • configure: error: You do not seem to have makeinfo >= 4.6, and your
    source tree does not seem to have pre-built manuals in the `info' directory.
    Either install a suitable version of makeinfo, or re-run configure
    with the `--without-makeinfo' option to build without the manuals.
    
    Solution:
    $ sudo apt-get install texinfo
    
  • configure: error: The following required libraries were not found:
         libjpeg libgif/libungif libtiff
    Maybe some development libraries/packages are missing?
    If you don't want to link with them give
         --with-jpeg=no --with-gif=no --with-tiff=no
    as options to configure
    
    Solution:
    $ sudo apt-get install libjpeg-dev libgif-dev libtiff4-dev
    
  • Warning: arch-dependent data dir (/home/saltycrane/lib/emacs-bzr/libexec/emacs/23.1.92/i686-pc-linux-gnu/) does not exist.
    Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/site-lisp' does not exist.
    Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/site-lisp' does not exist.
    Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/lisp' does not exist.
    Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/leim' does not exist.
    Warning: Could not find simple.el nor simple.elc
    

    This happened because I originally used ./configure --prefix=/home/saltycrane/lib/emacs-bzr and then renamed ~/lib/emacs-bzr to ~/lib/emacs-bzr-20100210. Solution: don't rename the directory.