Saltycrane logo

SaltyCrane Blog

Notes on Python, Django, and web development on Ubuntu Linux

    

Quick notes on trying the Twisted websocket branch example

Date: 2010-05-24  |   Modified: 2010-05-25  |   Tags: python, twisted   |   4 Comments

Here are my quick notes on trying out the websocket example in Twisted's websocket branch. The documentation is here. The Twisted ticket is here. This came about after some conversation with @clemesha on Twitter.

(A Web Socket is a new, still-in-development technology, introduced in HTML5, and may be used for real-time web applications. It provides a simple (maybe better) alternative to existing Comet technology.)

(Note: The WebSocket API is still changing. Google Chrome supports (a version of) it. Firefox as of version 3.6, does not support it yet.)

(I am no expert on Web Sockets. I just think ...

... read more »

Using a Python timeout decorator for uploading to S3

Date: 2010-04-27  |   Tags: aws, python, webservices   |   3 Comments

At work we are uploading many images to S3 using Python's boto library. However we are experiencing a RequestTimeTooSkewed error once every 100 uploads on average. We googled, but did not find a solution. Our system time was in sync and our file sizes were small (~50KB).

Since we couldn't find the root cause, we added a watchdog timer as a bandaid solution. We already use a retry decorator to retry uploads to S3 when we get a 500 Internal Server Error response. To this we added a timeout decorator which cancels the S3 upload if it takes ...

... read more »

Options for listing the contents of a directory with Python

Date: 2010-04-19  |   Modified: 2010-05-04  |   Tags: files_directories, python   |   3 Comments

I do a lot of sysadmin-type work with Python so I often need to list the contents of directory on a filesystem. Here are 4 methods I've used so far to do that. Let me know if you have any good alternatives. The examples were run on my Ubuntu Karmic machine.

OPTION 1 - os.listdir()

This is probably the simplest way to list the contents of a directory in Python.

import os
dirlist = os.listdir("/usr")

from pprint import pprint
pprint(dirlist)

Results:

['lib',
 'shareFeisty',
 'src',
 'bin',
 'local',
 'X11R6',
 'lib64',
 'sbin',
 'share',
 'include',
 'lib32',
 'man',
 'games']

OPTION 2 - glob ...

... read more »

Notes on using Gearman with Python

Date: 2010-04-17  |   Modified: 2010-05-08  |   Tags: concurrency, python   |   2 Comments

We recently looked at some lightweight message queue options for a S3 uploader tool. One of the options we tried was Gearman. Gearman was originally developed by Brad Fitzpatrick (author of memcached). Gearman seems to be mature and actively developed. The original Perl version has been rewritten in C for improved performance and I found it easy to use. Here are my notes for getting started on Ubuntu Karmic.

See also

... read more »

Monitoring a filesystem with Python and Pyinotify

Date: 2010-04-09  |   Tags: files_directories, linux, python, ubuntu

Pyinotify is a Python library for monitoring filesystem events on Linux through the inotify Linux kernel subsystem. It can monitor when a file is created, accessed, deleted, modified, etc. For a full list of Pyinotify events see the documentation.

Install Pyinotify

  • Install pip
  • Install Pyinotify 
    $ sudo pip install pyinotify

Example

import pyinotify

class MyEventHandler(pyinotify.ProcessEvent):
    def process_IN_ACCESS(self, event):
        print "ACCESS event:", event.pathname

    def process_IN_ATTRIB(self, event):
        print "ATTRIB event:", event.pathname

    def process_IN_CLOSE_NOWRITE(self, event):
        print "CLOSE_NOWRITE event:", event.pathname

    def process_IN_CLOSE_WRITE(self, event):
        print "CLOSE_WRITE event:", event.pathname

    def process_IN_CREATE(self, event):
        print "CREATE event:", event ...
... read more »
« Older Posts
Created with Django | Hosted by Slicehost