SaltyCrane Blog

Do you have a lot of short, single-use, private functions in your Python code?

Date: 2013-04-12 | Modified: 2013-04-16 | Tags: python | 6 Comments

Do you have a lot of short, single-use, private functions in your Python code? For example, below is some stubbed out authentication code I've been working on. It checks if a user's password is correct and updates the hash algorithm from SHA-1 to bcrypt. The 4 private functions with the leading underscore are from 1 to 10 lines long and are only used by the check_password function. These functions are part of a larger module with about 20 functions. I don't like that these 4 functions add clutter to the module and are not grouped with the ...

Using Python's gzip and StringIO to compress data in memory

Date: 2012-11-07 | Modified: 2013-05-17 | Tags: python | 1 Comment

I needed to gzip some data in memory that would eventually end up saved to disk as a .gz file. I thought, That's easy, just use Python's built in gzip module.

However, I needed to pass the data to pycurl as a file-like object. I didn't want to write the data to disk and then read it again just to pass to pycurl. I thought, That's easy also-- just use Python's cStringIO module.

The solution did end up being simple, but figuring out the solution was a lot harder than I thought. Below is my ...

How to start a long-running process in screen and detach from it

Date: 2012-10-26 | Tags: linux | 2 Comments

How to start a long-running process in screen, detach from it, and reattach to it later.

Start a long running process in screen and detach¶

Ssh to the remote host, myremote:
```
eliot@mylocal:~$ ssh myremote 
```
Start a new screen session
```
eliot@myremote:~$ screen 
```
Start a long running process, "sleep 3600":
```
eliot@myremote:~$ sleep 3600 
```
Detach from the screen session:
```
eliot@myremote:~$ CTRL-A : detach ENTER 
```
(Hit [CTRL-A], then type a colon character, then type "detach", then hit [ENTER])
Exit your remote SSH session:
```
eliot@myremote:~$ exit 
```

Reattach to the existing screen session¶

Ssh to the remote host again:
```
eliot@mylocal:~$ ssh ...
```

How to use pip with crate.io

Date: 2012-10-24 | Tags: install_setup, python | 2 Comments

Here's how to use pip with crate.io (in case pypi.python.org goes down):

$ pip install --index-url=https://simple.crate.io yolk

Or with logging to see what's happening:

$ pip install --log=my-pip-debug.log --index-url=https://simple.crate.io yolk

How to run a Django local development server on a remote machine and access it in your browser on your local machine using SSH port forwarding

Date: 2012-10-23 | Modified: 2013-01-28 | Tags: django, linux, python, ssh | 1 Comment

Here is how to run a Django local development server on a remote machine and access it in your browser on your local machine using SSH port forwarding. (This is useful if there is a firewall blocking access to the port of your Django local dev server (port 8000).

On the local host, SSH to the remote host:

$ ssh -v -L 9000:localhost:8000 eliot@my.remotehost.com

On the remote host, run the Django dev server:

eliot@my.remotehost.com:/path/to/my/django/project$ python manage.py runserver 0.0.0.0:8000

On the local host, go ...

Testing HTTPS w/ Flask's development server using stunnel on Ubuntu

Date: 2012-10-17 | Tags: python, ubuntu, webdev | 2 Comments

Our website is served over HTTPS. To more easily test certain issues (e.g. mixed mode content warnings, or Mapquest SSL tile servers), I wanted to access my Flask local development server over HTTPS. These two articles describe how to do this using stunnel: Testing HTTPS with Django's Development Server, Django Development Server with HTTPS. Using stunnel, you can hit pages on your Django/Flask local dev server over HTTPS instead of HTTP. Here is how I installed it on Ubuntu Precise 12.04:

Install SSL development files
```
$ sudo apt-get install libssl-dev 
```
Go to https://www.stunnel.org/downloads ...

Setting up a Linux DVR w/ MythTV, Ubuntu 12.04, and a Hauppauge WinTV-HVR 1250 TV tuner card

Date: 2012-07-29 | Modified: 2012-11-08 | Tags: hardware, linux | 3 Comments

Setting up MythTV involves a little pain, but once it's set up, it's pretty great. And you don't have to spend lots of money on a DVR from the cable company. With my modest hardware specs, playback is smooth and clear, however Picture in Picture is too jittery to be useful. Here's what I did to get my MythTV DVR running on my Ubuntu machine.

Parameters¶

Ubuntu 12.04 LTS (Precise Pangolin)
MythTV 0.25
Hauppauge 1196 WinTV-HVR-1250 PCI-E x1 TV Tuner 1196 (The included remote does not work with Linux. However this Anker Mini Bluetooth ...

How to prevent nose (unittest) from using the docstring when verbosity >= 2

Date: 2012-07-06 | Tags: python

Some of our Python unit tests have docstrings. I find it annoying that, when using a verbosity level >= 2, nose prints the docstring instead of the class name and method name. Here's a hack to prevent it from doing that: Add a shortDescription() method to the test case class that returns None.

Here is an example of normal behavior:

import unittest

class MyTestCase(unittest.TestCase):
    def test_with_docstring(self):
        """Test that something does something
        """

    def test_without_docstring(self):
        pass

$ nosetests --verbosity=2 tmp.py
Test that something does something ... ok
test_without_docstring (tmp.MyTestCase) ... ok

Here is an example with the hack ...

Hack to share & sync Google contacts between Android phones

Date: 2012-06-17 | Tags: android | 1 Comment

I want to share and sync (in real time) Google (Gmail) contacts with my wife on our Android 2.3.6 Gingerbread phones. Google does not make this easy to do. Here's the best solution I could come up with (ref whitenack on androidcentral). (Note: these are not our real email addresses.)

This contact list resides only on the eliot@gmail.com account.
Contacts are removed from the "My Contacts" group and instead stored in groups called "Angela" and/or "Eliot". For shared contacts, the contact is in both groups. (Contact groups are like tags. A contact can be ...

Test coverage with nose and coverage.py

Date: 2012-04-07 | Modified: 2012-10-12 | Tags: python, testing, wide | 1 Comment

It's fun to use nose + coverage.py to show my progress as I write tests. Seeing the bar next to my code change from red to green makes me happy. 100% test coverage does not mean tests are complete. For example, a boolean OR'ed conditional expression may not test all conditions even though the line is marked as covered. Other limitations are discussed here: Flaws in coverage measurement. However, good test coverage is at least a step towards having a good test suite.