Posts tagged "decorators" « SaltyCrane Blog

Fabric post-run processing Python decorator

l Date: 2010-11-06 | Modified: 2011-12-01 | Tags: decorators, python, ssh

import traceback
from functools import wraps

from fabric.api import env


# global variable for add_hooks()
parent_task_name = ''


def add_post_run_hook(hook, *args, **kwargs):
    '''Run hook after Fabric tasks have completed on all hosts

    Example usage:
        @add_post_run_hook(postrunfunc, 'arg1', 'arg2')
        def mytask():
            # ...

    '''
    def true_decorator(f):
        return add_hooks(post=hook, post_args=args, post_kwargs=kwargs)(f)
    return true_decorator


def add_hooks(pre=None, pre_args=(), pre_kwargs={},
              post=None, post_args=(), post_kwargs={}):
    '''
    Function decorator to be used with Fabric tasks.  Adds pre-run
    and/or post-run hooks to a Fabric task.  Uses env.all_hosts to
    determine when to run the post hook.  Uses the global variable,
    parent_task_name, to check if ...

Using a Python timeout decorator for uploading to S3

l Date: 2010-04-27 | Tags: aws, decorators, python | 6 Comments

At work we are uploading many images to S3 using Python's boto library. However we are experiencing a RequestTimeTooSkewed error once every 100 uploads on average. We googled, but did not find a solution. Our system time was in sync and our file sizes were small (~50KB).

Since we couldn't find the root cause, we added a watchdog timer as a bandaid solution. We already use a retry decorator to retry uploads to S3 when we get a 500 Internal Server Error response. To this we added a timeout decorator which cancels the S3 upload if it takes ...

Two of the simplest Python decorator examples

l Date: 2010-03-09 | Modified: 2010-04-13 | Tags: decorators, python | 6 Comments

After trying for about the fifth time, I think I am starting to understand Python decorators due largely to Jack Diederich's PyCon 2009 talk, Class Decorators: Radically Simple.

Jack's practical definition of a decorator is:

A function that takes one argument
Returns something useful

In many simple cases, a function decorator can be described more specifically:

A function that takes one argument (the function being decorated)
Returns the same function or a function with a similar signature

As Jack states in his talk, a decorator is merely syntactic sugar. The same functionality can be achieved without using the ...

Trying out a Retry decorator in Python

l Date: 2009-11-17 | Modified: 2011-10-07 | Tags: decorators, python | 3 Comments

The Python wiki has a Retry decorator example which retries calling a failure-prone function using an exponential backoff algorithm. I modified it slightly to check for exceptions instead of a False return value to indicate failure. Each time the decorated function throws an exception, the decorator will wait a period of time and retry calling the function until the maximum number of tries is used up. If the decorated function fails on the last try, the exception will occur unhandled.

import time

def retry(ExceptionToCheck, tries=4, delay=3, backoff=2, logger=None):
    """Retry calling the decorated function using an ...

SaltyCrane Blog