Posts tagged "webdev" « SaltyCrane Blog

Setting the Expires header for S3 media using Python and boto

l Date: 2012-02-11 | Tags: aws, notestoself, python, webdev

Install boto¶

$ pip install boto 
$ pip freeze |grep boto 
boto==2.2.1

Script¶

This script sets the "Expires" header 25 years from the current date for all the files starting with the prefix "mydirectory". Replace the access key id, secret access key, and bucket.

import mimetypes
from datetime import datetime, timedelta

from boto.s3.connection import S3Connection


AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
BUCKET_NAME = 'mybucket'
PREFIX = 'mydirectory'


def main():
    conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
    bucket = conn.get_bucket(BUCKET_NAME)
    key_list = bucket.get_all_keys(prefix=PREFIX)
    for key in key_list:
        content_type, unused = mimetypes.guess_type(key.name)
        if not content_type:
            content_type = 'text/plain'
        expires = datetime.utcnow ...

How to list attributes of an EC2 instance with Python and boto

l Date: 2010-03-08 | Modified: 2010-04-13 | Tags: aws, python, webdev | 7 Comments

Here's how to find out information about your Amazon EC2 instances using the Python boto library.

Install boto

Install pip
Install boto
```
sudo pip install boto
```

Example

from pprint import pprint
from boto import ec2

AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

ec2conn = ec2.connection.EC2Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
reservations = ec2conn.get_all_instances()
instances = [i for r in reservations for i in r.instances]
for i in instances:
    pprint(i.__dict__)
    break # remove this to list all instances

Results:

{'_in_monitoring_element': False,
 'ami_launch_index': u'0',
 'architecture': u'x86_64',
 'block_device_mapping': {},
 'connection': EC2Connection:ec2.amazonaws.com,
 'dns_name': u'ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com',
 'id': u'i-xxxxxxxx',
 'image_id': u ...

How to escape (percent-encode) a URL with Python

l Date: 2008-10-29 | Tags: python, webdev | 3 Comments

import urllib

print urllib.quote_plus("http://www.yahoo.com/")
print urllib.quote_plus("Kruder & Dorfmeister")

Results:

http%3A%2F%2Fwww.yahoo.com%2F
Kruder+%26+Dorfmeister

It is easy to be drawn to the urlencode function in the Python urllib module documentation. But for simple escaping, only quote_plus, or possibly quote is needed. I believe this is the appropriate solution to Python urlencode annoyance and O'Reilly's Amazon Hack #92.

For reference: Percent-encoding on Wikipedia

Python urlparse example

l Date: 2008-09-24 | Tags: python, webdev

Here is an example of how to parse a URL using Python's urlparse module. See the urlparse module documentation for more information.

from urlparse import urlparse

url = 'http://www.gurlge.com:80/path/file.html;params?a=1#fragment'
o = urlparse(url)
print o.scheme
print o.netloc
print o.hostname
print o.port
print o.path
print o.params
print o.query
print o.fragment
print o.username
print o.password

Results:

http
www.gurlge.com:80
www.gurlge.com
80
/path/file.html
params
a=1
fragment
None
None

On using Python, the Digg API, and simplejson

l Date: 2008-08-19 | Modified: 2009-03-25 | Tags: python, webdev | 1 Comment

Here are some quick notes on using the Digg API with a Python script. Note, there is a Python toolkit for Digg but I just used urllib2 and the Digg API endpoints for the sake of simplicity.

I wanted the output in JSON format so I specified the response type as JSON. To decode JSON directly to a Python data structure, I used simplejson.

Here is a simple example which returns the JSON output for the Digg story Dell vs. Apple: This Time it's Personal which has a "clean title" of Dell_vs_Apple_This_Time_it_s_Personal.

#!/usr/bin/env python

import urllib2

APPKEY ...

How to use gnip-python to retrieve activity from Twitter, Delicious, Digg, etc.

l Date: 2008-08-01 | Modified: 2009-03-25 | Tags: python, webdev

Create an account at http://www.gnipcentral.com/
Download gnip-python from github.com.

Unpack it:

$ tar -zxvf gnip-gnip-python-028364a70bd40dda0069ecdd3e7f6fff23bb985e.tar.gz

Move it to your example directory:

$ mkdir ~/src/python/gnip-example
$ mv gnip-gnip-python-028364a70bd40dda0069ecdd3e7f6fff23bb985e/*.py ~/src/python/gnip-example

Create an example file called ~/src/python/gnip-example/gnip-example.py:

#!/usr/bin/env python

from gnip import *

gnip = Gnip("yourgniplogin@email.com", "yourpassword")

for publisher in ["twitter", "digg", "delicious"]:
    activities = gnip.get_publisher_activities(publisher)
    print
    print publisher
    for activity in activities[:5]:
        print activity

Run it:

$ python gnip-example.py

And get the following results:

twitter
[derricklo, 2008-08-01T22:49:59+00:00, tweet, http://twitter.com ...

SaltyCrane Blog