After 7 years, I've yielded to the Dark Side and switched from Ubuntu to OS X on my work laptop. I've also switched from Python and back end web development to Javascript and front end development. The former is mostly to support the latter.

Linux is rare¹, especially among front end developers, and I want to make it easy to collaborate as I learn new things. I've had problems working with Photoshop files in GIMP and I couldn't run the iOS simulator. Issues with Linux device drivers don't help.

I'm choosing front end development because I want to code closer to the end user.² In small part like Ian Bicking wrote last year, I feel unexcited about back end development and really excited about Javascript and front end development. I'm excited about ES 2015 and React and React Native and CSS transitions.³ I'm even coming around to Node.js. Javascript is uglier than Python, but it's getting better and there are things Python can't do that Javascript can.⁴ If only beauty mattered, maybe I'd use Scheme.⁵

I'm sure I will hate OS X at first, but hopefully it will be good in the long run. If anyone can recommend a tiling window manager like Qtile for OS X, please let me know.

(I will continue using Emacs because Emacs rocks! ⁶)

I never learned how to use reduce in Python since our BDFL recommended against it. But since Javascript doesn't have list comprehensions, I'm learning some functional constructs in Underscore.js.

Underscore's reduce() function can be used to reduce an array of things into a single thing. A common use is computing the sum of an array of numbers. Like map and filter, it provides an alternative to a for loop. Reduce is not limited to returning a single thing and can be used to combine the functionality of map and filter like a Python list comprehension. Learning Underscore's reduce also helped me understand MapReduce and reducers in Redux.

Here is an example that uses reduce to create a single Javascript object from an array of objects. An empty object, {}, is passed in to _.reduce() as the initial state. It is then extended with each item in the array and finally returned. I also did the same example using _.each() for comparison, (update) and using Array.prototype.reduce and ES6 w/o Underscore.

Example using Underscore's reduce¶

var myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

var myObj = _.reduce( myArr, function( memo, item ) {
    return _.extend( memo, item ); }, {} );
console.log( myObj );

Here is the output:

{ rating: 5, price: 200, distance: 10 }

Example using Underscore's each¶

Here is the same example using _.each:

var myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

var myObj = {};
_.each( myArr, function( item ) {
    myObj = _.extend( myObj, item ); });
console.log( myObj );

The output is the same:

{ rating: 5, price: 200, distance: 10 }

Example using ES6 and Array.prototype.reduce¶

Update 2015-12-11: I have been writing ES6 with React thanks to Babel so here is an ES6 version without Underscore.js. It looks very similar to the Underscore version. reduce() is now a method of the array and Object.assign() takes the place of _.extend(). ( Array.prototype.reduce is actually ES5, but Object.assign and const are ES6. )

const myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

const myObj = myArr.reduce( function( memo, item ) {
    return Object.assign( {}, memo, item ); }, {} );
console.log( myObj );

The output is the same:

{ rating: 5, price: 200, distance: 10 }

The Elasticsearch Python library has a cool feature that allows you to log the equivalent curl command of the query you are making. To use it you just need to configure the elasticsearch.trace logger. There is one annoyance-- by design, "localhost:9200" is logged even if you have configured another host. It is more useful to log the actual host I am using so I can copy and paste the command to co-workers without changing the hostname. At first I thought I'd monkeypatch the library to do what I wanted. Then I remembered I could use a logging.Filter to rewrite the log message.

I am using the high level elasticsearch-dsl library in my example, but the logging is actually done by the low level official elasticsearch library which elasticsearch-dsl wraps.

$ pip install elasticsearch-dsl

import logging

import elasticsearch_dsl as esdsl  # high level elasticsearch library
from elasticsearch import Elasticsearch  # low level official elasticsearch library


ELASTICSEARCH_HOSTS = [
    'http://es1.myhost.com:9200',
    'http://es2.myhost.com:9200',
    'http://es3.myhost.com:9200',
]


class ReplaceHostFilter(logging.Filter):
    """
    Replace "localhost:9200" with real host
    """
    def __init__(self, real_host, *args, **kwargs):
        self.real_host = real_host
        super(ReplaceHostFilter, self).__init__(*args, **kwargs)

    def filter(self, record):
        record.msg = record.msg.replace('http://localhost:9200', self.real_host)
        record.msg = record.msg.replace('curl', 'curl -sS')
        record.msg += ' | python -mjson.tool'
        return True


# Set up logging
host_replacer = ReplaceHostFilter(ELASTICSEARCH_HOSTS[0])
formatter = logging.Formatter("%(levelname)s %(asctime)s %(name)s - %(message)s")
sh = logging.StreamHandler()
sh.setFormatter(formatter)
eslogger = logging.getLogger('elasticsearch.trace')
eslogger.setLevel('INFO')
eslogger.addFilter(host_replacer)
eslogger.addHandler(sh)


# Make a query
esclient = Elasticsearch(hosts=ELASTICSEARCH_HOSTS)
search = esdsl.Search(using=esclient, index='my_index')
filt = esdsl.F('term', myTerm=1234)
search = search.filter(filt)
response = search.execute()
print response.hits.total

Log output showing the actual host instead of localhost:

INFO 2015-07-28 13:48:54,464 elasticsearch.trace - curl -sS -XGET 'http://es1.myhost.com:9200/my_index/_search?pretty' -d '{
  "query": {
    "filtered": {
      "filter": {
        "term": {
          "myTerm": 1234
        }
      },
      "query": {
        "match_all": {}
      }
    }
  }
}' | python -mjson.tool

Yesterday I wrote a script to scrape my own web page because I screwed up the CSV export feature and Product needed the data. One problem was that the CloudFlare CDN obfuscated the email addresses on the page. My solutioncrazy hack: running a Node.js script to de-obfuscate the email from my Python scraping script.

Example obfuscated email stuff from CloudFlare:

<a href="/cdn-cgi/l/email-protection#d4b7b5a6b194a4b1a0e7e2e4fab7bbb9">
    <span class="__cf_email__" data-cfemail="0162607364417164753237312f626e6c">[email&#160;protected]</span>
    <script cf-hash='f9e31' type="text/javascript">
     /* <![CDATA[ */!function(){try{var t="currentScript"in document?document.currentScript:function(){for(var t=document.getElementsByTagName("script"),e=t.length;e--;)if(t[e].getAttribute("cf-hash"))return t[e]}();if(t&&t.previousSibling;){var e,r,n,i,c=t.previousSibling,a=c.getAttribute("data-cfemail");if(a){for(e="",r=parseInt(a.substr(0,2),16),n=2;a.length-n;n+=2)i=parseInt(a.substr(n,2),16)^r,e+=String.fromCharCode(i);e=document.createTextNode(e),c.parentNode.replaceChild(e,c)}}}catch(u){}}();/* ]]> */
    </script>
</a>

Using jsbeautifier.org, I adapted the Javascript from above into this Node.js script, decloudflare.js.

var e, r, n, i, a = process.argv[2];
for (e = "", r = parseInt(a.substr(0, 2), 16), n = 2; a.length - n; n += 2) i = parseInt(a.substr(n, 2), 16) ^ r, e += String.fromCharCode(i);
console.log(e);

Example usage:

$ node decloudflare.js 0162607364417164753237312f626e6c
care@pet360.com

I used the Naked library (thanks to Sweetmeat) to call the Node.js script. (Though probably I could've just used the subprocess module.)

$ pip install Naked 

from Naked.toolshed.shell import muterun_js

def decloudflare_email(cfemail):
    resp = muterun_js('decloudflare.js', cfemail)
    return resp.stdout.rstrip()

cfemail = '0162607364417164753237312f626e6c'
print 'cfemail from python: ' + cfemail
email = decloudflare_email(cfemail)
print 'email from python: ' + email

cfemail from python: 0162607364417164753237312f626e6c
email from python: care@pet360.com

Update 2015-12-02: I updated the post to use npm instead of Bower to install Bootstrap because it eliminates an extra tool and I hear this is the preferred method.

Update 2015-02-22: My co-workers informed me that Grunt is so last year and Gulp is the new hotness. You wish this post covered Gulp, but instead it covers Grunt.

I recently converted this blog to use Twitter's Bootstrap CSS framework to make it responsive. (In particular, I wanted to read it on my phone.) I was using Blueprint CSS from 2008 so it was about time for an update. Unfortunately my design is also from 2008 but I won't update that. (Web design is hard.)

My problem was that 2014 Bootstrap's 200px gutter width didn't match my 2008 design's 2px gutter. So I wanted to customize the gutter width.

There are a few ways to customize Bootstrap. One option is creating a custom version on Bootstrap's website and downloading it for use on your site. However this doesn't allow you to change things quickly as you develop. Another option is overriding Bootstrap's style in your site stylesheet. However, it's hacky to write compilcated selectors to override something in multiple places when it is set in a single variable in Bootstrap. For example, to change the gutter width, here is the single variable that needs to be changed.

Approach¶

Here is the approach I took to customize Bootstrap. I'm running Ubuntu 14.10 Utopic Unicorn 64-bit.

• Install the Sass version of Bootstrap using npm. The standard Bootstrap project uses Less but my limited knowledge of frontend technology tells me Sass (specifically SCSS) is a better choice. I previously used Bower to install Bootstrap, but I learned that npm can now be used instead of Bower in most cases.
• Override Bootstrap's Sass variables, remove unneeded components, and combine Bootstrap with my site's stylesheet using Sass @imports.
• Compile the Sass SCSS files to CSS using Grunt and grunt-contrib-sass.
• Commit the compiled Sass files to git. An alternative is to compile the files as part of the deploy process. At work, it is a pain keeping these files in git because there are many merge conflicts and differences in build tool versions and platforms between developers. For my blog, I am the only committer so I won't run into this.

Project directory structure¶

Here is what my project directory structure looks like:

my-project
├── Gruntfile.js
├── node_modules
│   ├── bootstrap-sass
│   ├── grunt
│   ├── grunt-contrib-sass
│   └── grunt-contrib-watch
├── package.json
├── sass
│   ├── _bootstrap-variables-override.scss
│   ├── _bootstrap.scss
│   └── mystyle.scss
└── static
    └── css
        └── mystyle.css

Install Sass¶

Sass is installed using RubyGems, the Ruby package manager.

$ sudo apt-get install ruby 
$ sudo gem install sass 
$ sass --version 
Sass 3.4.13 (Selective Steve)

Install grunt-cli¶

Grunt depends on npm, the Node.js package manager.

$ sudo add-apt-repository ppa:chris-lea/node.js 
$ sudo apt-get update 
$ sudo apt-get install nodejs 
$ npm --version
1.4.28

$ sudo npm install -g grunt-cli 
$ grunt --version 
grunt-cli v0.1.13

Install bootstrap-sass with npm¶

bootstrap-sass is the official Sass port of Bootstrap. I created this package.json file, /tmp/my-project/package.json. For more information about the package.json file, see the npm documentation.

{
  "name": "my-project",
  "version": "0.1.0",
  "dependencies": {
    "bootstrap-sass": "3.3.1"
  }
}

Then I ran npm install to download the files. The packages are stored in the node_modules directory.

$ cd /tmp/my-project 
$ npm install 

Customize Boostrap¶

To customize Boostrap, I copied the node_modules/bootstrap-sass/assets/stylesheets/boostrap.scss file to my sass directory, removed the components that I didn't need, and added a line for my custom variable overrides. Here is my final file, named /tmp/my-project/sass/_bootstrap.scss:

// Variable overrides come first (without !default). Bootstrap default variables
// come second because they use !default (they won't get set if set already) and
// some of them depend on our overrides.
@import "bootstrap-variables-override";

// Core variables and mixins
@import "bootstrap/variables";
@import "bootstrap/mixins";

// Reset and dependencies
@import "bootstrap/normalize";
@import "bootstrap/print";
@import "bootstrap/glyphicons";

// Core CSS
@import "bootstrap/scaffolding";
@import "bootstrap/type";
@import "bootstrap/code";
@import "bootstrap/grid";
@import "bootstrap/tables";
@import "bootstrap/forms";
@import "bootstrap/buttons";

// // I am not using these components so I commented them out to make the CSS file smaller
// // Components
// @import "bootstrap/component-animations";
// @import "bootstrap/dropdowns";
// @import "bootstrap/button-groups";
// @import "bootstrap/input-groups";
// @import "bootstrap/navs";
// @import "bootstrap/navbar";
// @import "bootstrap/breadcrumbs";
// @import "bootstrap/pagination";
// @import "bootstrap/pager";
// @import "bootstrap/labels";
// @import "bootstrap/badges";
// @import "bootstrap/jumbotron";
// @import "bootstrap/thumbnails";
// @import "bootstrap/alerts";
// @import "bootstrap/progress-bars";
// @import "bootstrap/media";
// @import "bootstrap/list-group";
// @import "bootstrap/panels";
// @import "bootstrap/responsive-embed";
// @import "bootstrap/wells";
// @import "bootstrap/close";

// // Components w/ JavaScript
// @import "bootstrap/modals";
// @import "bootstrap/tooltip";
// @import "bootstrap/popovers";
// @import "bootstrap/carousel";

// Utility classes
@import "bootstrap/utilities";
@import "bootstrap/responsive-utilities";

The Boostrap variables are located in node_modules/bootstrap-sass/assets/stylesheets/bootstrap/_variables.scss I copied the ones that I was interested in into a _bootstrap-variables-override.scss file. Be sure to remove the !default flag and import this file before the default bootstrap variables. For more information about !default, see the Sass documentation. Here is my /tmp/my-project/sass/_bootstrap-variables-override.scss file:

// overrides here (do not use !default here)
$grid-gutter-width: 10px;
$font-size-base: 16px;
$headings-font-weight: 800;

In my site stylesheet, I import _bootstrap.scss. Here is my /tmp/my-project/sass/mystyle.scss file:

@import "bootstrap";
/* add site style here */

Install grunt plugins¶

To compile the Sass files to CSS, I am using grunt-contrib-sass. I added a devDependencies section to my package.json file which contains grunt and two grunt plugins. Here is my /tmp/my-project/package.json. For more information about installing plugins, see the Grunt documentation.

{
  "name": "my-project",
  "version": "0.1.0",
  "dependencies": {
    "bootstrap-sass": "3.3.1"
  },
  "devDependencies": {
    "grunt": "0.4.5",
    "grunt-contrib-sass": "0.8.1",
    "grunt-contrib-watch": "0.6.1"
  }
}

I ran npm install again to install grunt, grunt-contrib-sass, and grunt-contrib-watch.

$ cd /tmp/my-project 
$ npm install 

Create Gruntfile.js¶

My Gruntfile.js uses the grunt-contrib-sass plugin to compile the Sass to CSS. My input file is sass/mystyle.scss and my output file is static/css/mystyle.css. I set the loadPath so that Sass can find the Bootstrap SCSS files in the node_modules directory to import. Alternatively, I could specify the full path to import in /tmp/my-project/sass/_bootstrap.scss. For example: @import "../node_modules/bootstrap-sass/assets/stylesheets/bootstrap/variables"; I am also using the grunt-contrib-watch plugin during development to automatically run the compilation whenever a SCSS file changes. For more information on configuring tasks, see the Grunt documentation. Here is my Gruntfile, /tmp/my-project/Gruntfile.js:

module.exports = function(grunt) {
    grunt.initConfig({
        sass: {
            // this is the "dev" Sass config used with "grunt watch" command
            dev: {
                options: {
                    style: 'expanded',
                    // tell Sass to look in the Bootstrap stylesheets directory when compiling
                    loadPath: 'node_modules/bootstrap-sass/assets/stylesheets'
                },
                files: {
                    // the first path is the output and the second is the input
                    'static/css/mystyle.css': 'sass/mystyle.scss'
                }
            },
            // this is the "production" Sass config used with the "grunt buildcss" command
            dist: {
                options: {
                    style: 'compressed',
                    loadPath: 'node_modules/bootstrap-sass/assets/stylesheets'
                },
                files: {
                    'static/css/mystyle.css': 'sass/mystyle.scss'
                }
            }
        },
        // configure the "grunt watch" task
        watch: {
            sass: {
                files: 'sass/*.scss',
                tasks: ['sass:dev']
            }
        }
    });
    grunt.loadNpmTasks('grunt-contrib-sass');
    grunt.loadNpmTasks('grunt-contrib-watch');
    // "grunt buildcss" is the same as running "grunt sass:dist".
    // if I had other tasks, I could add them to this array.
    grunt.registerTask('buildcss', ['sass:dist']);
};

Run grunt¶

To compile the Sass to CSS for production, I run grunt buildcss. This creates the output CSS file /tmp/my-project/static/css/mystyle.css.

$ cd /tmp/my-project 
$ grunt buildcss 

During development, I use grunt watch to build whenever a SCSS (Sass) file changes.

$ cd /tmp/my-project 
$ grunt watch 

Other Notes¶

• I don't commit the node_modules directory to git. Sass uses the Bootstrap files in node_modules to create the final CSS file.
• To configure Bootstrap's Javascript and glyphicons in your project, you can use grunt-contrib-concat and grunt-contrib-copy as spacepope nancho described in his comment.

See Also / References¶

• Getting started with Grunt and Sass - Ryan Christiani
• Frontend automation with Grunt, Sass + BrowserSync.
• Setting up Sass with Grunt | Culttt
• Lightning fast Sass compiling with libsass, Node-sass and Grunt-sass

I ordered a ~$1200 2.8 lb. 14" Lenovo Thinkpad X1 Carbon (2014, 2nd generation, 20A7) today at 40% off for the Black Friday / Cyber Monday sale. I hope I like the keyboard. I splurged for an Intel Core i7-4600U, 8GB RAM, 256GB SSD, and the WQHD 2560x1440 display. I have enjoyed 7 months with my ~$200 2.8 lb. 11" Acer C720 Chromebook. It's a pleasant contrast to my work-issued 5.9 lb. 17" Lenovo Thinkpad W510. Crouton worked well for running Linux on the Chromebook but I wanted a dedicated Linux laptop and a better screen. I will bequeath the C720 to my wife since she currently uses my 7+ year old Dell Inspiron E1405. I considered the 3.5 lb. 14" Lenovo Thinkpad T440s due to better battery/keyboard/flexibility, but the X1 Carbon was ~$200 cheaper for a similarly spec'd model due to a bigger discount and I liked the thinner and lighter design and better display. Many reviews complained about the new keyboard layout and adaptive function row. I hope key remapping will reduce the pain enough. LWN.net editor, Jonathan Corbet mentioned he bought an X1 Carbon in High-DPI displays and Linux, so it can't be too bad, right?

Here is the full 2014 2nd Gen ThinkPad X1 Carbon (Type 20A7, 20A8) spec sheet.

Update 2015-03-14: Lenovo released the 3rd generation of the X1 Carbon which fixes the keyboard and improves battery life. Too bad I didn't wait a few months. Here is the new 2015 3rd Gen spec sheet.

Grunt is a Javascript task runner that can be used to compile Sass, run JSHint, or run many other plugins. It depends on the Node.js package manager, npm.

If you use the standard Ubuntu Trusty Tahr repository to install nodejs/npm, you will get this error when running grunt:

/usr/bin/env: node: No such file or directory

Install nodejs, npm, and grunt-cli¶

$ sudo add-apt-repository ppa:chris-lea/node.js 
$ sudo apt-get update 
$ sudo apt-get install nodejs 
$ sudo npm install -g grunt-cli 

Install grunt in your project directory¶

$ cd ~/myproject 
$ echo "{}" > package.json 
$ npm install grunt --save-dev 

Verify grunt is installed¶

$ nodejs --version 
v0.10.33
$ npm --version 
1.4.28
$ grunt --version 
grunt-cli v0.1.13
grunt v0.4.5

Run a simple grunt task¶

1. $ cd ~/myproject
   

2. Create a package.json file:

   {
     "name": "my-project-name",
     "version": "0.1.0",
     "devDependencies": {
       "grunt": "~0.4.5",
       "grunt-contrib-uglify": "~0.5.0"
     }
   }
   

3. Install grunt-contrib-uglify

   $ npm install 
   npm WARN package.json my-project-name@0.1.0 No description
   npm WARN package.json my-project-name@0.1.0 No repository field.
   npm WARN package.json my-project-name@0.1.0 No README data
   grunt-contrib-uglify@0.5.1 node_modules/grunt-contrib-uglify
   ├── chalk@0.5.1 (ansi-styles@1.1.0, escape-string-regexp@1.0.2, supports-color@0.2.0, strip-ansi@0.3.0, has-ansi@0.1.0)
   ├── lodash@2.4.1
   ├── maxmin@0.2.2 (figures@1.3.5, pretty-bytes@0.1.2, gzip-size@0.2.0)
   └── uglify-js@2.4.15 (uglify-to-browserify@1.0.2, async@0.2.10, optimist@0.3.7, source-map@0.1.34)
   

4. Get an example unminified JS file:

   $ wget http://code.jquery.com/jquery-2.1.1.js 
   --2014-11-22 00:47:31--  http://code.jquery.com/jquery-2.1.1.js
   Resolving code.jquery.com (code.jquery.com)... 94.31.29.53, 94.31.29.230
   Connecting to code.jquery.com (code.jquery.com)|94.31.29.53|:80... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 247351 (242K) [application/x-javascript]
   Saving to: ‘jquery-2.1.1.js’
   
   100%[================================================================================================================>] 247,351     --.-K/s   in 0.1s    
   
   2014-11-22 00:47:31 (1.71 MB/s) - ‘jquery-2.1.1.js’ saved [247351/247351]
   

5. Create a Gruntfile.js file:

   module.exports = function(grunt) {
     grunt.initConfig({
       pkg: grunt.file.readJSON('package.json'),
       uglify: {
         build: {
           src: 'jquery-2.1.1.js',
           dest: 'jquery-2.1.1.min.js'
         }
       }
     });
     grunt.loadNpmTasks('grunt-contrib-uglify');
     grunt.registerTask('default', ['uglify']);
   };
   

6. Run the grunt task:

   $ grunt 
   Running "uglify:build" (uglify) task
   
   Done, without errors.
   

7. You should now have a minified file, jquery-2.1.1.min.js

   $ ls -gG jquery* 
   -rw-rw-r-- 1 247351 2014 10/23 17:16 jquery-2.1.1.js
   -rw-rw-r-- 1  84113 2014 11/22 00:48 jquery-2.1.1.min.js
   

References

• https://github.com/joyent/node/issues/3911
• http://gruntjs.com/getting-started

Here is some data that I want to group by model:

SOME_DATA = [
    {'model': u'Yaris', 'some_value': 11202, 'trim_name': u'3-Door L Manual'},
    {'model': u'Yaris', 'some_value': 19269, 'trim_name': u'3-Door LE Automatic'},
    {'model': u'Corolla', 'some_value': 27119, 'trim_name': u'L Automatic'},
    {'model': u'Corolla', 'some_value': 32262, 'trim_name': u'LE'},
    {'model': u'Corolla', 'some_value': 37976, 'trim_name': u'S Premium'},
    {'model': u'Camry', 'some_value': 39730, 'trim_name': u'LE 4-Cyl'},
    {'model': u'Camry', 'some_value': 45761, 'trim_name': u'XSE 4-Cyl'},
    {'model': u'Yaris', 'some_value': 48412, 'trim_name': u'3-Door L Automatic'},
    {'model': u'Camry', 'some_value': 55423, 'trim_name': u'XLE 4-Cyl'},
    {'model': u'Corolla', 'some_value': 57055, 'trim_name': u'ECO Premium'},
    {'model': u'Corolla', 'some_value': 61296, 'trim_name': u'ECO Plus'},
    {'model': u'Camry', 'some_value': 63660, 'trim_name': u'XSE V6'},
    {'model': u'Yaris', 'some_value': 65570, 'trim_name': u'5-Door LE Automatic'},
    {'model': u'Camry', 'some_value': 67461, 'trim_name': u'XLE V6'},
    {'model': u'Corolla', 'some_value': 73602, 'trim_name': u'S'},
    {'model': u'Yaris', 'some_value': 74158, 'trim_name': u'5-Door SE Manual'},
    {'model': u'Corolla', 'some_value': 74249, 'trim_name': u'LE Plus'},
    {'model': u'Corolla', 'some_value': 78386, 'trim_name': u'ECO'},
    {'model': u'Camry', 'some_value': 82747, 'trim_name': u'SE 4-Cyl'},
    {'model': u'Corolla', 'some_value': 83162, 'trim_name': u'LE Premium'},
    {'model': u'Corolla', 'some_value': 84863, 'trim_name': u'S Plus Manual'},
    {'model': u'Yaris', 'some_value': 90313, 'trim_name': u'5-Door L Automatic'},
    {'model': u'Corolla', 'some_value': 90452, 'trim_name': u'L Manual'},
    {'model': u'Yaris', 'some_value': 93152, 'trim_name': u'5-Door SE Automatic'},
    {'model': u'Corolla', 'some_value': 94973, 'trim_name': u'S Plus CVT'},
]

This can be done using defaultdict from the collections module.

import collections

grouped = collections.defaultdict(list)
for item in SOME_DATA:
    grouped[item['model']].append(item)

for model, group in grouped.items():
    print
    print model
    pprint(group, width=150)

Here are the results:

Yaris
[{'model': u'Yaris', 'some_value': 27065, 'trim_name': u'5-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 32757, 'trim_name': u'5-Door SE Automatic'},
 {'model': u'Yaris', 'some_value': 57344, 'trim_name': u'3-Door L Manual'},
 {'model': u'Yaris', 'some_value': 64002, 'trim_name': u'5-Door SE Manual'},
 {'model': u'Yaris', 'some_value': 77974, 'trim_name': u'3-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 92658, 'trim_name': u'3-Door LE Automatic'},
 {'model': u'Yaris', 'some_value': 98769, 'trim_name': u'5-Door LE Automatic'}]

Camry
[{'model': u'Camry', 'some_value': 30247, 'trim_name': u'XSE 4-Cyl'},
 {'model': u'Camry', 'some_value': 33809, 'trim_name': u'XSE V6'},
 {'model': u'Camry', 'some_value': 65637, 'trim_name': u'LE 4-Cyl'},
 {'model': u'Camry', 'some_value': 67329, 'trim_name': u'SE 4-Cyl'},
 {'model': u'Camry', 'some_value': 76269, 'trim_name': u'XLE 4-Cyl'},
 {'model': u'Camry', 'some_value': 87438, 'trim_name': u'XLE V6'}]

Corolla
[{'model': u'Corolla', 'some_value': 11239, 'trim_name': u'S'},
 {'model': u'Corolla', 'some_value': 27356, 'trim_name': u'S Plus Manual'},
 {'model': u'Corolla', 'some_value': 44792, 'trim_name': u'L Manual'},
 {'model': u'Corolla', 'some_value': 56252, 'trim_name': u'ECO Premium'},
 {'model': u'Corolla', 'some_value': 78570, 'trim_name': u'S Plus CVT'},
 {'model': u'Corolla', 'some_value': 78964, 'trim_name': u'LE Premium'},
 {'model': u'Corolla', 'some_value': 82116, 'trim_name': u'ECO'},
 {'model': u'Corolla', 'some_value': 85467, 'trim_name': u'S Premium'},
 {'model': u'Corolla', 'some_value': 87099, 'trim_name': u'L Automatic'},
 {'model': u'Corolla', 'some_value': 91974, 'trim_name': u'LE Plus'},
 {'model': u'Corolla', 'some_value': 94862, 'trim_name': u'LE'},
 {'model': u'Corolla', 'some_value': 97625, 'trim_name': u'ECO Plus'}]

This can also be done using itertools.groupby. This method is probably better when working with large datasets because groupby returns the group as an iterator. (This is the reason I convert it to a list before printing.)

import itertools

def keyfunc(x):
    return x['model']

SOME_DATA = sorted(SOME_DATA, key=keyfunc)
for model, group in itertools.groupby(SOME_DATA, keyfunc):
    print
    print model
    pprint(list(group), width=150)

Here are the results:

Camry
[{'model': u'Camry', 'some_value': 36776, 'trim_name': u'SE 4-Cyl'},
 {'model': u'Camry', 'some_value': 56569, 'trim_name': u'LE 4-Cyl'},
 {'model': u'Camry', 'some_value': 57052, 'trim_name': u'XSE 4-Cyl'},
 {'model': u'Camry', 'some_value': 92360, 'trim_name': u'XLE V6'},
 {'model': u'Camry', 'some_value': 92756, 'trim_name': u'XSE V6'},
 {'model': u'Camry', 'some_value': 94413, 'trim_name': u'XLE 4-Cyl'}]

Corolla
[{'model': u'Corolla', 'some_value': 13307, 'trim_name': u'L Automatic'},
 {'model': u'Corolla', 'some_value': 15726, 'trim_name': u'ECO Plus'},
 {'model': u'Corolla', 'some_value': 25579, 'trim_name': u'S'},
 {'model': u'Corolla', 'some_value': 31920, 'trim_name': u'ECO Premium'},
 {'model': u'Corolla', 'some_value': 34480, 'trim_name': u'LE'},
 {'model': u'Corolla', 'some_value': 44958, 'trim_name': u'S Plus Manual'},
 {'model': u'Corolla', 'some_value': 49606, 'trim_name': u'LE Premium'},
 {'model': u'Corolla', 'some_value': 59629, 'trim_name': u'LE Plus'},
 {'model': u'Corolla', 'some_value': 74226, 'trim_name': u'S Plus CVT'},
 {'model': u'Corolla', 'some_value': 75725, 'trim_name': u'L Manual'},
 {'model': u'Corolla', 'some_value': 82382, 'trim_name': u'ECO'},
 {'model': u'Corolla', 'some_value': 95633, 'trim_name': u'S Premium'}]

Yaris
[{'model': u'Yaris', 'some_value': 16789, 'trim_name': u'3-Door L Manual'},
 {'model': u'Yaris', 'some_value': 20349, 'trim_name': u'5-Door LE Automatic'},
 {'model': u'Yaris', 'some_value': 42897, 'trim_name': u'5-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 62045, 'trim_name': u'5-Door SE Automatic'},
 {'model': u'Yaris', 'some_value': 91913, 'trim_name': u'3-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 94218, 'trim_name': u'5-Door SE Manual'},
 {'model': u'Yaris', 'some_value': 97979, 'trim_name': u'3-Door LE Automatic'}]

This is an example that uses Javascript to display a session timeout warning modal 10 minutes before session expiration. It also resets the session expiration whenever the user clicks the mouse. It uses Javascript, jQuery, and Bootstrap on the frontend and Python, Flask, Flask-Login, and WTForms on the backend.

• Mouse clicks anywhere on the page ping the server at a maximum frequecy of once per minute and reset the session expiration.
• 10 minutes before the session expiration, a warning modal is displayed with two buttons: "Log out" and "Stay Logged In".
• If the user clicks "Stay Logged In" the session expiration is reset.
• If the user clicks "Log out", the user is logged out.
• If the user does nothing for 10 minutes, the user is logged out and displayed a message that the session timed out.

Here is the Javascript (session-monitor.js):

sessionMonitor = function(options) {
    "use strict";

    var defaults = {
            // Session lifetime (milliseconds)
            sessionLifetime: 60 * 60 * 1000,
            // Amount of time before session expiration when the warning is shown (milliseconds)
            timeBeforeWarning: 10 * 60 * 1000,
            // Minimum time between pings to the server (milliseconds)
            minPingInterval: 1 * 60 * 1000,
            // Space-separated list of events passed to $(document).on() that indicate a user is active
            activityEvents: 'mouseup',
            // URL to ping the server using HTTP POST to extend the session
            pingUrl: '/ping',
            // URL used to log out when the user clicks a "Log out" button
            logoutUrl: '/logout',
            // URL used to log out when the session times out
            timeoutUrl: '/logout?timeout=1',
            ping: function() {
                // Ping the server to extend the session expiration using a POST request.
                $.ajax({
                    type: 'POST',
                    url: self.pingUrl
                });
            },
            logout: function() {
                // Go to the logout page.
                window.location.href = self.logoutUrl;
            },
            onwarning: function() {
                // Below is example code to demonstrate basic functionality. Use this to warn
                // the user that the session will expire and allow the user to take action.
                // Override this method to customize the warning.
                var warningMinutes = Math.round(self.timeBeforeWarning / 60 / 1000),
                    $alert = $('<div id="jqsm-warning">Your session will expire in ' + warningMinutes + ' minutes. ' +
                               '<button id="jqsm-stay-logged-in">Stay Logged In</button>' +
                               '<button id="jqsm-log-out">Log Out</button>' +
                               '</div>');

                if (!$('body').children('div#jqsm-warning').length) {
                    $('body').prepend($alert);
                }
                $('div#jqsm-warning').show();
                $('button#jqsm-stay-logged-in').on('click', self.extendsess)
                    .on('click', function() { $alert.hide(); });
                $('button#jqsm-log-out').on('click', self.logout);
            },
            onbeforetimeout: function() {
                // By default this does nothing. Override this method to perform actions
                // (such as saving draft data) before the user is automatically logged out.
                // This may optionally return a jQuery Deferred object, in which case
                // ontimeout will be executed when the deferred is resolved or rejected.
            },
            ontimeout: function() {
                // Go to the timeout page.
                window.location.href = self.timeoutUrl;
            }
        },
        self = {},
        _warningTimeoutID,
        _expirationTimeoutID,
        // The time of the last ping to the server.
        _lastPingTime = 0;

    function extendsess() {
        // Extend the session expiration. Ping the server and reset the timers if
        // the minimum interval has passed since the last ping.
        var now = $.now(),
            timeSinceLastPing = now - _lastPingTime;

        if (timeSinceLastPing > self.minPingInterval) {
            _lastPingTime = now;
            _resetTimers();
            self.ping();
        }
    }

    function _resetTimers() {
        // Reset the session warning and session expiration timers.
        var warningTimeout = self.sessionLifetime - self.timeBeforeWarning;

        window.clearTimeout(_warningTimeoutID);
        window.clearTimeout(_expirationTimeoutID);
        _warningTimeoutID = window.setTimeout(self.onwarning, warningTimeout);
        _expirationTimeoutID = window.setTimeout(_onTimeout, self.sessionLifetime);
    }

    function _onTimeout() {
        // A wrapper that calls onbeforetimeout and ontimeout and supports asynchronous code.
        $.when(self.onbeforetimeout()).always(self.ontimeout);
    }

    // Add default variables and methods, user specified options, and non-overridable
    // public methods to the session monitor instance.
    $.extend(self, defaults, options, {
        extendsess: extendsess
    });
    // Set an event handler to extend the session upon user activity (e.g. mouseup).
    $(document).on(self.activityEvents, extendsess);
    // Start the timers and ping the server to ensure they are in sync with the backend session expiration.
    extendsess();

    return self;
};

Here is the important HTML:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8" />
        <title>Login</title>
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css" />
    </head>
    <body>
        <div id="session-warning-modal" class="modal">
            <div class="modal-dialog">
                <div class="modal-content">
                    <div class="modal-header">
                        <h4 class="modal-title" id="sessWarnLabel">Your session is about to expire<a class="sectionlink" title="Section permalink" href="#sessWarnLabel">¶</a></h4>
                    </div>
                    <div class="modal-body">
                        Your session will expire in <span id="remaining-time"></span> minutes due to inactivity.
                    </div>
                    <div class="modal-footer">
                        <button id="log-out" class="btn btn-default" type="button" data-dismiss="modal">Log Out</button>
                        <button id="stay-logged-in" class="btn btn-warning" type="button" data-dismiss="modal">Stay Logged In</button>
                    </div>
                </div>
            </div>
        </div>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
        <script src="//maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
        <script src="{{ url_for('static', filename='session-monitor.js')}}"></script>
        <script type="text/javascript">
            // Configure and start the session timeout monitor
            sessMon = sessionMonitor({
                // Subtract 1 minute to ensure the backend doesn't expire the session first
                sessionLifetime: {{ PERMANENT_SESSION_LIFETIME_MS }} - (1 * 60 * 1000),
                timeBeforeWarning: 10 * 60 * 1000,  // 10 minutes
                minPingInterval: 1 * 60 * 1000,  // 1 minute
                pingUrl: '/ping',
                logoutUrl: '/logout',
                timeoutUrl: '/logged-out?timeout=1&next=' + encodeURIComponent(
                    window.location.pathname + window.location.search + window.location.hash),
                // The "mouseup" event was used instead of "click" because some of the
                // inner elements on some pages have click event handlers that stop propagation.
                activityEvents: 'mouseup',
                onwarning: function() {
                    $("#session-warning-modal").modal("show");
                }
            });
            $(document).ready( function() {
                // Configure the session timeout warning modal
                $("#session-warning-modal")
                    .modal({
                        "backdrop": "static",
                        "keyboard": false,
                        "show": false
                    })
                    .on("click", "#stay-logged-in", sessMon.extendsess)
                    .on("click", "#log-out", sessMon.logout)
                    .find("#remaining-time").text(Math.round(sessMon.timeBeforeWarning / 60 / 1000));
            });
            window.sessMon = sessMon;
        </script>
    </body>
</html>

Here is the Python Flask app (myapp.py):

import collections
import datetime

from flask import Flask, request, render_template, redirect, url_for, session
from flask.ext.login import (
    LoginManager, login_user, logout_user,  UserMixin, login_required)
from wtforms.fields import PasswordField, StringField
from wtforms.form import Form


UserRow = collections.namedtuple('UserRow', ['id', 'password'])
TOY_USER_DATABASE = {
    'george': UserRow(id=1, password='george'),
}


# settings ###############################################################
# Set a secret key to sign the session (Flask config value)
SECRET_KEY = 'insert secret key here'

# The amount of time after which the user's session expires
# (this is a Flask setting and is also used by the Javascript)
PERMANENT_SESSION_LIFETIME = datetime.timedelta(minutes=60)


# init ###############################################################
app = Flask(__name__)
app.config.from_object(__name__)
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.login_view = '.login'


@login_manager.user_loader
def load_user(userid):
    return User(userid)


@app.context_processor
def add_session_config():
    """Add current_app.permanent_session_lifetime converted to milliseconds
    to context. The config variable PERMANENT_SESSION_LIFETIME is not
    used because it could be either a timedelta object or an integer
    representing seconds.
    """
    return {
        'PERMANENT_SESSION_LIFETIME_MS': (
            app.permanent_session_lifetime.seconds * 1000),
    }


# models ###############################################################
class User(UserMixin):
    def __init__(self, id):
        self.id = id


# forms ###############################################################
class LoginForm(Form):
    username = StringField()
    password = PasswordField()


# views ###############################################################
@app.route("/login", methods=['GET', 'POST'])
def login():
    form = LoginForm(request.form)
    message = ''

    if request.method == 'POST' and form.validate():
        db_user = TOY_USER_DATABASE.get(form.username.data)
        if form.password.data == db_user.password:
            user = User(db_user.id)
            login_user(user)
            return redirect(url_for('.home'))
        else:
            message = 'Login failed.'

    context = {
        'form': form,
        'message': message,
    }
    return render_template('login.html', **context)


@app.route("/")
@login_required
def home():
    return render_template('home.html')


@app.route("/another-page")
@login_required
def another_page():
    return render_template('another_page.html')


@app.route("/logout")
@login_required
def logout():
    logout_user()
    return redirect(url_for('.logged_out') + '?' + request.query_string)


@app.route("/logged-out")
def logged_out():
    timed_out = request.args.get('timeout')
    return render_template('logged_out.html', timed_out=timed_out)


@app.route("/ping", methods=['POST'])
def ping():
    session.modified = True
    return 'OK'


if __name__ == "__main__":
    app.run(debug=True)

The full example is also on github at: https://github.com/saltycrane/session-timeout-example.

Initial ideas were taken from http://www.itworld.com/development/335546/how-create-session-timeout-warning-your-web-application-using-jquery but, from what I could tell, it pinged the server whether the user was active or not.

Today I learned that the old style "%" string formatting and the new string .format() method behave differently when interpolating unicode strings. I was suprised to find out that one of these lines raised an error while one did not:

'%s' % u'O\u2019Connor'

'{}'.format(u'O\u2019Connor')

The old style "%" formatting operation returns a unicode string if one of the values is a unicode string even when the format string is a non-unicode string:

Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> '%s' % u'O\u2019Connor'
u'O\u2019Connor'

The new string .format() method called on a non-unicode string with a unicode string argument tries to encode the unicode string to a non-unicode string (bytestring) possibly raising a UnicodeEncodeError:

Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> '{}'.format(u'O\u2019Connor')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 1: ordinal not in range(128)</module></stdin>

I guess the correct thing to do is to start with a unicode format string:

Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> u'{}'.format(u'O\u2019Connor')
u'O\u2019Connor'

See also

• Documentation on the old style "%" string formatting
• Documentation on the new str.format method and the new format string syntax
• My old blog post on UnicodeEncodeError