SaltyCrane Blog — Notes on PythonJavascript and web development

Switching from Emacs to Vim (actually Spacemacs)

I recently switched from Linux to OS X and Python to Javascript. To complete my fall from the Light, I've switched from Emacs to Vim. Actually I just switched to Evil Mode and Spacemacs. This is how I came to switch:

  • I discovered OS X uses many Emacs key bindings by default and I could set even more.
  • I went back to the default key bindings in Emacs to be consistent with OS X.1
  • I remapped Return to Control and started using both Control keys to help use the default Emacs key bindings. Using both Control keys felt amazing compared to just one...
  • ...until I began feeling Emacs Pinky since Return was slightly farther than Caps Lock.2
  • I tried remapping Spacebar to Control and this felt even more amazing...
  • ...until I tried to type a sentence at normal speed.
  • I decided I didn't want to buy a foot pedal.
  • I tried Spacemacs.
  • I set bash and Karabiner to Vim mode.3
  • I set Caps Lock to Escape and Control.
  • I started looking for Vim screencasts.4

Even after 3 months, I'm still working a lot slower, but I'm hoping it's a good investment. One thing I've noticed is that Vim seems to use a lot of number and symbol keys. I need to learn to touch type my numbers!

Spacemacs

spacemacs screenshot

Spacemacs is an Emacs starter kit5 (like Emacs Prelude) optimized for Vim key bindings. It provides the "best of both worlds" – the efficiency of Vim's modal editing and the extensibility of Emacs Lisp.

Spacemacs replaces many Emacs modifier combinations by setting a leader key to the Spacebar (hence the name spacemacs). To open a file, use SPC f f instead of C-x C-f. Spacemacs makes commands easily discoverable using which-key. Just press SPC to see a list of commands, press a key and see more commands.

Spacemacs has a good out-of-the-box configuration for Javascript and React development. It uses js2-mode, web-mode6 for JSX, flycheck w/ eslint, tern, and some things I haven't used.

Install Spacemacs

Here's how to install Spacemacs on OS X.

  • Install Emacs
    $ brew install emacs --cocoa 
    
  • Install Spacemacs
    $ mv ~/.emacs.d ~/.emacs.d.bak  # if you have an exisiting .emacs.d directory
    $ git clone https://github.com/syl20bnr/spacemacs ~/.emacs.d 
    
  • Start Emacs (in terminal mode). This will download and compile packages and ask if you want to use vim mode or emacs mode.
    $ emacs 
    

Start Emacs in client/server mode

  • Start the Emacs server
    $ emacs --daemon 
    
  • Start an Emacs client in the terminal
    $ emacsclient -t 
    
  • Start a graphical Emacs client
    $ emacsclient -c 
    

Spacemacs config

Spacemacs has its own configuration file located at ~/.spacemacs or ~/.spacemacs.d/init.el. For more information, see the configuration documentation. My personal Spacemacs configuration is on github.

Useful Spacemacs commands

SPC q q - quit
SPC w / - split window vertically
SPC w - - split window horizontally
SPC 1   - switch to window 1
SPC 2   - switch to window 2
SPC w c - delete current window
SPC TAB - switch to previous buffer
SPC b b - switch buffers
SPC f f - find a file
SPC f s - save a file (:w also works)
SPC p p - open project
SPC p h - find a file in current project
SPC b d - kill current buffer
SPC b M - move buffer to another window
SPC v   - enter expand-region mode

Useful Vim key bindings

movement
--------
0 - beginning of line
^ - beginning of non-whitespace
$ - end of line
9j - move down 9 lines
w - move forward by word
b - move backward by word
gg - first line
G - last line
C-u - up half page
C-d - down half page
f/ - move forward to first "/" character
t/ = move forward right before the first "/" character
; - repeat that command again
H - head of the screen
M - middle of the screen
L - last of the screen
} - move forward by paragraph or block
{ - move backwards by paragraph or block
* - search for word under the cursor
    n - search again forward
    N - search again backwards
# - search backwards for word under cursor
/ - search forward
? - search backward
% - find matching brace, paren, etc
ma - mark a line in a file with marker "a"
`a - after moving around, go back to the exact position of marker "a"
'a - after moving around, go back to line of marker "a"
:marks - view all the marks
'' - go to the last place you were
[{ - jump back to the "{" at the beginning of the current code block

editing
-------
x - delete char under cursor
X - delete char before cursor
A - add to end of line
I - insert at the beginning of the line
dd - delete line
D - delete from cursor to end of line
di' - delete text inside single quotes
yy - copy line
Y - copy from cursor to end of line
cc - change line
C - change from cursor to end of line
cit - change text inside html tag
ci' - change text inside single quotes
ci{ - change text inside curly brackets.
ci... - etc
p - paste after cursor
P = paste before cursor
o - add line below
O - add line above
. = repeat last comment
r - replace character
R - replace. (overwrite) (good for columns of text)
J - join line (cursor can be anywhere on line)

visual mode
-----------
v - visual char mode
V - visual line mode
C-v - block visual mode

  1. And to be consistent with other machines or programs using Emacs or Emacs key bindings. Mandatory Eclipse for a 4 day Hadoop training emphasized this need. [back]
  2. At this point, I probably could have learned to move my hand 2 cm to the right, but I have been curious about Vim for a while now. [back]
  3. Later I also installed Vimium. [back]
  4. I found some excellent Vim screencasts here: http://derekwyatt.org/vim/tutorials/. [back]
  5. Though some don't call it a starter kit. [back]
  6. Hat tip to web-mode's author/maintainer. I created an issue about JSX indentation and it was fixed in less than a day. [back]

Modules and import in ES6 for Python developers

Here's a comparison of Python and Javascript (ES6) imports. There are a few differences between the two:

  1. Javascript imports are static; Python's are dynamic.
  2. Javascript items must be explicitly exported. In Python, all items are available for import.
  3. Javascript has a concept of a default export. Python does not.

PythonES6 (ES 2015)
 
import mymodule
mymodule.myfunc()
mymodule.py:
def myfunc(): pass

import mymodule as myalias
myalias.myfunc()
mymodule.py:
def myfunc(): pass
Namespaced imports
import * as myalias from "./mymodule";
myalias.myfunc();
mymodule.js:
export function myfunc() {}
Note: this form covers both Python's import mymodule and import mymodule as myalias forms.
 
from mymodule import myvar, myfunc
print myvar
myfunc()
mymodule.py:
myvar = 42
def myfunc(): pass
Named imports
import { myvar, myfunc } from "./mymodule";
console.log(myvar);
myfunc();
mymodule.js:
export var myvar = 42;
// no semicolon for inline exports
// of functions and classes
export function myfunc() {}
Note: curly braces are required even if only importing a single item. This is not destructuring. It is syntax specific to modules. Destructuring on import is not supported in ES6.
  No equivalentDefault imports (preferred form)
import myalias from "./mymodule";
myalias();
mymodule.js:
export default function myfunc() {}

Note: this import syntax has no curly braces because export default is used instead of just export in mymodule.js. There can be only one default export per module. Using this syntax is the preferred form. Unlike the form with curly braces, you will always supply your own name for the imported item. It may or may not be the same as the original name of the exported item. You may also combine the default import syntax with the non-default syntax.

import mydefault, { myother } from "./mymodule";
mydefault();
myother();
mymodule.js:
export function myother() {}
export default function myfunc() {}
 
from mymodule import myfunc as myalias
myalias()
mymodule.py:
def myfunc(): pass
Renaming an import
import { myfunc as myalias } from "./mymodule";
myalias();
mymodule.js:
export function myfunc() {}
import * from mymodule
print myvar
myfunc()
mymodule.py:
myvar = 42
def myfunc(): pass
Note: this form is not recommended
No equivalent
 
from mydir.mymodule import myfunc
myfunc()
mydir/mymodule.py:
def myfunc(): pass
Note: mydir contains a __init__.py file and is a module (package)
Importing from a subdirectory
import { myfunc } from "mydir/mymodule";
myfunc();
mydir/mymodule.js:
export function myfunc() {}

Names vs. paths

Modules can be referenced by name or by path. Names are often used with external libraries. For example, below "react" is the name.

import React from "react";

Paths are often used with your project modules. Here the module is referenced by the path "./MyComponent". (The .js extension is implicit.)

import MyComponent from "./MyComponent";

When are curly braces needed?

Curly braces are needed when importing non-default exports from a module. If the item is exported with default, use the import syntax without curly braces.

// mymodule.js
export default Something;
import Something from "mymodule";

If the item is exported without default, you must import the item with curly braces.

// mymodule.js
export Something;
import { Something } from "mymodule";

References

You Don't Know JS: ES6 & Beyond

Exploring ES6

Switching to OS X and front end development

After 7 years, I've yielded to the Dark Side and switched from Ubuntu to OS X on my work laptop. I've also switched from Python and back end web development to Javascript and front end development. The former is mostly to support the latter.

Linux is rare1, especially among front end developers, and I want to make it easy to collaborate as I learn new things. I've had problems working with Photoshop files in GIMP and I couldn't run the iOS simulator. Issues with Linux device drivers don't help.

I'm choosing front end development because I want to code closer to the end user.2 In small part like Ian Bicking wrote last year, I feel unexcited about back end development and really excited about Javascript and front end development. I'm excited about ES 2015 and React and React Native and CSS transitions.3 I'm even coming around to Node.js. Javascript is uglier than Python, but it's getting better and there are things Python can't do that Javascript can.4 If only beauty mattered, maybe I'd use Scheme.5

I'm sure I will hate OS X at first, but hopefully it will be good in the long run. If anyone can recommend a tiling window manager like Qtile for OS X, please let me know.

(I will continue using Emacs because Emacs rocks! 6)


  1. I think I was the last person at my company running Linux.
  2. I've been trying to do front end work for years now, but I finally got a sensible chance to switch as my company is changing it's technology stack from Python to Ruby and Backbone/Angular to React.
  3. Update 2016-01-04: Here are even more exciting web technologies: Electron, progressive web apps, and WebAssembly.
  4. Update 2016-01-01: I found James Hague had similar thoughts on Python and Javascript.
  5. Speaking of functional languages and Javascript, Elm sounds pretty cool.
  6. Update 2016-01-26: Or will I?

An example using reduce() in Underscore.js

I never learned how to use reduce in Python since our BDFL recommended against it. But since Javascript doesn't have list comprehensions, I'm learning some functional constructs in Underscore.js.

Underscore's reduce() function can be used to reduce an array of things into a single thing. A common use is computing the sum of an array of numbers. Like map and filter, it provides an alternative to a for loop. Reduce is not limited to returning a single thing and can be used to combine the functionality of map and filter like a Python list comprehension. Learning Underscore's reduce also helped me understand MapReduce and reducers in Redux.

Here is an example that uses reduce to create a single Javascript object from an array of objects. An empty object, {}, is passed in to _.reduce() as the initial state. It is then extended with each item in the array and finally returned. I also did the same example using _.each() for comparison, (update) and using Array.prototype.reduce and ES6 w/o Underscore.

Example using Underscore's reduce

var myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

var myObj = _.reduce( myArr, function( memo, item ) {
    return _.extend( memo, item ); }, {} );
console.log( myObj );

Here is the output:

{ rating: 5, price: 200, distance: 10 }

Example using Underscore's each

Here is the same example using _.each:

var myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

var myObj = {};
_.each( myArr, function( item ) {
    myObj = _.extend( myObj, item ); });
console.log( myObj );

The output is the same:

{ rating: 5, price: 200, distance: 10 }

Example using ES6 and Array.prototype.reduce

Update 2015-12-11: I have been writing ES6 with React thanks to Babel so here is an ES6 version without Underscore.js. It looks very similar to the Underscore version. reduce() is now a method of the array and Object.assign() takes the place of _.extend(). ( Array.prototype.reduce is actually ES5, but Object.assign and const are ES6. )

const myArr = [
    { rating: 5 },
    { price: 200 },
    { distance: 10 }
];

const myObj = myArr.reduce( function( memo, item ) {
    return Object.assign( {}, memo, item ); }, {} );
console.log( myObj );

The output is the same:

{ rating: 5, price: 200, distance: 10 }

Logging the actual host (instead of localhost) with the Python elasticsearch client

The Elasticsearch Python library has a cool feature that allows you to log the equivalent curl command of the query you are making. To use it you just need to configure the elasticsearch.trace logger. There is one annoyance-- by design, "localhost:9200" is logged even if you have configured another host. It is more useful to log the actual host I am using so I can copy and paste the command to co-workers without changing the hostname. At first I thought I'd monkeypatch the library to do what I wanted. Then I remembered I could use a logging.Filter to rewrite the log message.

I am using the high level elasticsearch-dsl library in my example, but the logging is actually done by the low level official elasticsearch library which elasticsearch-dsl wraps.

$ pip install elasticsearch-dsl
import logging

import elasticsearch_dsl as esdsl  # high level elasticsearch library
from elasticsearch import Elasticsearch  # low level official elasticsearch library


ELASTICSEARCH_HOSTS = [
    'http://es1.myhost.com:9200',
    'http://es2.myhost.com:9200',
    'http://es3.myhost.com:9200',
]


class ReplaceHostFilter(logging.Filter):
    """
    Replace "localhost:9200" with real host
    """
    def __init__(self, real_host, *args, **kwargs):
        self.real_host = real_host
        super(ReplaceHostFilter, self).__init__(*args, **kwargs)

    def filter(self, record):
        record.msg = record.msg.replace('http://localhost:9200', self.real_host)
        record.msg = record.msg.replace('curl', 'curl -sS')
        record.msg += ' | python -mjson.tool'
        return True


# Set up logging
host_replacer = ReplaceHostFilter(ELASTICSEARCH_HOSTS[0])
formatter = logging.Formatter("%(levelname)s %(asctime)s %(name)s - %(message)s")
sh = logging.StreamHandler()
sh.setFormatter(formatter)
eslogger = logging.getLogger('elasticsearch.trace')
eslogger.setLevel('INFO')
eslogger.addFilter(host_replacer)
eslogger.addHandler(sh)


# Make a query
esclient = Elasticsearch(hosts=ELASTICSEARCH_HOSTS)
search = esdsl.Search(using=esclient, index='my_index')
filt = esdsl.F('term', myTerm=1234)
search = search.filter(filt)
response = search.execute()
print response.hits.total

Log output showing the actual host instead of localhost:

INFO 2015-07-28 13:48:54,464 elasticsearch.trace - curl -sS -XGET 'http://es1.myhost.com:9200/my_index/_search?pretty' -d '{
  "query": {
    "filtered": {
      "filter": {
        "term": {
          "myTerm": 1234
        }
      },
      "query": {
        "match_all": {}
      }
    }
  }
}' | python -mjson.tool

Calling Javascript from Python to de-CloudFlare scraped content

Yesterday I wrote a script to scrape my own web page because I screwed up the CSV export feature and Product needed the data. One problem was that the CloudFlare CDN obfuscated the email addresses on the page. My solutioncrazy hack: running a Node.js script to de-obfuscate the email from my Python scraping script.

Example obfuscated email stuff from CloudFlare:

<a href="/cdn-cgi/l/email-protection#d4b7b5a6b194a4b1a0e7e2e4fab7bbb9">
    <span class="__cf_email__" data-cfemail="0162607364417164753237312f626e6c">[email&#160;protected]</span>
    <script cf-hash='f9e31' type="text/javascript">
     /* <![CDATA[ */!function(){try{var t="currentScript"in document?document.currentScript:function(){for(var t=document.getElementsByTagName("script"),e=t.length;e--;)if(t[e].getAttribute("cf-hash"))return t[e]}();if(t&&t.previousSibling;){var e,r,n,i,c=t.previousSibling,a=c.getAttribute("data-cfemail");if(a){for(e="",r=parseInt(a.substr(0,2),16),n=2;a.length-n;n+=2)i=parseInt(a.substr(n,2),16)^r,e+=String.fromCharCode(i);e=document.createTextNode(e),c.parentNode.replaceChild(e,c)}}}catch(u){}}();/* ]]> */
    </script>
</a>

Using jsbeautifier.org, I adapted the Javascript from above into this Node.js script, decloudflare.js.

var e, r, n, i, a = process.argv[2];
for (e = "", r = parseInt(a.substr(0, 2), 16), n = 2; a.length - n; n += 2) i = parseInt(a.substr(n, 2), 16) ^ r, e += String.fromCharCode(i);
console.log(e);

Example usage:

$ node decloudflare.js 0162607364417164753237312f626e6c
care@pet360.com

I used the Naked library (thanks to Sweetmeat) to call the Node.js script. (Though probably I could've just used the subprocess module.)

$ pip install Naked 
from Naked.toolshed.shell import muterun_js

def decloudflare_email(cfemail):
    resp = muterun_js('decloudflare.js', cfemail)
    return resp.stdout.rstrip()

cfemail = '0162607364417164753237312f626e6c'
print 'cfemail from python: ' + cfemail
email = decloudflare_email(cfemail)
print 'email from python: ' + email
cfemail from python: 0162607364417164753237312f626e6c
email from python: care@pet360.com

Customizing Bootstrap (Sass) using Grunt

Update 2015-12-02: I updated the post to use npm instead of Bower to install Bootstrap because it eliminates an extra tool and I hear this is the preferred method.

Update 2015-02-22: My co-workers informed me that Grunt is so last year and Gulp is the new hotness. You wish this post covered Gulp, but instead it covers Grunt.

I recently converted this blog to use Twitter's Bootstrap CSS framework to make it responsive. (In particular, I wanted to read it on my phone.) I was using Blueprint CSS from 2008 so it was about time for an update. Unfortunately my design is also from 2008 but I won't update that. (Web design is hard.)

My problem was that 2014 Bootstrap's 200px gutter width didn't match my 2008 design's 2px gutter. So I wanted to customize the gutter width.

There are a few ways to customize Bootstrap. One option is creating a custom version on Bootstrap's website and downloading it for use on your site. However this doesn't allow you to change things quickly as you develop. Another option is overriding Bootstrap's style in your site stylesheet. However, it's hacky to write compilcated selectors to override something in multiple places when it is set in a single variable in Bootstrap. For example, to change the gutter width, here is the single variable that needs to be changed.

Approach

Here is the approach I took to customize Bootstrap. I'm running Ubuntu 14.10 Utopic Unicorn 64-bit.

  • Install the Sass version of Bootstrap using npm. The standard Bootstrap project uses Less but my limited knowledge of frontend technology tells me Sass (specifically SCSS) is a better choice. I previously used Bower to install Bootstrap, but I learned that npm can now be used instead of Bower in most cases.
  • Override Bootstrap's Sass variables, remove unneeded components, and combine Bootstrap with my site's stylesheet using Sass @imports.
  • Compile the Sass SCSS files to CSS using Grunt and grunt-contrib-sass.
  • Commit the compiled Sass files to git. An alternative is to compile the files as part of the deploy process. At work, it is a pain keeping these files in git because there are many merge conflicts and differences in build tool versions and platforms between developers. For my blog, I am the only committer so I won't run into this.

Project directory structure

Here is what my project directory structure looks like:

my-project
├── Gruntfile.js
├── node_modules
│   ├── bootstrap-sass
│   ├── grunt
│   ├── grunt-contrib-sass
│   └── grunt-contrib-watch
├── package.json
├── sass
│   ├── _bootstrap-variables-override.scss
│   ├── _bootstrap.scss
│   └── mystyle.scss
└── static
    └── css
        └── mystyle.css

Install Sass

Sass is installed using RubyGems, the Ruby package manager.

$ sudo apt-get install ruby 
$ sudo gem install sass 
$ sass --version 
Sass 3.4.13 (Selective Steve)

Install grunt-cli

Grunt depends on npm, the Node.js package manager.

$ sudo add-apt-repository ppa:chris-lea/node.js 
$ sudo apt-get update 
$ sudo apt-get install nodejs 
$ npm --version
1.4.28
$ sudo npm install -g grunt-cli 
$ grunt --version 
grunt-cli v0.1.13

Install bootstrap-sass with npm

bootstrap-sass is the official Sass port of Bootstrap. I created this package.json file, /tmp/my-project/package.json. For more information about the package.json file, see the npm documentation.

{
  "name": "my-project",
  "version": "0.1.0",
  "dependencies": {
    "bootstrap-sass": "3.3.1"
  }
}

Then I ran npm install to download the files. The packages are stored in the node_modules directory.

$ cd /tmp/my-project 
$ npm install 

Customize Boostrap

To customize Boostrap, I copied the node_modules/bootstrap-sass/assets/stylesheets/boostrap.scss file to my sass directory, removed the components that I didn't need, and added a line for my custom variable overrides. Here is my final file, named /tmp/my-project/sass/_bootstrap.scss:

// Variable overrides come first (without !default). Bootstrap default variables
// come second because they use !default (they won't get set if set already) and
// some of them depend on our overrides.
@import "bootstrap-variables-override";

// Core variables and mixins
@import "bootstrap/variables";
@import "bootstrap/mixins";

// Reset and dependencies
@import "bootstrap/normalize";
@import "bootstrap/print";
@import "bootstrap/glyphicons";

// Core CSS
@import "bootstrap/scaffolding";
@import "bootstrap/type";
@import "bootstrap/code";
@import "bootstrap/grid";
@import "bootstrap/tables";
@import "bootstrap/forms";
@import "bootstrap/buttons";

// // I am not using these components so I commented them out to make the CSS file smaller
// // Components
// @import "bootstrap/component-animations";
// @import "bootstrap/dropdowns";
// @import "bootstrap/button-groups";
// @import "bootstrap/input-groups";
// @import "bootstrap/navs";
// @import "bootstrap/navbar";
// @import "bootstrap/breadcrumbs";
// @import "bootstrap/pagination";
// @import "bootstrap/pager";
// @import "bootstrap/labels";
// @import "bootstrap/badges";
// @import "bootstrap/jumbotron";
// @import "bootstrap/thumbnails";
// @import "bootstrap/alerts";
// @import "bootstrap/progress-bars";
// @import "bootstrap/media";
// @import "bootstrap/list-group";
// @import "bootstrap/panels";
// @import "bootstrap/responsive-embed";
// @import "bootstrap/wells";
// @import "bootstrap/close";

// // Components w/ JavaScript
// @import "bootstrap/modals";
// @import "bootstrap/tooltip";
// @import "bootstrap/popovers";
// @import "bootstrap/carousel";

// Utility classes
@import "bootstrap/utilities";
@import "bootstrap/responsive-utilities";

The Boostrap variables are located in node_modules/bootstrap-sass/assets/stylesheets/bootstrap/_variables.scss I copied the ones that I was interested in into a _bootstrap-variables-override.scss file. Be sure to remove the !default flag and import this file before the default bootstrap variables. For more information about !default, see the Sass documentation. Here is my /tmp/my-project/sass/_bootstrap-variables-override.scss file:

// overrides here (do not use !default here)
$grid-gutter-width: 10px;
$font-size-base: 16px;
$headings-font-weight: 800;

In my site stylesheet, I import _bootstrap.scss. Here is my /tmp/my-project/sass/mystyle.scss file:

@import "bootstrap";
/* add site style here */

Install grunt plugins

To compile the Sass files to CSS, I am using grunt-contrib-sass. I added a devDependencies section to my package.json file which contains grunt and two grunt plugins. Here is my /tmp/my-project/package.json. For more information about installing plugins, see the Grunt documentation.

{
  "name": "my-project",
  "version": "0.1.0",
  "dependencies": {
    "bootstrap-sass": "3.3.1"
  },
  "devDependencies": {
    "grunt": "0.4.5",
    "grunt-contrib-sass": "0.8.1",
    "grunt-contrib-watch": "0.6.1"
  }
}

I ran npm install again to install grunt, grunt-contrib-sass, and grunt-contrib-watch.

$ cd /tmp/my-project 
$ npm install 

Create Gruntfile.js

My Gruntfile.js uses the grunt-contrib-sass plugin to compile the Sass to CSS. My input file is sass/mystyle.scss and my output file is static/css/mystyle.css. I set the loadPath so that Sass can find the Bootstrap SCSS files in the node_modules directory to import. Alternatively, I could specify the full path to import in /tmp/my-project/sass/_bootstrap.scss. For example: @import "../node_modules/bootstrap-sass/assets/stylesheets/bootstrap/variables"; I am also using the grunt-contrib-watch plugin during development to automatically run the compilation whenever a SCSS file changes. For more information on configuring tasks, see the Grunt documentation. Here is my Gruntfile, /tmp/my-project/Gruntfile.js:

module.exports = function(grunt) {
    grunt.initConfig({
        sass: {
            // this is the "dev" Sass config used with "grunt watch" command
            dev: {
                options: {
                    style: 'expanded',
                    // tell Sass to look in the Bootstrap stylesheets directory when compiling
                    loadPath: 'node_modules/bootstrap-sass/assets/stylesheets'
                },
                files: {
                    // the first path is the output and the second is the input
                    'static/css/mystyle.css': 'sass/mystyle.scss'
                }
            },
            // this is the "production" Sass config used with the "grunt buildcss" command
            dist: {
                options: {
                    style: 'compressed',
                    loadPath: 'node_modules/bootstrap-sass/assets/stylesheets'
                },
                files: {
                    'static/css/mystyle.css': 'sass/mystyle.scss'
                }
            }
        },
        // configure the "grunt watch" task
        watch: {
            sass: {
                files: 'sass/*.scss',
                tasks: ['sass:dev']
            }
        }
    });
    grunt.loadNpmTasks('grunt-contrib-sass');
    grunt.loadNpmTasks('grunt-contrib-watch');
    // "grunt buildcss" is the same as running "grunt sass:dist".
    // if I had other tasks, I could add them to this array.
    grunt.registerTask('buildcss', ['sass:dist']);
};

Run grunt

To compile the Sass to CSS for production, I run grunt buildcss. This creates the output CSS file /tmp/my-project/static/css/mystyle.css.

$ cd /tmp/my-project 
$ grunt buildcss 

During development, I use grunt watch to build whenever a SCSS (Sass) file changes.

$ cd /tmp/my-project 
$ grunt watch 

Other Notes

See Also / References

Ordered a Lenovo X1 Carbon for 40% off

I ordered a ~$1200 2.8 lb. 14" Lenovo Thinkpad X1 Carbon (2014, 2nd generation, 20A7) today at 40% off for the Black Friday / Cyber Monday sale. I hope I like the keyboard. I splurged for an Intel Core i7-4600U, 8GB RAM, 256GB SSD, and the WQHD 2560x1440 display. I have enjoyed 7 months with my ~$200 2.8 lb. 11" Acer C720 Chromebook. It's a pleasant contrast to my work-issued 5.9 lb. 17" Lenovo Thinkpad W510. Crouton worked well for running Linux on the Chromebook but I wanted a dedicated Linux laptop and a better screen. I will bequeath the C720 to my wife since she currently uses my 7+ year old Dell Inspiron E1405. I considered the 3.5 lb. 14" Lenovo Thinkpad T440s due to better battery/keyboard/flexibility, but the X1 Carbon was ~$200 cheaper for a similarly spec'd model due to a bigger discount and I liked the thinner and lighter design and better display. Many reviews complained about the new keyboard layout and adaptive function row. I hope key remapping will reduce the pain enough. LWN.net editor, Jonathan Corbet mentioned he bought an X1 Carbon in High-DPI displays and Linux, so it can't be too bad, right?

Here is the full 2014 2nd Gen ThinkPad X1 Carbon (Type 20A7, 20A8) spec sheet.

Update 2015-03-14: Lenovo released the 3rd generation of the X1 Carbon which fixes the keyboard and improves battery life. Too bad I didn't wait a few months. Here is the new 2015 3rd Gen spec sheet.

How to install grunt on Ubuntu 14.04

Grunt is a Javascript task runner that can be used to compile Sass, run JSHint, or run many other plugins. It depends on the Node.js package manager, npm.

If you use the standard Ubuntu Trusty Tahr repository to install nodejs/npm, you will get this error when running grunt:

/usr/bin/env: node: No such file or directory
Instead, use the chris-lea nodejs repository.

Install nodejs, npm, and grunt-cli

$ sudo add-apt-repository ppa:chris-lea/node.js 
$ sudo apt-get update 
$ sudo apt-get install nodejs 
$ sudo npm install -g grunt-cli 

Install grunt in your project directory

$ cd ~/myproject 
$ echo "{}" > package.json 
$ npm install grunt --save-dev 

Verify grunt is installed

$ nodejs --version 
v0.10.33
$ npm --version 
1.4.28
$ grunt --version 
grunt-cli v0.1.13
grunt v0.4.5

Run a simple grunt task

  1. $ cd ~/myproject
    
  2. Create a package.json file:
    {
      "name": "my-project-name",
      "version": "0.1.0",
      "devDependencies": {
        "grunt": "~0.4.5",
        "grunt-contrib-uglify": "~0.5.0"
      }
    }
    
  3. Install grunt-contrib-uglify
    $ npm install 
    npm WARN package.json my-project-name@0.1.0 No description
    npm WARN package.json my-project-name@0.1.0 No repository field.
    npm WARN package.json my-project-name@0.1.0 No README data
    grunt-contrib-uglify@0.5.1 node_modules/grunt-contrib-uglify
    ├── chalk@0.5.1 (ansi-styles@1.1.0, escape-string-regexp@1.0.2, supports-color@0.2.0, strip-ansi@0.3.0, has-ansi@0.1.0)
    ├── lodash@2.4.1
    ├── maxmin@0.2.2 (figures@1.3.5, pretty-bytes@0.1.2, gzip-size@0.2.0)
    └── uglify-js@2.4.15 (uglify-to-browserify@1.0.2, async@0.2.10, optimist@0.3.7, source-map@0.1.34)
    
  4. Get an example unminified JS file:
    $ wget http://code.jquery.com/jquery-2.1.1.js 
    --2014-11-22 00:47:31--  http://code.jquery.com/jquery-2.1.1.js
    Resolving code.jquery.com (code.jquery.com)... 94.31.29.53, 94.31.29.230
    Connecting to code.jquery.com (code.jquery.com)|94.31.29.53|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 247351 (242K) [application/x-javascript]
    Saving to: ‘jquery-2.1.1.js’
    
    100%[================================================================================================================>] 247,351     --.-K/s   in 0.1s    
    
    2014-11-22 00:47:31 (1.71 MB/s) - ‘jquery-2.1.1.js’ saved [247351/247351]
    
  5. Create a Gruntfile.js file:
    module.exports = function(grunt) {
      grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        uglify: {
          build: {
            src: 'jquery-2.1.1.js',
            dest: 'jquery-2.1.1.min.js'
          }
        }
      });
      grunt.loadNpmTasks('grunt-contrib-uglify');
      grunt.registerTask('default', ['uglify']);
    };
    
  6. Run the grunt task:
    $ grunt 
    Running "uglify:build" (uglify) task
    
    Done, without errors.
    
  7. You should now have a minified file, jquery-2.1.1.min.js
    $ ls -gG jquery* 
    -rw-rw-r-- 1 247351 2014 10/23 17:16 jquery-2.1.1.js
    -rw-rw-r-- 1  84113 2014 11/22 00:48 jquery-2.1.1.min.js
    

References

An example using Python's groupby and defaultdict to do the same task

Here is some data that I want to group by model:

SOME_DATA = [
    {'model': u'Yaris', 'some_value': 11202, 'trim_name': u'3-Door L Manual'},
    {'model': u'Yaris', 'some_value': 19269, 'trim_name': u'3-Door LE Automatic'},
    {'model': u'Corolla', 'some_value': 27119, 'trim_name': u'L Automatic'},
    {'model': u'Corolla', 'some_value': 32262, 'trim_name': u'LE'},
    {'model': u'Corolla', 'some_value': 37976, 'trim_name': u'S Premium'},
    {'model': u'Camry', 'some_value': 39730, 'trim_name': u'LE 4-Cyl'},
    {'model': u'Camry', 'some_value': 45761, 'trim_name': u'XSE 4-Cyl'},
    {'model': u'Yaris', 'some_value': 48412, 'trim_name': u'3-Door L Automatic'},
    {'model': u'Camry', 'some_value': 55423, 'trim_name': u'XLE 4-Cyl'},
    {'model': u'Corolla', 'some_value': 57055, 'trim_name': u'ECO Premium'},
    {'model': u'Corolla', 'some_value': 61296, 'trim_name': u'ECO Plus'},
    {'model': u'Camry', 'some_value': 63660, 'trim_name': u'XSE V6'},
    {'model': u'Yaris', 'some_value': 65570, 'trim_name': u'5-Door LE Automatic'},
    {'model': u'Camry', 'some_value': 67461, 'trim_name': u'XLE V6'},
    {'model': u'Corolla', 'some_value': 73602, 'trim_name': u'S'},
    {'model': u'Yaris', 'some_value': 74158, 'trim_name': u'5-Door SE Manual'},
    {'model': u'Corolla', 'some_value': 74249, 'trim_name': u'LE Plus'},
    {'model': u'Corolla', 'some_value': 78386, 'trim_name': u'ECO'},
    {'model': u'Camry', 'some_value': 82747, 'trim_name': u'SE 4-Cyl'},
    {'model': u'Corolla', 'some_value': 83162, 'trim_name': u'LE Premium'},
    {'model': u'Corolla', 'some_value': 84863, 'trim_name': u'S Plus Manual'},
    {'model': u'Yaris', 'some_value': 90313, 'trim_name': u'5-Door L Automatic'},
    {'model': u'Corolla', 'some_value': 90452, 'trim_name': u'L Manual'},
    {'model': u'Yaris', 'some_value': 93152, 'trim_name': u'5-Door SE Automatic'},
    {'model': u'Corolla', 'some_value': 94973, 'trim_name': u'S Plus CVT'},
]

This can be done using defaultdict from the collections module.

import collections

grouped = collections.defaultdict(list)
for item in SOME_DATA:
    grouped[item['model']].append(item)

for model, group in grouped.items():
    print
    print model
    pprint(group, width=150)

Here are the results:

Yaris
[{'model': u'Yaris', 'some_value': 27065, 'trim_name': u'5-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 32757, 'trim_name': u'5-Door SE Automatic'},
 {'model': u'Yaris', 'some_value': 57344, 'trim_name': u'3-Door L Manual'},
 {'model': u'Yaris', 'some_value': 64002, 'trim_name': u'5-Door SE Manual'},
 {'model': u'Yaris', 'some_value': 77974, 'trim_name': u'3-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 92658, 'trim_name': u'3-Door LE Automatic'},
 {'model': u'Yaris', 'some_value': 98769, 'trim_name': u'5-Door LE Automatic'}]

Camry
[{'model': u'Camry', 'some_value': 30247, 'trim_name': u'XSE 4-Cyl'},
 {'model': u'Camry', 'some_value': 33809, 'trim_name': u'XSE V6'},
 {'model': u'Camry', 'some_value': 65637, 'trim_name': u'LE 4-Cyl'},
 {'model': u'Camry', 'some_value': 67329, 'trim_name': u'SE 4-Cyl'},
 {'model': u'Camry', 'some_value': 76269, 'trim_name': u'XLE 4-Cyl'},
 {'model': u'Camry', 'some_value': 87438, 'trim_name': u'XLE V6'}]

Corolla
[{'model': u'Corolla', 'some_value': 11239, 'trim_name': u'S'},
 {'model': u'Corolla', 'some_value': 27356, 'trim_name': u'S Plus Manual'},
 {'model': u'Corolla', 'some_value': 44792, 'trim_name': u'L Manual'},
 {'model': u'Corolla', 'some_value': 56252, 'trim_name': u'ECO Premium'},
 {'model': u'Corolla', 'some_value': 78570, 'trim_name': u'S Plus CVT'},
 {'model': u'Corolla', 'some_value': 78964, 'trim_name': u'LE Premium'},
 {'model': u'Corolla', 'some_value': 82116, 'trim_name': u'ECO'},
 {'model': u'Corolla', 'some_value': 85467, 'trim_name': u'S Premium'},
 {'model': u'Corolla', 'some_value': 87099, 'trim_name': u'L Automatic'},
 {'model': u'Corolla', 'some_value': 91974, 'trim_name': u'LE Plus'},
 {'model': u'Corolla', 'some_value': 94862, 'trim_name': u'LE'},
 {'model': u'Corolla', 'some_value': 97625, 'trim_name': u'ECO Plus'}]

This can also be done using itertools.groupby. This method is probably better when working with large datasets because groupby returns the group as an iterator. (This is the reason I convert it to a list before printing.)

import itertools

def keyfunc(x):
    return x['model']

SOME_DATA = sorted(SOME_DATA, key=keyfunc)
for model, group in itertools.groupby(SOME_DATA, keyfunc):
    print
    print model
    pprint(list(group), width=150)

Here are the results:

Camry
[{'model': u'Camry', 'some_value': 36776, 'trim_name': u'SE 4-Cyl'},
 {'model': u'Camry', 'some_value': 56569, 'trim_name': u'LE 4-Cyl'},
 {'model': u'Camry', 'some_value': 57052, 'trim_name': u'XSE 4-Cyl'},
 {'model': u'Camry', 'some_value': 92360, 'trim_name': u'XLE V6'},
 {'model': u'Camry', 'some_value': 92756, 'trim_name': u'XSE V6'},
 {'model': u'Camry', 'some_value': 94413, 'trim_name': u'XLE 4-Cyl'}]

Corolla
[{'model': u'Corolla', 'some_value': 13307, 'trim_name': u'L Automatic'},
 {'model': u'Corolla', 'some_value': 15726, 'trim_name': u'ECO Plus'},
 {'model': u'Corolla', 'some_value': 25579, 'trim_name': u'S'},
 {'model': u'Corolla', 'some_value': 31920, 'trim_name': u'ECO Premium'},
 {'model': u'Corolla', 'some_value': 34480, 'trim_name': u'LE'},
 {'model': u'Corolla', 'some_value': 44958, 'trim_name': u'S Plus Manual'},
 {'model': u'Corolla', 'some_value': 49606, 'trim_name': u'LE Premium'},
 {'model': u'Corolla', 'some_value': 59629, 'trim_name': u'LE Plus'},
 {'model': u'Corolla', 'some_value': 74226, 'trim_name': u'S Plus CVT'},
 {'model': u'Corolla', 'some_value': 75725, 'trim_name': u'L Manual'},
 {'model': u'Corolla', 'some_value': 82382, 'trim_name': u'ECO'},
 {'model': u'Corolla', 'some_value': 95633, 'trim_name': u'S Premium'}]

Yaris
[{'model': u'Yaris', 'some_value': 16789, 'trim_name': u'3-Door L Manual'},
 {'model': u'Yaris', 'some_value': 20349, 'trim_name': u'5-Door LE Automatic'},
 {'model': u'Yaris', 'some_value': 42897, 'trim_name': u'5-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 62045, 'trim_name': u'5-Door SE Automatic'},
 {'model': u'Yaris', 'some_value': 91913, 'trim_name': u'3-Door L Automatic'},
 {'model': u'Yaris', 'some_value': 94218, 'trim_name': u'5-Door SE Manual'},
 {'model': u'Yaris', 'some_value': 97979, 'trim_name': u'3-Door LE Automatic'}]