SaltyCrane: ubuntuhttps://www.saltycrane.com/blog/2015-09-10T08:26:36-07:00Switching to OS X and front end development
2015-09-10T08:26:36-07:00https://www.saltycrane.com/blog/2015/09/switching-osx-and-front-end-development/<p>After 7 years, I've yielded to the Dark Side and switched from Ubuntu to OS X on my work laptop.
I've also switched from Python and back end web development to JavaScript and front end development.
The former is mostly to support the latter.
</p>
<p>Linux is rare<sup>1</sup>, especially among front end developers,
and I want to make it easy to collaborate as I learn new things.
I've had problems working with Photoshop files in GIMP and I couldn't run
the iOS simulator. <a href="https://news.ycombinator.com/item?id=10812214">
Issues with Linux device drivers</a> don't help.
</p>
<p>I'm choosing front end development because I want to code closer to the end user.<sup>2</sup>
In small part like Ian Bicking
<a href="http://www.ianbicking.org/blog/2014/02/saying-goodbye-to-python.html">
wrote last year</a>,
I feel unexcited about back end development and really excited about
JavaScript and front end development. I'm excited about
ES 2015 and
React and
React Native and
CSS transitions.<sup>3</sup>
I'm even coming around to Node.js.
JavaScript is <a href="https://www.destroyallsoftware.com/talks/wat">uglier</a>
than Python, but it's getting
<a href="https://medium.com/google-developers/exploring-es7-decorators-76ecb65fb841">
better</a>
and there are things
<a href="https://glyph.twistedmatrix.com/2015/09/software-you-can-use.html">
Python can't do</a> that
<a href="http://steve-yegge.blogspot.com/2007/02/next-big-language.html">
JavaScript can</a>.<sup>4</sup> If only beauty mattered, maybe I'd use Scheme.<sup>5</sup>
</p>
<p>I'm sure I will hate OS X at first, but hopefully it will be good in the
long run. If anyone can recommend a tiling window manager like
<a href="http://www.qtile.org/">Qtile</a> for OS X, please let me know.
</p>
<p><em>(I will continue using Emacs because <a href="http://emacsrocks.com/">Emacs rocks!</a> <sup>6</sup>)</em>
</p>
<hr>
<small>
<ol>
<li>I think I was the last person at my company running Linux.</li>
<li>I've been trying to do front end work for years now, but
I finally got a sensible chance to switch as my company is
changing it's technology stack from Python to Ruby and
Backbone/Angular to React.</li>
<li><em>Update 2016-01-04:</em> Here are even <a href="http://www.2ality.com/2016/01/web-technologies-2015.html">
more exciting web technologies</a>: Electron, progressive web apps, and WebAssembly.</li>
<li><em>Update 2016-01-01:</em> I found James Hague had <a href="http://prog21.dadgum.com/203.html">
similar thoughts</a> on Python and JavaScript.</li>
<li>Speaking of functional languages and JavaScript, <a href="http://elm-lang.org/">Elm</a> sounds pretty cool.</li>
<li><em>Update 2016-01-26:</em> <a href="/blog/2015/12/switching-emacs-vim-actually-spacemacs/">Or will I?</a></li>
</ol>
</small>
Setting up the Lenovo Thinkpad X1 Carbon (2nd gen) fingerprint reader in Ubuntu 14.04
2014-12-20T23:34:50-08:00https://www.saltycrane.com/blog/2014/12/setting-lenovo-thinkpad-x1-carbon-2nd-gen-fingerprint-reader-ubuntu-1404/<p>
These are the steps I used to get the fingerprint sensor on the Lenovo Thinkpad X1 Carbon (2014 2nd generation) to work on Ubuntu Linux 14.04 Trusty Tahr 64-bit. After googling through many articles, this is the one that solved it for me:
<a href="http://thinkpadmint.blogspot.com/2014/02/getting-t440s-validity-vfs5011.html">
http://thinkpadmint.blogspot.com/2014/02/getting-t440s-validity-vfs5011.html
</a> Thank you!
</p>
<p>
The X1 Carbon has a Validity Sensors 138a:0017 sensor.
As of this writing, this sensor is not officially supported by libfprint, but you can compile a driver.
</p>
<h4 id="find-sensor">See what sensor you have</h4>
<pre class="console">$ lsusb
Bus 001 Device 003: ID 04ca:7036 Lite-On Technology Corp.
Bus 001 Device 002: ID 8087:8000 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 003: ID 8087:07dc Intel Corp.
Bus 002 Device 002: ID 138a:0017 Validity Sensors, Inc.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub</pre>
<h4 id="dependencies">Install dependencies</h4>
<pre class="console">
$ sudo apt-get install libmagickcore-dev
$ sudo apt-get install libusb-1.0.0-dev libnss3-dev libglib2.0-dev
$ sudo apt-get install libxv-dev
$ sudo apt-get install libtool
$ sudo apt-get install fprintd
$ sudo apt-get install automake
</pre>
<h4 id="compile-driver">Compile the driver</h4>
<ul>
<li>Download tarball from
<a href="https://github.com/abbradar/fprint_vfs5011/tree/faa090818200ca3ea6bfac8bb510e5e01a246c34">
https://github.com/abbradar/fprint_vfs5011/tree/faa090818200ca3ea6bfac8bb510e5e01a246c34
</a>
</li>
<li>Unzip, autogen, configure, make, install
<pre class="console">
$ cd ~/Downloads
$ unzip fprint_vfs5011-faa090818200ca3ea6bfac8bb510e5e01a246c34.zip
$ cd fprint_vfs5011-faa090818200ca3ea6bfac8bb510e5e01a246c34
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install
</pre>
</li>
<li>Copy file
<pre class="console">
$ sudo cp /lib/udev/rules.d/40-libfprint0.rules /etc/udev/rules.d/
</pre>
</li>
<li>Add a section to the file
<pre class="console">
$ sudo vi /etc/udev/rules.d/40-libfprint0.rules
</pre>
<pre>
# Validity VFS5011
SUBSYSTEM=="usb", ATTRS{idVendor}=="138a", ATTRS{idProduct}=="0017", ATTRS{dev}=="*", ATTR{power/control}="auto", MODE="0664", GROUP="plugdev"
</pre>
</li>
</ul>
<h4 id="install-gui">Install the fingerprint GUI</h4>
<pre class="console">
$ sudo apt-add-repository ppa:fingerprint/fingerprint-gui
$ sudo apt-get update
$ sudo apt-get install libbsapi policykit-1-fingerprint-gui fingerprint-gui
</pre>
<h4 id="run-gui">Run the fingerprint GUI and enroll a finger</h4>
<pre class="console">
$ fingerprint-gui
</pre>
<h4>Other links</h4>
<ul>
<li><a href="http://zecheru.com/thinkpad-fingerprint-ubuntu-14-04/">http://zecheru.com/thinkpad-fingerprint-ubuntu-14-04/</a></li>
<li><a href="https://wiki.archlinux.org/index.php/Lenovo_ThinkPad_X1_Carbon_%28Gen_2%29#Fingerprint_Reader">https://wiki.archlinux.org/index.php/Lenovo_ThinkPad_X1_Carbon_%28Gen_2%29#Fingerprint_Reader</a></li>
<li><a href="http://askubuntu.com/questions/442838/fingerprint-reader-driver-for-validity-sensors-fingerprint-scanner">http://askubuntu.com/questions/442838/fingerprint-reader-driver-for-validity-sensors-fingerprint-scanner</a></li>
<li><a href="https://bugs.launchpad.net/ubuntu/+source/libfprint/+bug/1382165">https://bugs.launchpad.net/ubuntu/+source/libfprint/+bug/1382165</a></li>
<li><a href="http://thinkpadmint.blogspot.com/2014/02/getting-t440s-validity-vfs5011.html">http://thinkpadmint.blogspot.com/2014/02/getting-t440s-validity-vfs5011.html</a></li>
<li><a href="https://github.com/abbradar/fprint_vfs5011">https://github.com/abbradar/fprint_vfs5011</a></li>
<li><a href="https://github.com/ars3niy/fprint_vfs5011/issues/4">https://github.com/ars3niy/fprint_vfs5011/issues/4</a></li>
</ul>
Testing HTTPS w/ Flask's development server using stunnel on Ubuntu
2012-10-17T17:30:11-07:00https://www.saltycrane.com/blog/2012/10/testing-https-w-flasks-development-server-using-stunnel-ubuntu/<p>Our website is served over HTTPS. To more easily test certain issues (e.g. mixed mode content warnings, or Mapquest SSL tile servers), I wanted to access my Flask local development server over HTTPS. These two articles describe how to do this using <a href="https://www.stunnel.org/index.html">stunnel</a>: <a href="http://www.ianlewis.org/en/testing-https-djangos-development-server">Testing HTTPS with Django's Development Server</a>, <a href="http://tech.xster.net/tips/django-development-server-with-https/">Django Development Server with HTTPS</a>. Using stunnel, you can hit pages on your Django/Flask local dev server over HTTPS instead of HTTP. Here is how I installed it on Ubuntu Precise 12.04:</p>
<ul>
<li>Install SSL development files
<pre class="console">$ sudo apt-get install libssl-dev </pre>
</li>
<li>Go to <a href="https://www.stunnel.org/downloads.html">https://www.stunnel.org/downloads.html</a> and download stunnel-4.54.tar.gz</li>
<li>Unpack, compile, install.
<pre>$ tar xvf stunnel-4.54.tar.gz
$ cd stunnel-4.54
$ ./configure --prefix=/home/saltycrane/lib/stunnel-4.54
$ make
$ make install </pre>
NOTE: the <code>make install</code> step asked me a number of questions and created a certificate file at <code>/home/saltycrane/lib/stunnel-4.54/etc/stunnel/stunnel.pem</code>. Accept all the defaults for the certificate information (accurate certificate information isn't needed for this application).</li>
<li>Create a stunnel configuration file, /home/saltycrane/lib/stunnel-4.54/etc/stunnel/dev_https:
<pre>pid =
cert = /home/saltycrane/lib/stunnel-4.54/etc/stunnel/stunnel.pem
debug = 7
foreground = yes
[https]
accept = 7000
connect = 5000</pre>
</li>
<li>Start stunnel:
<pre class="console">$ /home/saltycrane/lib/stunnel-4.54/bin/stunnel /home/saltycrane/lib/stunnel-4.54/etc/stunnel/dev_https
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Clients allowed=500
2012.10.17 17:40:52 LOG5[12468:140357811214080]: stunnel 4.54 on x86_64-unknown-linux-gnu platform
2012.10.17 17:40:52 LOG5[12468:140357811214080]: Compiled/running with OpenSSL 1.0.1 14 Mar 2012
2012.10.17 17:40:52 LOG5[12468:140357811214080]: Threading:PTHREAD SSL:+ENGINE+OCSP Auth:none Sockets:POLL+IPv6
2012.10.17 17:40:52 LOG5[12468:140357811214080]: Reading configuration from file /home/saltycrane/lib/stunnel-4.54/etc/stunnel/dev_https
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Compression not enabled
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Snagged 64 random bytes from /home/saltycrane/.rnd
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Wrote 1024 new random bytes to /home/saltycrane/.rnd
2012.10.17 17:40:52 LOG7[12468:140357811214080]: PRNG seeded successfully
2012.10.17 17:40:52 LOG6[12468:140357811214080]: Initializing service [https]
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Certificate: /home/saltycrane/lib/stunnel-4.54/etc/stunnel/stunnel.pem
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Certificate loaded
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Key file: /home/saltycrane/lib/stunnel-4.54/etc/stunnel/stunnel.pem
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Private key loaded
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Using DH parameters from /home/saltycrane/lib/stunnel-4.54/etc/stunnel/stunnel.pem
2012.10.17 17:40:52 LOG7[12468:140357811214080]: DH initialized with 1024-bit key
2012.10.17 17:40:52 LOG7[12468:140357811214080]: ECDH initialized with curve prime256v1
2012.10.17 17:40:52 LOG7[12468:140357811214080]: SSL options set: 0x00000004
2012.10.17 17:40:52 LOG5[12468:140357811214080]: Configuration successful
2012.10.17 17:40:52 LOG7[12468:140357811214080]: Service [https] (FD=7) bound to 0.0.0.0:7000
2012.10.17 17:40:52 LOG7[12468:140357811214080]: No pid file being created</pre>
</li>
<li>Start the python dev server:
<pre>$ HTTPS=1 python bin/runserver.py 0.0.0.0 5000 </pre>
</li>
<li>Go to https://localhost:7000 in your browser</li>
</ul>
<h4>See also</h4>
<ul>
<li><a href="https://forums.aws.amazon.com/thread.jspa?messageID=310507">
https://forums.aws.amazon.com/thread.jspa?messageID=310507</a></li>
<li><a href="http://www.stunnel.org/faq.html">
http://www.stunnel.org/faq.html</a></li>
</ul>
Virtualbox with Windows guest and Ubuntu host
2012-06-13T16:47:30-07:00https://www.saltycrane.com/blog/2012/06/virtualbox-windows-guest-and-ubuntu-host/<p>
Virtualbox 4.1, Ubuntu 10.10
<a href="https://www.virtualbox.org/manual/UserManual.html">
https://www.virtualbox.org/manual/UserManual.html</a>
</p>
<h4 id="contents">Contents</h4>
<ul>
<li><a href="#install-virtualbox">
Install Virtualbox</a></li>
<li><a href="#run-virtualbox">
Run Virtualbox</a></li>
<li><a href="#create-vm">
Create a VM</a></li>
<li><a href="#install-windows">
Run VM and install Windows</a></li>
<li><a href="#install-guest-additions">
Install Guest Additions</a></li>
<li><a href="#add-hard-disk">
Add another virtual hard disk</a></li>
<li><a href="#resize-disk-image">
Resize the VM disk image file</a></li>
<li><a href="#dev-server">
Access a Flask dev server running on the Ubuntu host from the Windows guest</a></li>
</ul>
<h4 id="install-virtualbox">Install Virtualbox</h4>
<p><a href="https://www.virtualbox.org/wiki/Linux_Downloads">
https://www.virtualbox.org/wiki/Linux_Downloads</a>
</p>
<ul>
<li>edit /etc/apt/sources.list.d/virtualbox.list
<pre>deb http://download.virtualbox.org/virtualbox/debian maverick contrib non-free</pre>
</li>
<li>install
<pre class="console">$ wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install virtualbox-4.1
$ sudo apt-get install dkms </pre>
</li>
</ul>
<h4 id="run-virtualbox">Run Virtualbox</h4>
<pre class="console">$ VirtualBox & </pre>
<h4 id="create-vm">Create a VM</h4>
<p>
<a href="https://www.virtualbox.org/manual/ch01.html#gui-createvm">
https://www.virtualbox.org/manual/ch01.html#gui-createvm</a>
My machine has 4GB ram and 225GB hdd. I chose 1GB ram and 30GB hdd for the guest to run MS SQL Server Management Studio.
Windows 7 Professional 32-bit took 7.3GB after installing.
</p>
<ul>
<li>click "New"</li>
<li>follow the wizard</li>
</ul>
<h4 id="install-windows">Run VM and install Windows</h4>
<p>
Need windows disc.
Need to download driver?
<a href="http://support.lenovo.com/en_US/research/hints-or-tips/detail.page?&DocID=HT063291">
http://support.lenovo.com/en_US/research/hints-or-tips/detail.page?&DocID=HT063291</a>
</p>
<ul>
<li>Put Windows disc in the drive</li>
<li>click "Start"</li>
</ul>
<h4 id="install-guest-additions">Install Guest Additions</h4>
<p>
Info: <a href="https://www.virtualbox.org/manual/ch04.html">
https://www.virtualbox.org/manual/ch04.html</a>
Install instructions: <a href="https://www.virtualbox.org/manual/ch04.html#additions-windows">
https://www.virtualbox.org/manual/ch04.html#additions-windows</a>
Install Guest Addtions to allow sharing between guest/host, better screen resolution
</p>
<ul>
<li>Run the Windows VM</li>
<li>click "Devices" -> "Install Guest Additions..."</li>
<li>click "Run VBoxWindowsAdditions.exe", click on stuff to install the stuff</li>
</ul>
<h4 id="add-hard-disk">Add another virtual hard disk</h4>
<ul>
<li>create virtual hard disk file
<ul>
<li>Shut down the guest VM</li>
<li>In the Oracle VM VirtualBox Manager, click "Settings"</li>
<li>Storage -> SATA Controller -> Add Hard Disk</li>
</ul>
</li>
<li>add the hard disk in windows
<a href="http://windows.microsoft.com/en-us/windows7/install-or-remove-a-hard-disk-drive">
http://windows.microsoft.com/en-us/windows7/install-or-remove-a-hard-disk-drive</a>
<ul>
<li>Open Computer Management by clicking the Start button Picture of the Start button, clicking Control Panel, clicking System and Security, clicking Administrative Tools, and then double-clicking Computer Management. Administrator permission required If you're prompted for an administrator password or confirmation, type the password or provide confirmation.</li>
<li>Right click on the new disk (e.g. Disk 1) and click "New Simple Volume..", run the wizard</li>
</ul>
</li>
</ul>
<h4 id="resize-disk-image">Resize the VM disk image file</h4>
<ol>
<li>resize the disk image file
<a href="https://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvdi">
https://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvdi</a>
From the docs This command "currently works only for VDI
and VHD formats, and only for the dynamically allocated variants, and can only
be used to expand (not shrink) the capacity"
<pre class="console">$ cd ~/VirtualBox VMs/Windows_7_1GB
$ VBoxManage modifyhd Windows_7_1GB.vdi --resize 20480 </pre>
</li>
<li>resize partition in Windows
<ul>
<li>Open Computer Management by clicking the Start button Picture of the Start button, clicking Control Panel, clicking System and Security, clicking Administrative Tools, and then double-clicking Computer Management. Administrator permission required If you're prompted for an administrator password or confirmation, type the password or provide confirmation.</li>
<li>Right click on the C: partition in Disk 0 -> "Extend Volume..."</li>
</ul>
</ol>
<h4 id="dev-server">Access a Flask dev server running on the Ubuntu host from the Windows guest</h4>
<br>
<p>
Reference: <a href="http://stackoverflow.com/questions/1261975/addressing-localhost-from-a-virtualbox-virtual-machine">
http://stackoverflow.com/questions/1261975/addressing-localhost-from-a-virtualbox-virtual-machine</a>
</p>
<p>In Ubuntu host:</p>
<ul>
<li>Run Flask dev server as normal on 127.0.0.1:5000</li>
</ul>
<p>In Windows guest:</p>
<ul>
<li>Open cmd.exe and run "ipconfig". Find the "Ethernet adapter Local Area Connection" section.
Find the "Default Gateway" is 10.0.2.2</li>
<li>Point the web browser at http://10.0.2.2:5000</li>
</ul>
</pre>
<h4 id="share-folder">How to share a folder</h4>
<p>You need Guest Additions to be installed. See <a href="#install-guest-additions">above</a>.
Reference: <a href="https://help.ubuntu.com/community/VirtualBox/SharedFolders">https://help.ubuntu.com/community/VirtualBox/SharedFolders</a>
Note: I did not need to do the mount step.
</p>
<ol>
<li>Ubuntu host: mkdir /home/yourusername/share</li>
<li>Windows guest: Boot the Guest operating system in VirtualBox.</li>
<li>Windows guest: Select Devices -> Shared Folders...</li>
<li>Windows guest: Choose the 'Add' button.</li>
<li>Windows guest: Input the following:
<ul>
<li>Folder Path: /home/yourusername/share</li>
<li>Folder Name: share</li>
<li>Optionally select the 'Make permanent' option </li>
</ul>
</li>
<li>Windows guest: Open cmd.exe and run:
<pre>net use x: \\vboxsvr\share </pre>
</li>
</ol>
Notes on sqlalchemy w/ pyodbc, freetds on Ubuntu
2011-09-28T19:03:19-07:00https://www.saltycrane.com/blog/2011/09/notes-sqlalchemy-w-pyodbc-freetds-ubuntu/<p>These are my notes for connecting to a MS SQL Server using Python on Linux.
My developement environment is Ubuntu 10.10 Maverick (64-bit). The production environment
is Redhat/Centos-based Scientific Linux 6. Here is the layers of stuff:
</p>
<ul>
<li><a href="http://www.freetds.org/">FreeTDS</a>: an open source ODBC driver. Started with version
0.82 and moved to 0.91. Other options:
<a href="http://blogs.technet.com/b/dataplatforminsider/archive/2011/11/28/available-today-preview-release-of-the-sql-server-odbc-driver-for-linux.aspx">
Microsoft ODBC driver for Linux</a> Free, currently only for 64-bit Redhat.
<a href="http://www.easysoft.com/products/data_access/odbc-sql-server-driver/index.html">
Easysoft ODBC driver
</a> Commercial, not free.
</li>
<li><a href="http://www.unixodbc.org/">unixODBC</a>: driver manager, a thin wrapper around the ODBC driver.
Version 2.2 (may move to version 2.3 if we use the Microsoft driver).
</li>
<li><a href="http://code.google.com/p/pyodbc/">pyodbc</a>: python library.
Started with 2.1.11, currently using 3.0.5.
I chose pyodbc over pymssql because, after minimal
research, it seemed to be the preferred choice.
Other options: <a href="http://code.google.com/p/pymssql/">pymssql</a> (GPL, doesn't use ODBC),
<a href="http://www.egenix.com/products/python/mxODBC/">mxODBC</a>
(commercial, much better documentation, may be more robust)
</li>
<li><a href="http://www.sqlalchemy.org/">SQLAlchemy</a>: wraps pyodbc, provides an ORM and better interface.
I chose to use sqlalchemy because it supported named parameters.
pyodbc only supported the 'qmark' paramstyle and pymssql returns
'pyformat' as the paramstyle, but it seems to actually be 'format'.
See <a href="http://www.python.org/dev/peps/pep-0249/">PEP-0249</a>
for more information on paramstyle.
See <a href="http://www.sqlalchemy.org/docs/core/expression_api.html#sqlalchemy.sql.expression.text">
http://www.sqlalchemy.org/docs/core/expression_api.html#sqlalchemy.sql.expression.text</a>
for more information on using named parameters.
</li>
</ul>
<h4 id="contents">Contents</h4>
<ul>
<li><a href="#install">Install (on Ubuntu)</a></li>
<li><a href="#connect-tsql">Try to connect using tsql (FreeTDS)</a></li>
<li><a href="#connect-python">Try to connect using Python</a></li>
<li><a href="#unicode-error">Error inserting unicode</a></li>
<li><a href="#select-minus-1-bug">Pyodbc SELECT -1 bug</a></li>
<li><a href="#list-tables">How to list tables in the database with sqlalchemy</a></li>
<li><a href="#pyodbc-freetds-truncates-varchar-instead-of-error">
pyodbc/FreeTDS truncates a varchar(max) upon insert instead of raising an error
<em>(Install FreeTDS 0.91 from source)</em>
</a></li>
<li><a href="#segfault">Segfault error when using Free TDS 0.91 and pyodbc 2.1.11</a></li>
<li><a href="#sql-server-equivalent-of-mysqls-limit">What is the MS SQL server equivalent of MySQL's LIMIT in a SELECT</a></li>
<li><a href="#versions">How to get the versions of stuff</a></li>
<li><a href="#microsoft-driver">Install the Microsoft ODBC driver for Linux</a></li>
<li><a href="#install-mxodbc">Install mxodbc</a></li>
<li><a href="#unixodbc-gui-tools">unixODBC GUI tools</a></li>
<li><a href="#sql-developer-ubuntu">Install SQL Developer on Ubuntu</a></li>
<li><a href="#calling-sprocs">Calling sprocs from sqlalchemy and pyodbc</a></li>
<li><a href="#errors">Other misc errors</a></li>
<li><a href="#install-pyodbc-sl6">Install pyodbc on Scientific Linux 6</a></li>
<li><a href="#see-also">See also</a></li>
<li><a href="#location-odbcinst-file">How to find the location of the odbcinst.ini file</a></li>
</ul>
<h4 id="install">Install (on Ubuntu)</h4>
<p>Set up odbc, freetds. Notes thanks to
<a href="http://www.pauldeden.com/2008/12/how-to-setup-pyodbc-to-connect-to-mssql.html">
http://www.pauldeden.com/2008/12/how-to-setup-pyodbc-to-connect-to-mssql.html
</a>
</p>
<pre class="console">$ sudo apt-get install unixodbc unixodbc-dev freetds-dev tdsodbc </pre>
<p>Edit <code>/etc/odbcinst.ini</code>:</p>
<pre>[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/odbc/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsS.so
CPTimeout =
CPReuse =
TDS Version = 7.2
client charset = utf-8</pre>
<p><em>UPDATE 2012-07-09:</em> On Ubuntu 12.04 (64-bit), the libraries are in a different location.
Also, Ubuntu 12.04 includes FreeTDS version 0.91.
</p>
<pre>[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
CPTimeout =
CPReuse =
TDS Version = 7.2
client charset = utf-8</pre>
<p>Install pyodbc</p>
<pre class="console">$ pip install pyodbc </pre>
<p>Install sqlalchemy</p>
<pre class="console">$ pip install SQLAlchemy </pre>
<p>This installed the following versions:</p>
<pre class="console">$ apt-cache show tdsodbc | grep Version:
Version: 0.82-7</pre>
<pre class="console">$ odbcinst --version
unixODBC 2.2.14</pre>
<pre class="console">$ pip freeze | grep pyodbc
pyodbc==2.1.11</pre>
<pre class="console">$ pip freeze | grep Alchemy
SQLAlchemy==0.7.3</pre>
<h4 id="connect-tsql">Try to connect using tsql (FreeTDS)</h4>
<p>Run the following command to ensure you can connect using tsql.
For more information see: <a href="http://www.freetds.org/userguide/confirminstall.htm">http://www.freetds.org/userguide/confirminstall.htm</a>
See also: <a href="http://www.freetds.org/userguide/serverthere.htm">FreeTDS User Guide: Chapter 8. Troubleshooting: Is the server there?</a>
</p>
<pre class="console">$ tsql -S 10.2.14.18 -U myusername -P mypassword
locale is "LC_CTYPE=en_US.UTF-8;LC_NUMERIC=en_US.UTF-8;LC_TIME=en_US.UTF-8;LC_COLLATE=C;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=en_US.UTF-8;LC_ADDRESS=en_US.UTF-8;LC_TELEPHONE=en_US.UTF-8;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
1>
</pre>
<h4 id="connect-python">Try to connect using Python</h4>
<p>pyodbc - this works</p>
<pre class="python">
import pyodbc
conn = pyodbc.connect(
'DRIVER={FreeTDS};SERVER=dev01.myserver.com;'
'DATABASE=MyDatabase;UID=MyUsername;PWD=mypassword;port=1234;'
'TDS_Version=8.0;')
print conn
</pre>
<p>sqlalchemy - this doesn't work</p>
<pre class="python">
import sqlalchemy
engine = sqlalchemy.create_engine(
'mssql+pyodbc://MyUsername:mypassword@dev01.myserver.com:1234/MyDatabase')
conn = engine.connect()
print conn
</pre>
<pre>
Traceback (most recent call last):
File "debug_sqlalchemy.py", line 22, in <module>
conn = engine.connect()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 2310, in connect
return self._connection_cls(self, **kwargs)
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 872, in __init__
self.__connection = connection or engine.raw_connection()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 2396, in raw_connection
return self.pool.unique_connection()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 169, in unique_connection
return _ConnectionFairy(self).checkout()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 370, in __init__
rec = self._connection_record = pool._do_get()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 696, in _do_get
con = self._create_connection()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 174, in _create_connection
return _ConnectionRecord(self)
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 255, in __init__
self.connection = self.__connect()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/pool.py", line 315, in __connect
connection = self.__pool._creator()
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
return dialect.connect(*cargs, **cparams)
File "/home/saltycrane/.virtualenvs/myproject/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 275, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnectW)') None None
</pre>
<p>sqlalchemy - this works</p>
<pre class="python">
import sqlalchemy
import urllib
engine = sqlalchemy.create_engine(
'mssql+pyodbc:///?odbc_connect=%s' % (
urllib.quote_plus(
'DRIVER={FreeTDS};SERVER=dev01.myserver.com;'
'DATABASE=MyDatabase;UID=MyUsername;PWD=mypassword;port=1234;'
'TDS_Version=8.0;')))
conn = engine.connect()
print conn
</pre>
<p>sqlalchemy - this works also</p>
<pre class="python">
import pyodbc
import sqlalchemy
def connect():
return pyodbc.connect(
'DRIVER={FreeTDS};SERVER=dev01.myserver.com;'
'DATABASE=MyDatabase;UID=MyUsername;PWD=mypassword;port=1234;'
'TDS_Version=8.0;')
engine = sqlalchemy.create_engine('mssql://', creator=connect)
conn = engine.connect()
print conn
</pre>
<p>You can also connect by specifying the path to the driver directly and bypass /etc/odbcinst.ini.
Not sure of the disadvantages of doing this.<p>
<pre class="python">
SQL_SERVER_HOST = '10.5.18.11'
SQL_SERVER_UID = 'myusername'
SQL_SERVER_PWD = 'mypassword'
FREETDS_DRIVER = '/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so'
before_quoted = ';'.join(
[
'DRIVER={}'.format(FREETDS_DRIVER),
'SERVER={}'.format(SQL_SERVER_HOST),
'DATABASE=mydatabase',
'UID={}'.format(SQL_SERVER_UID),
'PWD={}'.format(SQL_SERVER_PWD),
'PORT=1433',
'TDS_VERSION=7.2'
]
)
print before_quoted + '\n'
connection_string = 'mssql+pyodbc:///?odbc_connect={}'.format(
urllib.quote_plus(before_quoted))
print connection_string + '\n'
engine = sqlalchemy.create_engine(connection_string)
print engine
</pre>
<pre>DRIVER=/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so;SERVER=10.5.18.11;DATABASE=mydatabase;UID=myusername;PWD=mypassword;PORT=1433;TDS_VERSION=7.2
mssql+pyodbc:///?odbc_connect=DRIVER%3D%2Fusr%2Flib%2Fx86_64-linux-gnu%2Fodbc%2Flibtdsodbc.so%3BSERVER%3D10.5.18.11%3BDATABASE%3Dmydatabase%3BUID%3Dmyusername%3BPWD%3Dmypassword%3BPORT%3D1433%3BTDS_VERSION%3D7.2
Engine(mssql+pyodbc:///?odbc_connect=DRIVER=/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so;SERVER=10.5.18.11;DATABASE=mydatabase;UID=myusername;PWD=mypassword;PORT=1433;TDS_VERSION=7.2)
</pre>
<h5>References</h5>
<ul>
<li><a href="http://www.pauldeden.com/2008/12/how-to-setup-pyodbc-to-connect-to-mssql.html">
http://www.pauldeden.com/2008/12/how-to-setup-pyodbc-to-connect-to-mssql.html
</a></li>
<li><a href="http://www.sqlalchemy.org/docs/dialects/mssql.html#id1">
http://www.sqlalchemy.org/docs/dialects/mssql.html#id1
</a></li>
<li><a href="http://kipb7.wordpress.com/2008/06/12/pyodbc-unixodbc-freetds-config/">
http://kipb7.wordpress.com/2008/06/12/pyodbc-unixodbc-freetds-config/
</a></li>
<li><a href="http://stackoverflow.com/questions/4493614/sqlalchemy-equivalent-of-pyodbc-connect-string-using-freetds">
http://stackoverflow.com/questions/4493614/sqlalchemy-equivalent-of-pyodbc-connect-string-using-freetds
</a></li>
</ul>
<h4 id="unicode-error">Error inserting unicode</h4>
<p>If you get this "Error: The incoming tabular data stream (TDS)
remote procedure call (RPC) protocol stream is incorrect." error during insert, it
may be due to unicode in your data.</p>
<pre>Traceback (most recent call last):
File "tmp.py", line 10, in <module>
(unicode(u'some unicode data: \xa1').encode("utf-8"), '123', '123456'),
pyodbc.ProgrammingError: ('42000', '[42000] [FreeTDS][SQL Server]The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Parameter 3: The parameter name is invalid. (8005) (SQLExecDirectW)')</pre>
<p>The fix for me was to change the TDS version number. "7.0" and "8.0" did not work. "7.1", "7.2", "9.0", and some others worked as well.
There is
<a href="http://www.freetds.org/userguide/tdshistory.htm">this FreeTDS user guide page on version numbers</a>,
and <a href="http://www.freetds.org/userguide/choosingtdsprotocol.htm">also this other user guide page</a>,
but I'm not sure I'm clear on it. The following works for me:
</p>
<pre class="python">import pyodbc
conn = pyodbc.connect(
'DRIVER={FreeTDS};SERVER=dev01.myserver.com;'
'DATABASE=MyDatabase;UID=MyUsername;PWD=mypassword;port=1234;'
'TDS_Version=7.2;')
cursor = conn.cursor()
cursor.execute(
"""exec MyStoredProc @Param1 = ?, @Param2 = ?, @Param3 = ?""",
(unicode(u'some unicode data: \xa1').encode("utf-8"), '123', '123456'),
)
conn.commit()</pre>
<h5><code>Unrecognized server version info</code> warning</h5>
<p>When I changed <code>TDS_Version</code> to 7.2, I got the following warning:</p>
<pre>/myvirtualenv/lib/python2.6/site-packages/sqlalchemy/connectors/pyodbc.py:119: SAWarning: Unrecognized server version info '95.10.13055'. Version specific behaviors may not function properly. If using ODBC with FreeTDS, ensure server version 7.0 or 8.0, not 4.2, is configured in the FreeTDS configuration.</pre>
<p>This bothered me for a while, but later, when I
<a href="/blog/2011/09/notes-sqlalchemy-w-pyodbc-freetds-ubuntu/#pyodbc-freetds-truncates-varchar-instead-of-error">
upgraded freetds to 0.91</a>, this warning went away.</p>
<h5>See also these pages for more on unicode and pyodbc:</h5>
<ul>
<li><a href="http://stackoverflow.com/questions/947077/using-pyodbc-on-linux-to-insert-unicode-or-utf-8-chars-in-a-nvarchar-mssql-field">
http://stackoverflow.com/questions/947077/using-pyodbc-on-linux-to-insert-unicode-or-utf-8-chars-in-a-nvarchar-mssql-field
</a></li>
<li><a href="http://stackoverflow.com/questions/2192982/django-sql-server-2005-text-encoding-problem">
http://stackoverflow.com/questions/2192982/django-sql-server-2005-text-encoding-problem
</a></li>
</ul>
<h4 id="select-minus-1-bug">Select -1 bug</h4>
<p>Lin alerted me to this bug in pyodbc 2.1.11:
<a href="http://code.google.com/p/pyodbc/issues/detail?id=157">Issue 157: Negative integers are retrieved as positive integers</a>
Executing a <code>SELECT -1</code> returns 2^32-1 (4294967295).
Solution is to apply
<a href="http://code.google.com/p/pyodbc/issues/attachmentText?id=157&aid=1570005000&name=p&token=c5ff21e481c7cdf536d8e138dbab03be">
this patch</a> or upgrade to 3.0.x of pyodbc. I had trouble installing some versions of
pyodbc. This is what worked for me:
<pre class="console">$ pip install --upgrade -e git+https://code.google.com/p/pyodbc#egg=pyodbc </pre>
This gave me:
<pre class="console">$ pip freeze | grep pyodbc
-e git+https://code.google.com/p/pyodbc@404a3ba414ab15863c34db9c49ff0a02caa63600#egg=pyodbc-dev </pre>
<pre>$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> pyodbc.version
'3.0.4-beta01'</pre>
</p>
<h4 id="list-tables">How to list the tables in the database</h4>
<pre class="python">import urllib
import sqlalchemy
# Recommended to be a module-level variable
# http://www.sqlalchemy.org/docs/core/connections.html
engine = sqlalchemy.create_engine(
'mssql+pyodbc:///?odbc_connect=%s' % (
urllib.quote_plus(
'DRIVER={FreeTDS};SERVER=myserver.mydomain.com;'
'DATABASE=MyDatabase;UID=myusername;PWD=mypassword;port=12345;'
'TDS_Version=7.2;')))
metadata = sqlalchemy.MetaData()
metadata.bind = engine
metadata.reflect()
for t in metadata.sorted_tables:
print t.name</pre>
<h4 id="pyodbc-freetds-truncates-varchar-instead-of-error">pyodbc/FreeTDS truncates a varchar(max) upon insert instead of raising an error</h4>
<br><p>When trying to insert a string with more than 100 characters into a varchar(100) field, pyodbc/FreeTDS truncated the string instead of raising an error.
This occurred because I was using FreeTDS version 0.82. I found this out from
<a href="http://comments.gmane.org/gmane.comp.db.tds.freetds/13751">this FreeTDS mailing list thread</a>.
The solution is to upgrade to FreeTDS version 0.91. I could not find any APT packages for Ubuntu 10.10.
Looks like there is a package for Ubuntu Precise (12.04). So I compiled it from source.
This assumes you've installed unixodbc as described above:
<code>$ sudo apt-get install unixodbc unixodbc-dev </code>
</p>
<pre class="console">$ cd /tmp
$ wget http://ibiblio.org/pub/Linux/ALPHA/freetds/stable/freetds-stable.tgz
$ tar zxvf freetds-stable.tgz
$ cd freetds-0.91
$ ./configure
$ make
$ sudo make install </pre>
<p>Then edit odbcinst.ini to point at the new library /usr/local/lib/libtdsodbc.so:</p>
<pre>[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/local/lib/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsS.so
CPTimeout =
CPReuse =
TDS Version = 7.2
client charset = utf-8</pre>
<p>After upgrading, pyodbc now gives me the following error instead of failing silently.</p>
<pre>pyodbc.DataError: ('22001', '[22001] [FreeTDS][SQL Server]String or binary data would be truncated. (8152) (SQLExecDirectW)')</pre>
<p>References:</p>
<ul>
<li><a href="http://www.unixodbc.org/doc/FreeTDS.html">http://www.unixodbc.org/doc/FreeTDS.html
</a></li>
<li><a href="http://serverfault.com/questions/312895/compiling-freetds-0-91-on-ubuntu-11-04-x64">
http://serverfault.com/questions/312895/compiling-freetds-0-91-on-ubuntu-11-04-x64
</a></li>
<li><a href="http://www.freetds.org/userguide/config.htm#EVERYONE">
http://www.freetds.org/userguide/config.htm#EVERYONE
</a></li>
</ul>
<h4 id="segfault">Segfault error when using FreeTDS 0.91 and pyodbc 2.1.11</h4>
<p>Got this error trying to connect to a database on production (Scientific Linux 6)</p>
<pre>*** glibc detected *** python: free(): invalid next size (fast): 0x00000000019ed440 ***</pre>
<p>Solution was to upgrade pyodbc to 3.0.5 because there was a bug.</p>
<ul>
<li><a href="http://code.google.com/p/pyodbc/issues/detail?id=170">
http://code.google.com/p/pyodbc/issues/detail?id=170
</a></li>
<li><a href="http://code.google.com/p/pyodbc/issues/detail?id=212">
http://code.google.com/p/pyodbc/issues/detail?id=212
</a></li>
</ul>
<h4 id="sql-server-equivalent-of-mysqls-limit">What is the MS SQL server equivalent of MySQL's LIMIT in a SELECT</h4>
<p>TOP
See <a href="http://www.w3schools.com/sql/sql_top.asp">http://www.w3schools.com/sql/sql_top.asp</a>
</p>
<pre>SELECT TOP * FROM mytable</pre>
<h4 id="versions">How to get the versions of stuff</h4>
<ul>
<li>unixODBC
<pre>$ odbcinst --version
unixODBC 2.3.0</pre>
or
<pre>$ odbc_config --version
2.3.0</pre>
</li>
<li>freetds
<pre>$ tsql -C
Compile-time settings (established with the "configure" script)
Version: freetds v0.91
freetds.conf directory: /etc
MS db-lib source compatibility: yes
Sybase binary compatibility: yes
Thread safety: yes
iconv library: yes
TDS version: 4.2
iODBC: no
unixodbc: yes
SSPI "trusted" logins: no
Kerberos: no</pre>
</li>
<li>pyodbc
<pre>$ python
Python 2.7.0+ (r27:82500, Sep 15 2010, 18:14:55)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> pyodbc.version
'3.0.5'</pre>
</li>
</ul>
<h4 id="microsoft-driver">Trying out the Microsoft ODBC driver for Linux</h4>
<p>This was installed on Scientific Linux 6. Ubuntu is not supported.</p>
<ul>
<li><a href="http://blogs.technet.com/b/dataplatforminsider/archive/2011/11/28/available-today-preview-release-of-the-sql-server-odbc-driver-for-linux.aspx">
http://blogs.technet.com/b/dataplatforminsider/archive/2011/11/28/available-today-preview-release-of-the-sql-server-odbc-driver-for-linux.aspx
</a></li>
<li><a href="http://www.microsoft.com/download/en/details.aspx?id=28160">
http://www.microsoft.com/download/en/details.aspx?id=28160
</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/hh568455%28v=sql.110%29.aspx">
http://msdn.microsoft.com/en-us/library/hh568455%28v=sql.110%29.aspx
</a></li>
<li><a href="http://www.connectionstrings.com/sql-server-2008">
http://www.connectionstrings.com/sql-server-2008
</a></li>
</ul>
<p>Install make</p>
<pre>$ yum install make </pre>
<p>Followed instructions to install the unixODBC driver manager and the driver.
It installed OK.</p>
<pre class="console">
$ sudo su root
$ tar xvf sqlncli-11.0.1720.0.tar.gz
$ cd sqlncli-11.0.1720.0
$ ./build_dm.sh --help
$ ./build_dm.sh --download-url=file://../unixODBC-2.3.0.tar.gz
$ cd /tmp/unixODBC.8922.6568.26033/unixODBC-2.3.0; make install
$ sudo su root
$ tar xvf sqlncli-11.0.1720.0.tar.gz
$ cd sqlncli-11.0.1720.0
$ ./install.sh --help
$ cp /etc/odbcinst.ini /etc/odbcinst.ini.bak
$ ./install.sh verify
Microsoft SQL Server ODBC Driver V1.0 for Linux Installation Script
Copyright Microsoft Corp.
Starting install for Microsoft SQL Server ODBC Driver V1.0 for Linux
Checking for 64 bit Linux compatible OS ..................................... OK
Checking required libs are installed ........................................ OK
unixODBC utilities (odbc_config and odbcinst) installed ..................... OK
unixODBC Driver Manager version 2.3.0 or later installed .................... OK
unixODBC Driver Manager configuration correct .............................. OK*
Microsoft SQL Server ODBC Driver V1.0 for Linux already installed .... NOT FOUND
$ ./install.sh install
...
Enter YES to accept the license or anything else to terminate the installation: YES
Checking for 64 bit Linux compatible OS ..................................... OK
Checking required libs are installed ........................................ OK
unixODBC utilities (odbc_config and odbcinst) installed ..................... OK
unixODBC Driver Manager version 2.3.0 or later installed .................... OK
unixODBC Driver Manager configuration correct .............................. OK*
Microsoft SQL Server ODBC Driver V1.0 for Linux already installed .... NOT FOUND
Microsoft SQL Server ODBC Driver V1.0 for Linux files copied ................ OK
Symbolic links for bcp and sqlcmd created ................................... OK
Microsoft SQL Server ODBC Driver V1.0 for Linux registered ........... INSTALLED
Install log created at /tmp/sqlncli.31091.8029.6563/install.log.
One or more steps may have an *. See README for more information regarding
these steps.
$ odbcinst -q -d -n "SQL Server Native Client 11.0"
[SQL Server Native Client 11.0]
Description=Microsoft SQL Server ODBC Driver V1.0 for Linux
Driver=/opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1720.0
UsageCount=1
</pre>
<p>Connection string: Driver is "SQL Server Native Client 11.0".
Port is specified with a comma after the host. Thanks to
<a href="http://www.connectionstrings.com/sql-server-2008">
http://www.connectionstrings.com/sql-server-2008
</a></p>
<pre> connection = pyodbc.connect(
';'.join([
'DRIVER={SQL Server Native Client 11.0}',
'SERVER=%s,%s' % (db_profile['host'], db_profile['port']),
'DATABASE=%s' % db_profile['name'],
'UID=%s' % db_profile['user'],
'PWD=%s' % db_profile['password'],
]) + ';')</pre>
<p>Ran my app and got the following error message:</p>
<pre>Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1720.0' : file not found (0) (SQLDriverConnect)")</pre>
<p>That's a dumb error message.
<code>/opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1720.0</code> existed and it had the correct
permissions. Turns out it was missing dependencies. Google help:
<a href="http://mailman.unixodbc.org/pipermail/unixodbc-support/2011-July/002902.html">
http://mailman.unixodbc.org/pipermail/unixodbc-support/2011-July/002902.html
</a> and
<a href="http://www.linuxquestions.org/questions/fedora-35/how-to-install-libcrypto-so-4-a-594511/">
http://www.linuxquestions.org/questions/fedora-35/how-to-install-libcrypto-so-4-a-594511/
</a>.
</p>
<pre>$ ldd /opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1720.0
linux-vdso.so.1 => (0x00007fff5258d000)
libcrypto.so.6 => not found
libdl.so.2 => /lib64/libdl.so.2 (0x00007feec91fd000)
libodbc.so.1 => /usr/lib64/libodbc.so.1 (0x00007feec8f90000)
librt.so.1 => /lib64/librt.so.1 (0x00007feec8d88000)
libssl.so.6 => not found
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007feec8b83000)
libodbcinst.so.1 => /usr/lib64/libodbcinst.so.1 (0x00007feec896c000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007feec868c000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007feec844b000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007feec8144000)
libm.so.6 => /lib64/libm.so.6 (0x00007feec7ebf000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007feec7ca9000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007feec7a8d000)
libc.so.6 => /lib64/libc.so.6 (0x00007feec76eb000)
/lib64/ld-linux-x86-64.so.2 (0x00007feec97b0000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007feec74bf000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007feec72bc000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007feec70b1000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007feec6eaf000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007feec6c95000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007feec6a75000)</pre>
<p>libcrypto.so.6 and libssl.so.6 are missing.</p>
<pre>$ yum install openssl098e
$ ldd /opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1720.0
linux-vdso.so.1 => (0x00007fff5c7fe000)
libcrypto.so.6 => /usr/lib64/libcrypto.so.6 (0x00007f7ebcdd8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f7ebcbd4000)
libodbc.so.1 => /usr/lib64/libodbc.so.1 (0x00007f7ebc966000)
librt.so.1 => /lib64/librt.so.1 (0x00007f7ebc75e000)
libssl.so.6 => /usr/lib64/libssl.so.6 (0x00007f7ebc50f000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f7ebc30a000)
libodbcinst.so.1 => /usr/lib64/libodbcinst.so.1 (0x00007f7ebc0f3000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f7ebbe14000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f7ebbbd1000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7ebb8ca000)
libm.so.6 => /lib64/libm.so.6 (0x00007f7ebb646000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7ebb42f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7ebb213000)
libc.so.6 => /lib64/libc.so.6 (0x00007f7ebae72000)
libz.so.1 => /lib64/libz.so.1 (0x00007f7ebac5c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7ebd4e3000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f7ebaa59000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f7eba82d000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f7eba621000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f7eba41f000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f7eba205000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7eb9fe5000)</pre>
<p>Good. It works.</p>
<h4 id="install-mxodbc">Install mxodbc</h4>
<pre class="console">$ python -c 'print("UCS%s"%len(u"x".encode("unicode-internal")))'
UCS4
$ pip install http://downloads.egenix.com/python/egenix-mx-base-3.2.3.linux-x86_64-py2.7_ucs4.prebuilt.zip
$ pip install http://downloads.egenix.com/python/egenix-mxodbc-3.1.1.linux-x86_64-py2.7_ucs4.prebuilt.zip
$ #copy license files to "site-packages" directory
</pre>
<h4 id="unixodbc-gui-tools">unixODBC GUI tools</h4>
<p>I didn't end up actually using this. I think this is useful if you need to set up DSNs (e.g. if you're using mxodbc).</p>
<pre class="console">$ sudo apt-get install unixodbc-bin </pre>
<p>Description: Graphical tools for ODBC management and browsing
This package contains three graphical applications for use with
unixODBC, the Open DataBase Connectivity suite: ODBCConfig, a graphical
configuration tool for managing database drivers and access to
individual databases; DataManager, a simple browser and query tool for
ODBC databases; and odbctest, a tool for testing the ODBC API itself.
</p>
<p>Provides:</p>
<pre>/usr/bin/DataManager
/usr/bin/DataManagerII
/usr/bin/ODBCConfig
/usr/bin/odbctest</pre>
<h4 id="sql-developer-ubuntu">Installing SQL Developer on Ubuntu</h4>
<p>For a GUI interface to MS SQL Server, I like Oracle's SQL Developer better than Eclipse. Thanks Kris. (Too bad it doesn't execute stored procedures.)</p>
<pre>
- $ sudo apt-get install openjdk-6-jdk
- Create an Oracle account
- Download "Oracle SQL Developer for other platforms" from http://www.oracle.com/technetwork/developer-tools/sql-developer/downloads/index.html
- $ unzip sqldeveloper-3.1.07.42-no-jre.zip
- $ cd sqldeveloper
- $ sh sqldeveloper.sh
Type the full pathname of a J2SE installation (or Ctrl-C to quit), the path will be stored in ~/.sqldeveloper/jdk
usr/lib/jvm/java-6-openjdk
http://djiang.wordpress.com/2007/02/27/using-oracle-sqldeveloper-to-access-sqlserver/
Download jTDS from here: http://sourceforge.net/projects/jtds/
mkdir jtds-1.2.5
mv jtds-1.2.5-dist.zip jtds-1.2.5
cd jtds-1.2.5
unzip jtds-1.2.5-dist.zip
- Start Oracle’s SQLDeveloper, Tools->Preferences->Database->Third Party JDBC Drivers. Click “Add Entry” and point to the jtds-1.2.jar
- File -> New... -> Database Connection -> choose SQLServer tab, type in hostname, port, username and password. It appears that the initial connection name has to be the same as the database and you can click the “Retrieve database” button. Once you found the database, you can rename the connection.
</pre>
<h4 id="calling-sprocs">Calling stored procs from sqlalchemy and pyodbc</h4>
<pre class="python">
connection_string = ';'.join([
'DRIVER={FreeTDS}',
'SERVER=myhost.com',
'DATABASE=my_database',
'UID=myusername',
'PWD=mypassword',
'port=1433',
'TDS_Version=7.2',
]) + ';'
print connection_string
def use_pyodbc():
import pyodbc
connection = pyodbc.connect(connection_string)
print connection
sql = " ".join([
"EXEC [dbo].my_sproc",
"@my_param = ?",
])
print sql
cursor = connection.cursor()
result = cursor.execute(sql, ('my_value',))
for r in result.fetchall():
print r
print r.a_param
connection.commit()
def use_sqlalchemy():
import urllib
import sqlalchemy
engine = sqlalchemy.create_engine(
'mssql+pyodbc:///?odbc_connect=%s' % (
urllib.quote_plus(connection_string)))
print engine
sql = sqlalchemy.sql.text(" ".join([
"EXEC [dbo].my_sproc",
"@my_param = :my_param",
]))
print sql
result = engine.execute(sql, {'my_param': 'my_value'})
for r in result.fetchall():
for k in r.keys():
try:
print '%50s %s' % (k, getattr(r, k))
except sqlalchemy.exc.InvalidRequestError as e:
print e
break
</pre>
<h4 id="install-pyodbc-sl6">Install pyodbc on Scientific Linux 6</h4>
<p>you will also need freetds installed</p>
<pre class="console">$ yum install unixODBC-devel
$ yum install gcc-c++
$ pip install pyodbc </pre>
<h4 id="errors">Other misc errors</h4>
<ul>
<li>
<pre>ProgrammingError: (ProgrammingError) ('42000', '[42000] [FreeTDS][SQL Server]Statement(s) could not be prepared. (8180) (SQLExecDirectW)') </pre>
Problem was missing commas.
</li>
<li>
<pre>DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)') None None</pre>
<ul>
<li>This occurred when I could not connect to the database server. To troubleshoot this, see:
<a href="http://www.freetds.org/userguide/serverthere.htm">FreeTDS User Guide: Chapter 8. Troubleshooting: Is the server there?</a>
</li>
<li>This also occurred when I did not configure <code>/etc/odbcinst.ini</code> correctly
as described in the <a href="#install">Install section</a>
</li>
</ul>
</li>
</ul>
<h4 id="see-also">See also</h4>
<ul>
<li><a href="http://www.easysoft.com/developer/interfaces/odbc/linux.html">
Easysoft Linux ODBC documentation
</a></li>
</ul>
<h4 id="location-odbcinst-file">How to find the location of the odbcinst.ini file</h4>
<pre class="console">$ odbcinst -j
unixODBC 2.3.0
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /home/eliot/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8</pre>
<h4 id="trace-odbc">Trace odbc</h4>
<p><a href="http://www.easysoft.com/support/kb/kb00945.html">http://www.easysoft.com/support/kb/kb00945.html</a>
Add the following to the odbcinst.ini file. Be sure to turn it off after debugging.</p>
<pre>[ODBC]
Trace = yes
TraceFile = /tmp/my_odbc_tracfile.log</pre>
Notes on debugging ssh connection problems
2011-08-31T16:12:43-07:00https://www.saltycrane.com/blog/2011/08/notes-debugging-ssh-connection-problems/<ul>
<li>Run the ssh client in verbose mode
<pre class="console">$ ssh -vvv user@host </pre>
</li>
<li>On the server, check auth.log for errors
<pre class="console">$ sudo tail -f /var/log/auth.log </pre>
<p>On Red Hat, it's <code>/var/log/secure</code></p>
</li>
<li>For more debugging info, (assuming you have control of the ssh server)
run the sshd server in debug mode on another port
<pre class="console">$ sudo /usr/sbin/sshd -ddd -p 33333 </pre>
Then specify the port, <code>-p 33333</code> with the ssh client. e.g.
<pre class="console">$ ssh -vvv -p 33333 user@host </pre>
</li>
</ul>
<p>Commands run on Ubuntu 10.04</p>
<h4 id="message-too-long">sftp error: <code>Received message too long 170160758</code></h4>
<p>Problem was in the .bashrc. See <a href="http://www.snailbook.com/faq/sftp-corruption.auto.html">http://www.snailbook.com/faq/sftp-corruption.auto.html</a></p>
How to install Apache Thrift on Ubuntu <del>10.10 Maverick 12.04 Precise</del> 14.04 Trusty
2011-06-09T09:34:45-07:00https://www.saltycrane.com/blog/2011/06/install-thrift-ubuntu-1010-maverick/<p><em>Update 2014-10-06:</em> Here's what I did to install on Ubuntu 14.04 Trusty.</p>
<pre class="console">$ # Install your choice of java
$ sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
$ cd /tmp
$ curl http://archive.apache.org/dist/thrift/0.9.1/thrift-0.9.1.tar.gz | tar zx
$ cd thrift-0.9.1/
$ ./configure
$ make
$ sudo make install
$ thrift --help </pre>
<p>References:</p>
<ul>
<li><a href="https://thrift.apache.org/docs/install/debian">Apache Thrift Docs: Debian or Ubuntu setup</a></li>
<li><a href="https://thrift.apache.org/docs/BuildingFromSource">Apache Thrift Docs: Building from source</a></li>
</ul>
<p><em>Update 2013-02-20:</em> Here's what I did to install on Ubuntu 12.04.</p>
<pre class="console">$ sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
$ cd /tmp
$ curl https://dist.apache.org/repos/dist/release/thrift/0.9.0/thrift-0.9.0.tar.gz | tar zx
$ cd thrift-0.9.0/
$ ./configure
$ make
$ sudo make install
$ thrift --help </pre>
<p>References:</p>
<ul>
<li><a href="http://thrift.apache.org/docs/install/ubuntu/">Apache Thrift Docs: Debian or Ubuntu setup</a></li>
<li><a href="http://thrift.apache.org/docs/BuildingFromSource/">Apache Thrift Docs: Building from source</a></li>
</ul>
<h4>Old information</h4>
<p>This is the post that showed me how to get all the dependencies I needed:
<a href="http://blog.birchroad.net/2011/03/15/installing-apache-thrift-in-ubuntu/">http://blog.birchroad.net/2011/03/15/installing-apache-thrift-in-ubuntu/</a>
</p>
<p>Other links:</p>
<ul>
<li>
<a href="http://thrift.apache.org/">Apache Thrift homepage</a>
</li>
<li><a href="http://wiki.apache.org/thrift/ThriftInstallation">
Thrift Installation instructions
</a></li>
</ul>
<p>Install dependencies</p>
<pre class="console">$ sudo apt-get install libboost-dev libevent-dev libtool flex bison g++ automake pkg-config
$ sudo apt-get install libboost-test-dev
$ sudo apt-get install libmono-dev ruby1.8-dev libcommons-lang-java php5-dev </pre>
<p>Build thrift</p>
<pre class="console">$ cd /tmp
$ curl http://apache.deathculture.net//thrift/0.6.1/thrift-0.6.1.tar.gz | tar zx
$ cd thrift-0.6.1/
$ ./configure
$ make
$ sudo make install
$ which thrift </pre>
<p>Other possible dependencies:</p>
<p>Build-Depends: debhelper (>= 5), build-essential, mono-gmcs, python-dev, ant, libmono-dev, erlang-base, ruby1.8-dev, autoconf, python-support, automake, pkg-config, libtool, bison, flex, libboost-dev | libboost1.40-dev, python-all, python-all-dev, python-all-dbg, ruby, openjdk-6-jdk | java-sdk, libcommons-lang-java, libboost-test-dev | libboost-test1.40-dev, libevent-dev, perl (>= 5.8.0-7), php5, php5-dev</p>
This time I decided to build my Linux desktop PC myself
2011-02-21T21:44:57-08:00https://www.saltycrane.com/blog/2011/02/decided-build-linux-desktop-pc-myself/<p>Three years ago, <a href="/blog/2008/02/dell-530n-ubuntu-vs-build-it-yourself/">I decided to buy a Dell Ubuntu desktop PC</a> instead of building it myself. This time around, I went the other way. My main reasons were: better parts and easier upgrades. We'll see what I decide in 2014.
</p>
<table>
<tr>
<td>Motherboard</td>
<td>ASUS AM3 AMD 880G HDMI and USB 3.0 Micro ATX Motherboard M4A88T-M/USB3
<a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16813131660">newegg</a>
</td>
<td>Amazon</td>
<td>$99.99</td>
</tr>
<tr>
<td>CPU</td>
<td>AMD Athlon II X3 445 Rana 3.1GHz Socket AM3 95W Triple-Core Desktop Processor ADX445WFGMBOX</td>
<td>Newegg</td>
<td>$77.00</td>
</tr>
<tr>
<td>Power supply</td>
<td>OCZ Technology StealthXStream 2 500-Watt Power Supply</td>
<td>Amazon</td>
<td>$59.99</td>
</tr>
<tr>
<td>Case</td>
<td>Cooler Master Elite 360 RC-360-KKN1-GP ATX Mid Tower/Desktop Case (Black)</td>
<td>Amazon</td>
<td>$39.99</td>
</tr>
<tr>
<td>Hard drive</td>
<td>Seagate Barracuda 7200.12 ST3500418AS 500GB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive</td>
<td>Newegg</td>
<td>$39.99</td>
</tr>
<tr>
<td>RAM</td>
<td>
<del>Already own 3GB DDR2</del><br>
Crucial 4GB 240-PIN Dimm DDR3 - 1333 PC3-10600
</td>
<td>Amazon</td>
<td>$46.99</td>
</tr>
<tr>
<td>Optical Drive</td>
<td>
Asus DVD RW
</td>
<td>Fry's Electronics</td>
<td>~$25.00</td>
</tr>
<tr>
<td>Shipping</td>
<td colspan=2>
Free shipping on all items
</td>
<td>$0.00</td>
</tr>
<tr>
<td colspan=3>
Total</td>
<td>$388.95</td>
</tr>
</table>
<p>I used this guide: <a href="http://www.extremetech.com/article2/0,2845,2367455,00.asp">Build a Linux Media Center PC</a> <em> August 6, 2010</em></p>
<p>See also: <a href="http://www.extremetech.com/article2/0,2845,2366286,00.asp">8 Reasons to Build Your Own PC</a></p>
<a href="https://picasaweb.google.com/lh/photo/u1xg-jWJkINCi8Sdvk-P2g?feat=embedwebsite"><img src="https://lh5.googleusercontent.com/_WnP2PKiLI14/TXT2v7AJZnI/AAAAAAAAAXk/dwhB7maXKZk/s400/IMG_20110226_133344.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/iD_tZ0lx2WeQRYwW7Hfx5w?feat=embedwebsite"><img src="https://lh3.googleusercontent.com/_WnP2PKiLI14/TXUhManYtjI/AAAAAAAAAXk/HIk9EbIRYcs/s400/IMG_20110226_133029.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/h09eexN4VrCm7y6kmpeydw?feat=embedwebsite"><img src="https://lh4.googleusercontent.com/_WnP2PKiLI14/TXT2XMYnQNI/AAAAAAAAAXk/xm-g0zaLQao/s400/IMG_20110226_134315.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/HSbEhPtaA0NodAUIV7I_PA?feat=embedwebsite"><img src="https://lh5.googleusercontent.com/_WnP2PKiLI14/TXT1L2WBKVI/AAAAAAAAAXk/iZuYZ58cBVc/s400/IMG_20110226_135846.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/K4vCzsGVnTIjZCcXDCh6Bw?feat=embedwebsite"><img src="https://lh4.googleusercontent.com/_WnP2PKiLI14/TXT1dolRmVI/AAAAAAAAAXk/i0QEzAXCVRs/s400/IMG_20110226_135202.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/cNvsz37aOZRaUFc2Egks_A?feat=embedwebsite"><img src="https://lh6.googleusercontent.com/_WnP2PKiLI14/TXT0y1zWzzI/AAAAAAAAAXk/u2fOeQ7bSvw/s400/IMG_20110226_140401.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/udqvCq0gROlCw18lPIbSaw?feat=embedwebsite"><img src="https://lh6.googleusercontent.com/_WnP2PKiLI14/TXT1AYeQoKI/AAAAAAAAAXk/nBgNo0M94vA/s400/IMG_20110226_141136.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/IWY4oJ8w9YJb2lxkKWbVBg?feat=embedwebsite"><img src="https://lh4.googleusercontent.com/_WnP2PKiLI14/TXTzIJbnhUI/AAAAAAAAAXk/IrNZjNgIWu0/s400/IMG_20110226_143826.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/KKqs7ns6YE2vLwLp3qSPyg?feat=embedwebsite"><img src="https://lh3.googleusercontent.com/_WnP2PKiLI14/TXTyZe1VknI/AAAAAAAAAXk/-YvAP-cDR7w/s400/IMG_20110226_162949.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/6P-5t0VPGsauAHgCoZzh9w?feat=embedwebsite"><img src="https://lh3.googleusercontent.com/_WnP2PKiLI14/TXTxqvOwa_I/AAAAAAAAAXk/tyW2uvvnKoM/s400/IMG_20110305_211916.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/mMlPRjlV_Al5FzX_YakkNQ?feat=embedwebsite"><img src="https://lh5.googleusercontent.com/_WnP2PKiLI14/TXTxdpMqD8I/AAAAAAAAAXk/7RTfjw5z9kM/s400/IMG_20110305_212128.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/KfBa1JVlvkapfJjctFEq5w?feat=embedwebsite"><img src="https://lh3.googleusercontent.com/_WnP2PKiLI14/TXTxRASSceI/AAAAAAAAAXk/aax0D77QIVg/s400/IMG_20110305_213251.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/xZYF9o-8jRdEcZEv5cn6wA?feat=embedwebsite"><img src="https://lh4.googleusercontent.com/_WnP2PKiLI14/TXTxF_t0zWI/AAAAAAAAAXk/jVGYZC10zYA/s400/IMG_20110305_214907.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/wG5EcNrZT9kh_vAABFdTVg?feat=embedwebsite"><img src="https://lh6.googleusercontent.com/_WnP2PKiLI14/TXTw5dmcy5I/AAAAAAAAAXk/WyJQWvNhSbM/s400/IMG_20110305_214919.jpg" height="400" width="299" /></a>
<a href="https://picasaweb.google.com/lh/photo/SD_bOtsBSk8Lw0pKGzwmWQ?feat=embedwebsite"><img src="https://lh5.googleusercontent.com/_WnP2PKiLI14/TXTwsvs2LzI/AAAAAAAAAXk/-pDniB8nU8M/s400/IMG_20110305_215412.jpg" height="299" width="400" /></a>
<a href="https://picasaweb.google.com/lh/photo/zelUyqDTZeyHfj1b6dS3VA?feat=embedwebsite"><img src="https://lh5.googleusercontent.com/_WnP2PKiLI14/TXTwWGKcl4I/AAAAAAAAAXk/rYRbJEjsTpw/s400/IMG_20110307_061932.jpg" height="400" width="299" /></a>
<a href="https://picasaweb.google.com/lh/photo/uDfmQ_5fKxfrkLdOrISW5Q?feat=embedwebsite"><img src="https://lh4.googleusercontent.com/_WnP2PKiLI14/TXTwBjW52yI/AAAAAAAAAXk/JNyUzcIt_cI/s400/IMG_20110307_062535.jpg" height="299" width="400" /></a>
Monitoring a filesystem with Python and Pyinotify
2010-04-09T10:02:39-07:00https://www.saltycrane.com/blog/2010/04/monitoring-filesystem-python-and-pyinotify/<p><a href="http://trac.dbzteam.org/pyinotify">Pyinotify</a>
is a Python library for monitoring filesystem events
on Linux through the
<a href="http://en.wikipedia.org/wiki/Inotify">inotify</a>
Linux kernel subsystem.
It can monitor when a file is created, accessed, deleted,
modified, etc. For a full list of Pyinotify events see
<a href="http://trac.dbzteam.org/pyinotify/wiki/DocumentationPage">
the documentation</a>.
</p>
<h4 id="install">Install Pyinotify</h4>
<ul>
<li><a href="/blog/2010/02/how-install-pip-ubuntu/">Install pip</a></li>
<li>Install Pyinotify
<pre class="console">$ sudo pip install pyinotify</pre>
</li>
</ul>
<h4 id="example">Example</h4>
<pre class="python">import pyinotify
class MyEventHandler(pyinotify.ProcessEvent):
def process_IN_ACCESS(self, event):
print "ACCESS event:", event.pathname
def process_IN_ATTRIB(self, event):
print "ATTRIB event:", event.pathname
def process_IN_CLOSE_NOWRITE(self, event):
print "CLOSE_NOWRITE event:", event.pathname
def process_IN_CLOSE_WRITE(self, event):
print "CLOSE_WRITE event:", event.pathname
def process_IN_CREATE(self, event):
print "CREATE event:", event.pathname
def process_IN_DELETE(self, event):
print "DELETE event:", event.pathname
def process_IN_MODIFY(self, event):
print "MODIFY event:", event.pathname
def process_IN_OPEN(self, event):
print "OPEN event:", event.pathname
def main():
# watch manager
wm = pyinotify.WatchManager()
wm.add_watch('/var/log', pyinotify.ALL_EVENTS, rec=True)
# event handler
eh = MyEventHandler()
# notifier
notifier = pyinotify.Notifier(wm, eh)
notifier.loop()
if __name__ == '__main__':
main()</pre>
<p>Results:</p>
<pre>MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
OPEN event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-node.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
CLOSE_WRITE event: /var/log/munin/munin-update.log
OPEN event: /var/log/munin/munin-limits.log
MODIFY event: /var/log/munin/munin-limits.log
CLOSE_WRITE event: /var/log/munin/munin-limits.log
OPEN event: /var/log/munin/munin-graph.log
MODIFY event: /var/log/munin/munin-graph.log
CLOSE_WRITE event: /var/log/munin/munin-graph.log
OPEN event: /var/log/munin/munin-html.log
MODIFY event: /var/log/munin/munin-html.log
CLOSE_WRITE event: /var/log/munin/munin-html.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
...
</pre>
Notes on sshfs on Ubuntu
2010-04-05T22:08:09-07:00https://www.saltycrane.com/blog/2010/04/notes-sshfs-ubuntu/<p><a href="http://fuse.sourceforge.net/sshfs.html">sshfs</a>
is an easy way to mount a remote filesystem using
<a href="http://en.wikipedia.org/wiki/Secure_Shell">ssh</a>
and <a href="http://en.wikipedia.org/wiki/Filesystem_in_Userspace">FUSE</a>.
If your remote server is already running a ssh server that supports
<a href="http://en.wikipedia.org/wiki/SSH_file_transfer_protocol">sftp</a>
(Ubuntu's ssh server does),
there is nothing to set up on the remote server and set up on the
client is relatively easy.</p>
<p>Other options for mounting a remote filesystem are
<a href="http://www.webdav.org/">WebDAV</a>,
<a href="http://www.samba.org/">Samba</a>, and
<a href="http://en.wikipedia.org/wiki/Network_File_System_%28protocol%29">NFS</a>.
I'm no expert, but from what I've gathered, sshfs is faster than
WebDAV and slower than Samba and NFS. However, Samba and NFS are typically
more difficult to set up than sshfs. Here are my notes for
setting up sshfs. I am running on Ubuntu Hardy.
</p>
<h4 id="cmdline">OPTION 1: Use sshfs from the command line</h4>
<ul>
<li>Install sshfs
<pre class="console">$ apt-get update
$ apt-get install sshfs</pre>
</li>
<li>Create a mount point
<pre class="console">$ mkdir -p /var/www/remote_files</pre>
</li>
<li>Mount the remote filesystem
<pre class="console">$ sshfs root@10.232.139.234:/mnt/files /var/www/remote_files \
> -o IdentityFile=/path/to/my_ssh_keyfile \
> -o ServerAliveInterval=60 -o allow_other</pre>
where:
<ul>
<li><code>root</code> is the ssh username</li>
<li><code>10.232.139.234</code> is the remote host</li>
<li><code>/mnt/files</code> is the remote path</li>
<li><code>/var/www/remote_files</code> is the local path</li>
<li><code>/path/to/my_ssh_keyfile</code> is the ssh keyfile</li>
<li>The <code>ServerAliveInterval</code> option will keep your connection
from timing out.</li>
<li>The <code>allow_other</code> option allows other users to access
the filesystem</li>
</ul>
</li>
</ul>
<h4 id="fstab">OPTION 2: Use sshfs with /etc/fstab</h4>
<ul>
<li>Install sshfs as above</li>
<li>Edit /etc/fstab:
<pre>sshfs#root@10.232.139.234:/mnt/files /var/www/remote_files fuse allow_other,IdentityFile=/path/to/my_ssh_keyfile,ServerAliveInterval=60 0 0</pre>
where the options are explained above.
</li>
<li>Create a mount point
<pre class="console">$ mkdir -p /var/www/remote_files</pre>
</li>
<li>Mount
<pre class="console">$ mount /var/www/remote_files</pre>
</li>
</ul>
<h4 id="help">For more help, try <code>sshfs --help</code></h4>
<pre>usage: sshfs [user@]host:[dir] mountpoint [options]
general options:
-o opt,[opt...] mount options
-h --help print help
-V --version print version
SSHFS options:
-p PORT equivalent to '-o port=PORT'
-C equivalent to '-o compression=yes'
-1 equivalent to '-o ssh_protocol=1'
-o reconnect reconnect to server
-o sshfs_sync synchronous writes
-o no_readahead synchronous reads (no speculative readahead)
-o sshfs_debug print some debugging information
-o cache=YESNO enable caching {yes,no} (default: yes)
-o cache_timeout=N sets timeout for caches in seconds (default: 20)
-o cache_X_timeout=N sets timeout for {stat,dir,link} cache
-o workaround=LIST colon separated list of workarounds
none no workarounds enabled
all all workarounds enabled
[no]rename fix renaming to existing file (default: off)
[no]nodelay set nodelay tcp flag in ssh (default: on)
[no]nodelaysrv set nodelay tcp flag in sshd (default: off)
[no]truncate fix truncate for old servers (default: off)
[no]buflimit fix buffer fillup bug in server (default: on)
-o idmap=TYPE user/group ID mapping, possible types are:
none no translation of the ID space (default)
user only translate UID of connecting user
-o ssh_command=CMD execute CMD instead of 'ssh'
-o ssh_protocol=N ssh protocol to use (default: 2)
-o sftp_server=SERV path to sftp server or subsystem (default: sftp)
-o directport=PORT directly connect to PORT bypassing ssh
-o transform_symlinks transform absolute symlinks to relative
-o follow_symlinks follow symlinks on the server
-o no_check_root don't check for existence of 'dir' on server
-o SSHOPT=VAL ssh options (see man ssh_config)
FUSE options:
-d -o debug enable debug output (implies -f)
-f foreground operation
-s disable multi-threaded operation
-o allow_other allow access to other users
-o allow_root allow access to root
-o nonempty allow mounts over non-empty file/dir
-o default_permissions enable permission checking by kernel
-o fsname=NAME set filesystem name
-o subtype=NAME set filesystem type
-o large_read issue large read requests (2.4 only)
-o max_read=N set maximum size of read requests
-o hard_remove immediate removal (don't hide files)
-o use_ino let filesystem set inode numbers
-o readdir_ino try to fill in d_ino in readdir
-o direct_io use direct I/O
-o kernel_cache cache files in kernel
-o [no]auto_cache enable caching based on modification times
-o umask=M set file permissions (octal)
-o uid=N set file owner
-o gid=N set file group
-o entry_timeout=T cache timeout for names (1.0s)
-o negative_timeout=T cache timeout for deleted names (0.0s)
-o attr_timeout=T cache timeout for attributes (1.0s)
-o ac_attr_timeout=T auto cache timeout for attributes (attr_timeout)
-o intr allow requests to be interrupted
-o intr_signal=NUM signal to send on interrupt (10)
-o modules=M1[:M2...] names of modules to push onto filesystem stack
-o max_write=N set maximum size of write requests
-o max_readahead=N set maximum readahead
-o async_read perform reads asynchronously (default)
-o sync_read perform reads synchronously
Module options:
[subdir]
-o subdir=DIR prepend this directory to all paths (mandatory)
-o [no]rellinks transform absolute symlinks to relative
[iconv]
-o from_code=CHARSET original encoding of file names (default: UTF-8)
-o to_code=CHARSET new encoding of the file names (default: ANSI_X3.4-1968)</pre>
<h4 id="references">References</h4>
<ul>
<li><a href="http://sysblogd.wordpress.com/2007/08/23/ubuntu-mounting-remote-filesystem-using-sshfs-fuse/">
Ubuntu: Mounting remote filesystem using sshfs (FUSE)</a></li>
<li><a href="http://www.debuntu.org/2006/04/27/39-mounting-a-fuse-filesystem-form-etcfstab">
Mounting a fuse Filesystem from /etc/fstab</a></li>
<li><a href="https://help.ubuntu.com/community/SSHFS">
SSHFS - Community Ubuntu Documentation</a></li>
<li><a href="http://sourceforge.net/apps/mediawiki/fuse/index.php?title=SshfsFaq">
sshfs FAQ</a></li>
<li><a href="http://www.linuxforums.org/forum/misc/159035-sshfs-passwordless-login.html">
sshfs passwordless login - Linux Forums</a></li>
<li><a href="http://blog.brianhartsock.com/2007/02/18/sshfs/">
Brian Hartsock's Blog - SSHFS</a></li>
</ul>
<h4 id="webdav-vs-sshfs">Webdav vs. sshfs</h4>
<ul>
<li><a href="http://jamiedubs.com/macfuse-sshfs-vs-webdav">
MacFUSE sshfs vs WebDAV benchmarks / Jamie Wilkinson</a></li>
<li><a href="http://gioorgi.com/2009/webdav-versus-sshfs/">
WebDAV versus Sshfs | Gioorgi.com</a></li>
</ul>
Samba notes
2010-02-19T17:46:11-08:00https://www.saltycrane.com/blog/2010/02/samba-notes/<h4>Assumptions</h4>
<ul>
<li>Trying to access filesystem on Ubuntu Hardy Linux server from Windows Vista
PC over the internet. Want to have read and write access.</li>
<li>Ubuntu Linux server IP address is: 99.99.99.99</li>
<li>Windows PC IP address is: 44.44.44.44</li>
<li>Username is "johnny"</li>
</ul>
<h4>On Ubuntu Hardy host</h4>
<ul>
<li>Install Samba
<pre>apt-get install samba</pre>
</li>
<li><p>Edit <code>/etc/samba/smb.conf</code></p>
<p>In the <code>[globals]</code> section:
<pre> hosts allow = 127.0.0.1 44.44.44.44
hosts deny = 0.0.0.0/0</pre>
<p><a href="http://www.hackorama.com/network/samba.shtml">Reference</a></p>
<p>Create a new share:</p>
<pre>[myshare]
comment = this is my share
path = /var/www
read only = no
guest ok = no
delete readonly = yes
create mask = 0644
force create mode = 0600
security mask = 0555
force security mode = 0600</pre>
<p><a href="http://tortoisesvn.tigris.org/faq.html#samba">Reference</a></p>
</li>
<li>Create a user
<pre>adduser johnny
smbpasswd -a johnny</pre>
<p>Enter a password.</p>
</li>
<li>Open ports in EC2 Security Group using ElasticFox
<pre>Port 135/TCP - used by smbd
Port 137/UDP - used by nmbd
Port 138/UDP - used by nmbd
Port 139/TCP - used by smbd
Port 445/TCP - used by smbd</pre>
<p><a href="http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/securing-samba.html#firewallports">
Reference</a></p>
</li>
</ul>
<h4>On Windows host</h4>
<ul>
<li>In Windows Explorer, enter <code>\\99.99.99.99\myshare</code> into the location
bar.</li>
<li>Enter the "johnny" for the username and the password for the password.</li>
</ul>
Using Python to write to an Excel / OpenOffice Calc spreadsheet on Ubuntu Linux
2010-02-15T17:41:25-08:00https://www.saltycrane.com/blog/2010/02/using-python-write-excel-openoffice-calc-spreadsheet-ubuntu-linux/<p>Via <a href="http://panela.blog-city.com/pyexcelerator_xlwt_cheatsheet_create_native_excel_from_pu.htm">
Matt Harrison's blog post</a>, here is how to write Excel or OpenOffice.org Calc
spreadsheet files using Python and the <a href="http://pypi.python.org/pypi/xlwt">xlwt</a>
library. Xlwt is a fork of <a href="http://pypi.python.org/pypi/pyExcelerator/">
pyExcelerator</a> which handles only writing spreadsheet files. For reading
spreadsheets, see <a href="http://pypi.python.org/pypi/xlrd">xlrd</a>.
Note, these libraries don't use <a href="http://en.wikipedia.org/wiki/Component_Object_Model">
COM</a>, so they will work on non-Windows OSes, such as Linux. For more information,
see <a href="http://panela.blog-city.com/pyexcelerator_xlwt_cheatsheet_create_native_excel_from_pu.htm">
Matt's blog post</a>. He even has a <a href="http://files.blog-city.com/files/F05/96843/b/cheatsheet.pdf">
PDF cheat sheet</a>.
</p>
<ul>
<li><a href="http://www.saltycrane.com/blog/2010/02/how-install-pip-ubuntu/">
Install pip</a></li>
<li>Install xlwt
<pre>sudo pip install xlwt</pre></li>
<li>Create an example script:
<pre class="python">import xlwt
DATA = (("The Essential Calvin and Hobbes", 1988,),
("The Authoritative Calvin and Hobbes", 1990,),
("The Indispensable Calvin and Hobbes", 1992,),
("Attack of the Deranged Mutant Killer Monster Snow Goons", 1992,),
("The Days Are Just Packed", 1993,),
("Homicidal Psycho Jungle Cat", 1994,),
("There's Treasure Everywhere", 1996,),
("It's a Magical World", 1996,),)
wb = xlwt.Workbook()
ws = wb.add_sheet("My Sheet")
for i, row in enumerate(DATA):
for j, col in enumerate(row):
ws.write(i, j, col)
ws.col(0).width = 256 * max([len(row[0]) for row in DATA])
wb.save("myworkbook.xls")</pre>
</li>
<li>Results:
<a href="http://picasaweb.google.com/lh/photo/MFnV-ijtOBQmBywJWIbJVw?feat=embedwebsite"><img src="http://lh5.ggpht.com/_WnP2PKiLI14/S3nySGDHAwI/AAAAAAAAAEk/udiOfW38N-s/s800/python_oocalc_ex.png" /></a>
</li>
</ul>
How to install MySQLdb in a virtualenv on Ubuntu Karmic
2010-02-15T12:07:08-08:00https://www.saltycrane.com/blog/2010/02/install-mysqldb-virtualenv-ubuntu-karmic/<ul>
<li>Install dependencies
<pre>sudo apt-get build-dep python-mysqldb</pre>
</li>
<li>Install MySQLdb with pip/virtualenv
<pre>cd ~/lib/python-environments
virtualenv --distribute --no-site-packages default
pip install -E default/ MySQL-python</pre>
</li>
</ul>
<h4>EnvironmentError: mysql_config not found</h4>
<p>The first time, I did not install the dependencies and got the following error:</p>
<pre>$ pip install -E default/ http://sourceforge.net/projects/mysql-python/files/mysql-python-test/1.2.3c1/MySQL-python-1.2.3c1.tar.gz/download
Downloading/unpacking http://sourceforge.net/projects/mysql-python/files/mysql-python-test/1.2.3c1/MySQL-python-1.2.3c1.tar.gz/download
Downloading download (89Kb): 89Kb downloaded
Running setup.py egg_info for package from http://sourceforge.net/projects/mysql-python/files/mysql-python-test/1.2.3c1/MySQL-python-1.2.3c1.tar.gz/download
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/tmp/pip-ZR_ZwM-build/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
Complete output from command python setup.py egg_info:
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/tmp/pip-ZR_ZwM-build/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in ./pip-log.txt</pre>
How to install pip on Ubuntu
2010-02-15T11:42:52-08:00https://www.saltycrane.com/blog/2010/02/how-install-pip-ubuntu/<p>
<a href="http://pypi.python.org/pypi/pip">Pip</a> is a better alternative to
<a href="http://pypi.python.org/pypi/setuptools">Easy Install</a> for installing Python packages.
It is most "nutritious" when used with its companion
<a href="http://pypi.python.org/pypi/virtualenv">virtualenv</a>.
For more information on pip and virtualenv see my blog post:
<em><a href="http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/">
Notes on using pip and virtualenv with Django</a></em>.
</p>
<h4>Install pip and virtualenv for Ubuntu 10.10 Maverick and newer</h4>
<pre class="console">$ sudo apt-get install python-pip python-dev build-essential
$ sudo pip install --upgrade pip
$ sudo pip install --upgrade virtualenv </pre>
<h4>For older versions of Ubuntu</h4>
<ul>
<li>Install Easy Install
<pre class="console">$ sudo apt-get install python-setuptools python-dev build-essential </pre>
</li>
<li>Install pip
<pre class="console">$ sudo easy_install pip </pre>
</li>
<li>Install virtualenv
<pre class="console">$ sudo pip install --upgrade virtualenv </pre>
</li>
</ul>
A hack to copy files between two remote hosts using Python
2010-02-08T12:51:29-08:00https://www.saltycrane.com/blog/2010/02/hack-copy-files-between-two-remote-hosts-using-python/<p>I sometimes need to copy a file (such as a database dump) between two
remote hosts on EC2. Normally this involves a few steps: scp'ing the ssh keyfile
to Host 1, ssh'ing to Host 1, looking up the address for Host 2, then scp'ing the desired
file from Host 1 to Host 2.</p>
<p>I was excited to read in the man page that scp can copy files between two
remote hosts directly. However, it didn't work for me.
<a href="http://fixunix.com/ssh/258554-should-i-able-scp-between-two-remote-hosts.html#post700563">Apparently</a>,
running <code>scp host1:myfile host2:</code> is like running
<code>ssh host1 scp myfile host2:</code> so I still need the address of host2
and my ssh keyfile on host1.
</p>
<p>My inablility to let go of this small efficiency increaser, led me to
(what else?) write a Python script.
I know this is a hack so if you know of a better way of doing this, let
me know.
</p>
<p>The script parses my <code>~/.ssh/config</code> file to find
the ssh keyfile and address for host 2, uses scp to copy the ssh keyfile
to host 1, then runs the <code>ssh host1 scp ...</code> command with the
appropriate options filled in. The script captures all of the ssh
options for host 2 and passes them on the command line to <code>scp</code>
via the <code>-o</code> command-line option. Note, I only tested this
to set the <code>User</code> option– I don't know if all ssh options
will work.
</p>
<p><em>Warning: the script disables the StrictHostKeyChecking SSH option,
so you are more vunerable to a man-in-the-middle attack.</em></p>
<p><em>Update 2010-02-16:</em> I've found there is already a SSH config
file parser in the <a href="http://www.lag.net/paramiko/">paramiko</a>
library. The source can be viewed
<a href="http://github.com/robey/paramiko/blob/master/paramiko/config.py">
on github</a>.
</p>
<p><em>Update 2010-05-04:</em> I modified my code to use the paramiko
library and also allow command line options to be passed directly to
the scp command.
The latest code is available in my github repository
<a href="http://github.com/saltycrane/remote-tools">remote-tools</a>.
</p>
<pre class="python">import itertools
import os
import re
import sys
SSH_CONFIG_FILE = '/home/eliot/.ssh/config'
def main():
host1, path1 = sys.argv[1].split(':', 1)
host2, path2 = sys.argv[2].split(':', 1)
o = get_ssh_options(host2)
keyfile_remote = '/tmp/%s' % os.path.basename(o['identityfile'])
ssh_options = ' -o'.join(['='.join([k, v]) for k, v in o.iteritems()
if k != 'hostname' and k != 'identityfile'])
run('scp %s %s:%s' % (o['identityfile'], host1, keyfile_remote))
run('ssh %s scp -p -i %s -oStrictHostKeyChecking=no -o%s %s %s:%s' % (
host1, keyfile_remote, ssh_options, path1, o['hostname'], path2))
def get_ssh_options(host):
"""Parse ~/.ssh/config file and return a dict of ssh options for host
Note: dict keys are all lowercase
"""
def remove_comment(line):
return re.sub(r'#.*$', '', line)
def get_value(line, key_arg):
m = re.search(r'^\s*%s\s+(.+)\s*$' % key_arg, line, re.I)
if m:
return m.group(1)
else:
return ''
def not_the_host(line):
return get_value(line, 'Host') != host
def not_a_host(line):
return get_value(line, 'Host') == ''
lines = [line.strip() for line in file(SSH_CONFIG_FILE)]
comments_removed = [remove_comment(line) for line in lines]
blanks_removed = [line for line in comments_removed if line]
top_removed = list(itertools.dropwhile(not_the_host, blanks_removed))[1:]
goodpart = itertools.takewhile(not_a_host, top_removed)
return dict([line.lower().split(None, 1) for line in goodpart])
def run(cmd):
print cmd
os.system(cmd)
if __name__ == '__main__':
main()</pre>
<p>Here is an example <code>~/.ssh/config</code> file:</p>
<pre>Host testhost1
User root
Hostname 48.879.24.567
IdentityFile /home/eliot/.ssh/test_keyfile
Host testhost2
User root
Hostname 56.384.58.212
IdentityFile /home/eliot/.ssh/test_keyfile</pre>
<p>Here is an example run. It copies <code>/tmp/testfile</code> from
<code>testhost1</code> to the same path on <code>testhost2</code>.</p>
<pre>python scp_r2r.py testhost1:/tmp/testfile testhost2:/tmp/testfile</pre>
<p>Here is the console output:</p>
<pre style="overflow: auto">scp /home/eliot/.ssh/test_keyfile testhost1:/tmp/test_keyfile
test_keyfile 100% 1674 1.6KB/s 00:00
ssh testhost1 scp -p -i /tmp/test_keyfile -oStrictHostKeyChecking=no -ouser=root /tmp/testfile 56.384.58.212:/tmp/testfile</pre>
<p>One inconvenience is that it doesn't show the progress for the main transfer.
If anyone knows how I can fix this, please let me know.</p>
Iterating over lines in multiple Linux log files using Python
2010-01-23T00:07:51-08:00https://www.saltycrane.com/blog/2010/01/iterating-over-lines-multiple-linux-log-files-using-python/<p>I needed to parse through my Nginx log files to debug a problem. However, the logs
are separated into many files, most of them are gzipped, and I wanted the
ordering within the files reversed. So I abstracted the logic to handle this
into a function. Now I can pass a glob pattern such as <em>/var/log/nginx/cache.log*</em>
to my function,
and iterate over each line in all the files as if they were one file. Here is my
function. Let me know if there is a better way to do this.</p>
<p><em>Update 2010-02-24:</em>To handle multiple log files on a remote host, see
my <a href="http://github.com/saltycrane/remote-tools/blob/master/multilog.py">script
on github</a>.</p>
<pre class="python">import glob
import gzip
import re
def get_lines(log_glob):
"""Return an iterator of each line in all files matching log_glob.
Lines are sorted most recent first.
Files are sorted by the integer in the suffix of the log filename.
Suffix may be one of the following:
.X (where X is an integer)
.X.gz (where X is an integer)
If the filename does not end in either suffix, it is treated as if X=0
"""
def sort_by_suffix(a, b):
def get_suffix(fname):
m = re.search(r'.(?:\.(\d+))?(?:\.gz)?$', fname)
if m.lastindex:
suf = int(m.group(1))
else:
suf = 0
return suf
return get_suffix(a) - get_suffix(b)
filelist = glob.glob(log_glob)
for filename in sorted(filelist, sort_by_suffix):
if filename.endswith('.gz'):
fh = gzip.open(filename)
else:
fh = open(filename)
for line in reversed(fh.readlines()):
yield line
fh.close()</pre>
<p>Here is an example run on my machine. It prints the first 15 characters of
every 1000th line of all my syslog files.</p>
<pre class="python">for i, line in enumerate(get_lines('/var/log/syslog*')):
if not i % 1000:
print line[:15]</pre>
<p>File listing:</p>
<pre>$ ls -l /var/log/syslog*
-rw-r----- 1 syslog adm 169965 2010 01/23 00:18 /var/log/syslog
-rw-r----- 1 syslog adm 350334 2010 01/22 08:03 /var/log/syslog.1
-rw-r----- 1 syslog adm 18078 2010 01/21 07:49 /var/log/syslog.2.gz
-rw-r----- 1 syslog adm 16700 2010 01/20 07:43 /var/log/syslog.3.gz
-rw-r----- 1 syslog adm 18197 2010 01/19 07:52 /var/log/syslog.4.gz
-rw-r----- 1 syslog adm 15737 2010 01/18 07:45 /var/log/syslog.5.gz
-rw-r----- 1 syslog adm 16157 2010 01/17 07:54 /var/log/syslog.6.gz
-rw-r----- 1 syslog adm 20285 2010 01/16 07:48 /var/log/syslog.7.gz</pre>
<p>Results:</p>
<pre>Jan 22 23:57:01
Jan 22 14:09:01
Jan 22 03:51:01
Jan 21 17:35:01
Jan 21 14:37:33
Jan 21 08:35:01
Jan 20 22:12:01
Jan 20 11:56:01
Jan 20 01:41:01
Jan 19 15:18:01
Jan 19 04:53:01
Jan 18 18:35:01
Jan 18 08:40:01
Jan 17 22:10:01
Jan 17 11:32:01
Jan 17 01:05:01
Jan 16 14:27:01
Jan 16 04:01:01
Jan 15 17:25:01
Jan 15 08:50:01</pre>
"ImportError: No module named pstats" error on Ubuntu
2010-01-21T00:09:17-08:00https://www.saltycrane.com/blog/2010/01/importerror-no-module-named-pstats-error-ubuntu/<p>Even though the <code>pstats</code> module (used by <a href="http://docs.python.org/library/profile.html"><code>cProfile</code></a>) is part of the Python Standard Library, Ubuntu requires installing a separate package because of its non-free license. For more information, see this <a href="https://bugs.launchpad.net/ubuntu/+source/python-defaults/+bug/123755">Ubuntu bug report.</a> (thanks <a href="#c7226">Luke</a>)
</p>
<p>Running this on Ubuntu Karmic:</p>
<pre class="python">import cProfile
def my_super_slow_routine(): pass
cProfile.run('my_super_slow_routine()')</pre>
<p>Produces this error:</p>
<pre>Traceback (most recent call last):
File "test_pstats_error.py", line 3, in <module>
cProfile.run('my_super_slow_routine()')
File "/usr/lib/python2.6/cProfile.py", line 36, in run
result = prof.print_stats(sort)
File "/usr/lib/python2.6/cProfile.py", line 80, in print_stats
import pstats
ImportError: No module named pstats</pre>
<p>Solution:</p>
<pre>sudo apt-get install python-profiler</pre>
<p><em>Note: the multiverse repository must be enabled via either <code>update-manager</code> > "Settings..." > "Ubuntu Software" or directly editing <code>/etc/apt/sources.list</code>. (thanks <a href="#c7226">Luke</a>)</em></p>
Wmii Python script to monitor remote machines
2009-12-22T01:00:05-08:00https://www.saltycrane.com/blog/2009/12/wmii-python-script-monitor-remote-machines/<object width="425" height="344"><param name="movie" value="http://www.youtube.com/v/jSes6lJdj5Y&hl=en_US&fs=1&rel=0&color1=0x3a3a3a&color2=0x999999"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/jSes6lJdj5Y&hl=en_US&fs=1&rel=0&color1=0x3a3a3a&color2=0x999999" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>
<p>I like to monitor our web servers by ssh'ing into the remote machine and watching
"top", tailing log files, etc. Normally, I open a terminal, ssh into the
remote machine, run the monitoring command (e.g. "top"), then repeat for the rest
of the remote machines. Then I adjust the window sizes so I can see everything
at once.</p>
<p>My window manager, <a href="http://wmii.suckless.org/">wmii</a>, is great for
tiling a bunch of windows at once. It is also scriptable with Python, so I wrote a
Python script to create my web server monitoring view. Below is my script. I also put a
<a href="http://www.youtube.com/watch?v=jSes6lJdj5Y">video on YouTube</a>.
</p>
<pre class="python">#!/usr/bin/env python
import os
import time
NGINX_MONITOR_CMD = "tail --follow=name /var/log/nginx/cache.log | grep --color -E '(HIT|MISS|EXPIRED|STALE|UPDATING|\*\*\*)'"
APACHE_MONITOR_CMD = "top"
MYSQL_MONITOR_CMD = "mysqladmin extended -i10 -r | grep -i 'questions\|aborted_clients\|opened_tables\|slow_queries\|threads_created' "
CMDS_COL1 = ['urxvt -title "Nginx 1" -e ssh -t us-ng1 "%s" &' % NGINX_MONITOR_CMD,
'urxvt -title "Nginx 2" -e ssh -t us-ng2 "%s" &' % NGINX_MONITOR_CMD,
]
CMDS_COL2 = ['urxvt -title "Apache 1" -e ssh -t us-med1 "%s" &' % APACHE_MONITOR_CMD,
'urxvt -title "Apache 2" -e ssh -t us-med2 "%s" &' % APACHE_MONITOR_CMD,
'urxvt -title "Apache 3" -e ssh -t us-med3 "%s" &' % APACHE_MONITOR_CMD,
]
CMDS_COL3 = ['urxvt -title "MySQL 1" -e ssh -t us-my1 "%s" &' % MYSQL_MONITOR_CMD,
'urxvt -title "MySQL 2" -e ssh -t us-my2 "%s" &' % MYSQL_MONITOR_CMD,
]
COLUMNS = [CMDS_COL1, CMDS_COL2, CMDS_COL3]
def create_windows():
for i, col in enumerate(COLUMNS):
cindex = str(i+1)
for cmd in col:
os.system(cmd)
time.sleep(1)
os.system('wmiir xwrite /tag/sel/ctl send sel %s' % cindex)
os.system('wmiir xwrite /tag/sel/ctl colmode %s default-max' % cindex)
os.system('wmii.py 45.5 31.5 23')
if __name__ == '__main__':
create_windows()</pre>
<p><em>Note 1:</em> The script above uses another
<a href="http://www.saltycrane.com/blog/2009/04/scripting-wmii-column-widths-python/">
script I wrote previously, <code>wmii.py</code>,</a> to set the column widths.
</p>
<p><em>Note 2:</em> The remote server addresses are specified by the nicknames
<em>us-ng1</em>, <em>us-ng2</em>, <em>us-med1</em>, etc. configured in my
<code>~/.ssh/config</code> file as described
<a href="http://www.saltycrane.com/blog/2008/11/creating-remote-server-nicknames-sshconfig/">here</a>.
</p>
<p><em>Note 3 (on using ssh and top):</em> I first tried doing <code>ssh host top</code>,
but this gave me a <em><code>TERM environment variable not set.</code></em> error.
I then tried <code>ssh host "export TERM=rxvt-unicode; top"</code>, but this gave me
a <em><code>top: failed tty get</code></em> error. The solution that worked for me was
to use the <em><code>-t</code></em> option with <code>ssh</code>. E.g.
<code>ssh -t host top</code>. This is what I used in the script above.
</p>
<p><em>Note 4 (added 2010-03-05):</em> I used "tail --follow=name" instead of "tail -f"
so that tail will follow the log file even after it has been rotated. For more
information, see the
<a href="http://manpages.debian.net/cgi-bin/man.cgi?query=tail&apropos=0&sektion=0&manpath=Debian+5.0+lenny&format=html&locale=en">man
page for tail</a>.</p>
<p><em>Note 5 (added 2010-03-05):</em> To prevent your ssh session from timing out,
add the following 2 lines to your <code>~/.ssh/config</code> file
(<a href="http://ocaoimh.ie/how-to-fix-ssh-timeout-problems/">via</a>):</p>
<pre>Host *
ServerAliveInterval 60</pre>
Using Nginx as a caching proxy with Wordpress+Apache
2009-11-03T12:04:52-08:00https://www.saltycrane.com/blog/2009/11/using-nginx-caching-proxy-wordpressapache/<p>We have been evaluating caching reverse proxy servers at work. We looked at
<a href="http://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/">
Nginx+memcached</a>,
<a href="http://www.squid-cache.org/">Squid</a>, and
<a href="http://varnish.projects.linpro.no/">Varnish</a>. Most recently,
we found that <a href="http://wiki.nginx.org/Main">Nginx</a> version 0.7 has
<a href="http://www.ruby-forum.com/topic/183590">support
for caching static files</a> using the <code>proxy_cache</code> directive in
the <a href="http://wiki.nginx.org/NginxHttpProxyModule">NginxHttpProxyModule</a>.
This allows us to use Nginx as a caching proxy without having to handle the
complication (or flexibility depending on how you look at it) of setting and
invalidating the cache as with the Nginx+memcached setup.
Here are my notes for setting it up with an Apache+Wordpress backend.
</p>
<p><em>Update 2010-01-05:</em> Over a couple months, we switched to Nginx 0.8 and we made a few tweaks to our Nginx configuration. Here is our updated conf file: <a href="/site_media/code/nginx_wordpress_100105.conf.txt">nginx_wordpress_100105.conf</a>.
</p>
<h4 id="install">Install Nginx 0.7</h4>
<p>The version of Nginx in Ubuntu is an older version so we used a PPA
created by Jeff Waugh: <a href="https://launchpad.net/~jdub/+archive/ppa">
https://launchpad.net/~jdub/+archive/ppa</a>. (He also has a development
PPA which contains Nginx 0.8.)
</p>
<ul>
<li>Add the following to /etc/apt/sources.list:
<pre>deb http://ppa.launchpad.net/jdub/ppa/ubuntu hardy main
deb-src http://ppa.launchpad.net/jdub/ppa/ubuntu hardy main</pre>
</li>
<li>Tell Ubuntu how to authenticate the PPA
<pre>apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E9EEF4A1</pre>
<p>Alternively, if the keyserver is down, you can follow the instructions
for copying the public key
from <a href="http://forum.nginx.org/read.php?2,5177,11272">
http://forum.nginx.org/read.php?2,5177,11272</a>.
</p>
</li>
<li>Install Nginx from new PPA
<pre>apt-get update
apt-get install nginx</pre>
</li>
<li>Check the version of Nginx
<pre>nginx -V</pre>
<pre>nginx version: nginx/0.7.62
configure arguments: --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-debug --with-http_stub_status_module --with-http_flv_module --with-http_ssl_module --with-http_dav_module --with-http_gzip_static_module --with-ipv6 --with-http_realip_module --with-http_xslt_module --with-http_image_filter_module --with-sha1=/usr/include/openssl</pre>
</li>
</ul>
<h4 id="configure-logging">Configure Nginx cache logging</h4>
<p>Within the <code>http {}</code> block, add:</p>
<pre>
log_format cache '***$time_local '
'$upstream_cache_status '
'Cache-Control: $upstream_http_cache_control '
'Expires: $upstream_http_expires '
'"$request" ($status) '
'"$http_user_agent" ';
access_log /var/log/nginx/cache.log cache;
</pre>
<h4 id="configure-backends">Nginx configuration for backend servers</h4>
<p>Within the <code>http {}</code> block, add:</p>
<pre>
include /etc/nginx/app-servers.include;
</pre>
<p>And <code>/etc/nginx/app-servers.include</code> looks like:</p>
<pre>
upstream backend {
ip_hash;
server 10.245.275.88:80;
server 10.292.150.34:80;
}
</pre>
<h4 id="configure-paths">Configure cache path/parameters</h4>
<p>Within the <code>http {}</code> block, add:</p>
<pre>
proxy_cache_path /var/www/nginx_cache levels=1:2
keys_zone=one:10m;
inactive=7d max_size=200m;
proxy_temp_path /var/www/nginx_temp;
</pre>
<h4 id="more-config">More proxy cache configuration</h4>
<p>We added
the username from the <em>wordpress_logged_in_*</em> cookie as part of the cache key
so that different logged in users will get the appropriate page from
the cache. However, our Wordpress configuration sends HTTP headers disabling
the cache when a user is logged in so this is actually not used. But it does
not hurt to include this, in case we change our Wordpress configuration
in the future.
</p>
<p>Within the <code>server {}</code> block, add:</p>
<pre>
location / {
# capture cookie for use in cache key
if ($http_cookie ~* "wordpress_logged_in_[^=]*=([^%]+)%7C") {
set $my_cookie $1;
}
proxy_pass http://backend;
proxy_cache one;
proxy_cache_key $scheme$proxy_host$uri$is_args$args$my_cookie;
proxy_cache_valid 200 302 304 10m;
proxy_cache_valid 301 1h;
proxy_cache_valid any 1m;
}
</pre>
<h4 id="non-cached-locations">Configure locations that shouldn't be cached</h4>
<p>If WordPress sends the appropriate HTTP Cache-Control headers, this step
is not necessary. But we have added it to be on the safe side.
Within the <code>server {}</code> block, add:</p>
<pre>
location /wp-admin { proxy_pass http://backend; }
location /wp-login.php { proxy_pass http://backend; }
</pre>
<h4 id="restart-nginx">Restart Nginx</h4>
<p>The Nginx reverse proxy cache should work without modification to the
Apache configuration. In our case, we had to disable WP Super Cache
because we had been using that previously.</p>
<pre>/etc/init.d/nginx restart</pre>
<h4 id="view-log">View the log</h4>
<p>Check the <code>/var/log/nginx/cache.log</code> to see if everything is
working correctly. The log should diplay <em>HIT</em>, <em>MISS</em>,
and <em>EXPIRED</em> appropriately. If the log shows only misses, check
the Cache-Control and Expires HTTP headers that are sent from Apache+Wordpress.
</p>
<h4 id="cache-headers">Example Apache/Wordpress configuration that disabled the Nginx cache</h4>
<p>Part of the WP Super Cache configuration included the following in the
<code>.htaccess</code> file. It had to be removed for Nginx cache the pages.
(In particular, the <em>must-revalidate</em> part had to be removed.)
</p>
<pre> Header set Cache-Control 'max-age=300, must-revalidate'</pre>
<h4 id="links">Links</h4>
<ul>
<li><a href="http://wiki.nginx.org/NginxHttpProxyModule">
NginxHttpProxyModule Documentation</a></li>
<li><a href="http://forum.nginx.org/search.php?0,search=proxy_cache,author=,page=1,match_type=ALL,match_dates=0,match_forum=2,match_threads=0">
Nginx forum search of the English Mailing list for "proxy_cache"</a></li>
</ul>
How to make urxvt look like gnome-terminal
2009-11-02T01:09:15-08:00https://www.saltycrane.com/blog/2009/11/how-make-urxvt-look-gnome-terminal/<p>My terminal of choice is
<a href="http://software.schmorp.de/pkg/rxvt-unicode.html">rxvt-unicode</a>
(urxvt) because it is fast and lightweight.
However, I recently opened up gnome-terminal and it was so much prettier than
my urxvt. Here's how I made my urxvt look like gnome-terminal. The last step
involves compiling urxvt from source because the latest source includes a patch
to configure horizontal spacing of letters.
</p>
<h4>Set up colors</h4>
<p>Add the following to your <code>~/.Xdefaults</code> file:</p>
<pre>! to match gnome-terminal "Linux console" scheme
! foreground/background
URxvt*background: #000000
URxvt*foreground: #ffffff
! black
URxvt.color0 : #000000
URxvt.color8 : #555555
! red
URxvt.color1 : #AA0000
URxvt.color9 : #FF5555
! green
URxvt.color2 : #00AA00
URxvt.color10 : #55FF55
! yellow
URxvt.color3 : #AA5500
URxvt.color11 : #FFFF55
! blue
URxvt.color4 : #0000AA
URxvt.color12 : #5555FF
! magenta
URxvt.color5 : #AA00AA
URxvt.color13 : #FF55FF
! cyan
URxvt.color6 : #00AAAA
URxvt.color14 : #55FFFF
! white
URxvt.color7 : #AAAAAA
URxvt.color15 : #FFFFFF
</pre>
<h4>Select font</h4>
<p>Also add the following to your <code>~/.Xdefaults</code> file:</p>
<pre>URxvt*font: xft:Monospace:pixelsize=11
</pre>
<h4>Don't use a bold font</h4>
<p>Also add the following to your <code>~/.Xdefaults</code> file:</p>
<pre>URxvt*boldFont: xft:Monospace:pixelsize=11</pre>
<h4>Fix urxvt font width</h4>
<p>This is the most difficult thing to fix. It requires installing urxvt from
CVS source.</p>
<ul>
<li>Install prerequisites:
<pre>apt-get build-dep rxvt-unicode</pre></li>
<li>Get CVS source code:
<pre>cvs -z3 -d :pserver:anonymous@cvs.schmorp.de/schmorpforge co rxvt-unicode</pre></li>
<li>Configure:
<pre>cd rxvt-unicode
./configure --prefix=/home/eliot/lib/rxvt-unicode-20091102</pre></li>
<li>Make & make install:
<pre>make
make install</pre></li>
<li>Link urxvt executable to your <code>~/bin</code> directory:
<pre>cd ~/bin
ln -s ../lib/rxvt-unicode-20091102/bin/urxvt .</pre>
</li>
<li>Edit <code>~/.Xdefaults</code> once again:
<pre>URxvt*letterSpace: -1</pre>
</li>
</ul>
<h4>Also cool: Open links in Firefox</h4>
<p>Here is another trick (thanks to
<a href="http://ztatlock.blogspot.com/2009/02/configuring-urxvt.html">Zachary Tatlock</a>)
to make clicking on URLs open in your Firefox browser.
Add the following to your <code>~/.Xdefaults</code> (yes there's Perl in
your urxvt!):</p>
<pre>URxvt.perl-ext-common : default,matcher
URxvt.urlLauncher : firefox
URxvt.matcher.button : 1
</pre>
<h4>See also</h4>
<ul>
<li><a href="http://www.mail-archive.com/rxvt-unicode@lists.schmorp.de/msg00554.html">
http://www.mail-archive.com/rxvt-unicode@lists.schmorp.de/msg00554.html</a></li>
</ul>
<h4>Screenshots</h4>
<p>Urxvt (default):</p>
<img alt="ugly urxvt screenshot" src="/site_media/image/blog/urxvt_ugly.png" />
<p>Gnome-terminal:</p>
<img alt="gnome-terminal screenshot" src="/site_media/image/blog/gnome_terminal.png" />
<p>Urxvt (modified):</p>
<img alt="pretty urxvt screenshot" src="/site_media/image/blog/urxvt_pretty.png" />
<p>If you're interested, here is how I printed the terminal colors:</p>
<pre>#!/bin/bash
echo -e "\\e[0mCOLOR_NC (No color)"
echo -e "\\e[1;37mCOLOR_WHITE\\t\\e[0;30mCOLOR_BLACK"
echo -e "\\e[0;34mCOLOR_BLUE\\t\\e[1;34mCOLOR_LIGHT_BLUE"
echo -e "\\e[0;32mCOLOR_GREEN\\t\\e[1;32mCOLOR_LIGHT_GREEN"
echo -e "\\e[0;36mCOLOR_CYAN\\t\\e[1;36mCOLOR_LIGHT_CYAN"
echo -e "\\e[0;31mCOLOR_RED\\t\\e[1;31mCOLOR_LIGHT_RED"
echo -e "\\e[0;35mCOLOR_PURPLE\\t\\e[1;35mCOLOR_LIGHT_PURPLE"
echo -e "\\e[0;33mCOLOR_YELLOW\\t\\e[1;33mCOLOR_LIGHT_YELLOW"
echo -e "\\e[1;30mCOLOR_GRAY\\t\\e[0;37mCOLOR_LIGHT_GRAY"</pre>
Notes on switching my Djangos to mod_wsgi
2009-10-22T00:44:17-07:00https://www.saltycrane.com/blog/2009/10/notes-switching-my-djangos-mod_wsgi/<p>I'm slowly trying to make my Django web servers conform to current
best practices. I've
<a href="/blog/2009/04/notes-using-nginx-mod_python-and-django/">
set up an Nginx reverse proxy for serving static files</a>,
<a href="/blog/2009/05/notes-using-pip-and-virtualenv-django/">
started using virtualenv to isolate my Python environments</a>, and
<a href="/blog/2009/08/notes-migrating-blog-sqlite-postgresql/">
migrated my database to PostgreSQL</a>. I ultimately want to implement
<a href="http://bretthoerner.com/blog/2008/oct/27/using-nginx-memcached-module-django/">
memcached+Nginx caching</a> in my reverse proxy, but the next task on my to-do list
is switching from mod_python to <a href="http://code.google.com/p/modwsgi/">mod_wsgi</a>.
</p>
<p>
Within the past year (or maybe before), mod_wsgi has become the
<a href="http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/">
preferred</a>
<a href="http://simonwillison.net/2009/Apr/1/modwsgi/">method</a>
for serving Django applications. I also originally thought
<a href="http://collingrady.wordpress.com/2009/01/06/mod_python-versus-mod_wsgi/">
switching
from mod_python</a> to mod_wsgi would save me some much needed memory on my
256MB VPS. But after trying it out, running with a single Apache process
in each case, the memory footprint was about the same. Even switching
from mod_wsgi's embedded mode to daemon mode didn't make a significant difference.
Likely the performance is better with mod_wsgi, though.
</p>
<p>Here are my notes on installing mod_wsgi.</p>
<h4>Configuration References</h4>
<ul>
<li>Django docs: <a href="http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/#howto-deployment-modwsgi">
How to use Django with Apache and mod_wsgi</a></li>
<li>mod_wsgi docs: <a href="http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide">
Quick Configuration Guide</a></li>
<li>A good blog article: <a href="http://bretthoerner.com/blog/2008/oct/09/configs-nginx-and-apache-mod_wsgi/">
Configs for nginx and Apache with mod_wsgi</a></li>
</ul>
<h4>Advice from mod_wsgi author Graham Dumpleton</h4>
<ul>
<li><a href="http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html">
Load spikes and excessive memory usage in mod_python</a></li>
<li><a href="http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html">
Using mod_wsgi when developing Django sites</a></li>
<li><a href="http://groups.google.com/group/django-users/browse_thread/thread/6d670b0fa7c0d733/4ff111a1f00f7629?q=worker+daemon#4ff111a1f00f7629">
http://groups.google.com/group/django-users/browse_thread/thread/6d670b0fa7c0d733/4ff111a1f00f7629?q=worker+daemon#4ff111a1f00f7629</a></li>
<li><a href="http://groups.google.com/group/modwsgi/browse_thread/thread/cb21864a97d44ee9/38716433921a48cb?q=worker+daemon#38716433921a48cb">
http://groups.google.com/group/modwsgi/browse_thread/thread/cb21864a97d44ee9/38716433921a48cb?q=worker+daemon#38716433921a48cb</a></li>
</ul>
<h4>Install mod_wsgi and apache mpm-worker</h4>
<p>I'm not 100% sure about prefork vs. worker mpm, but Graham Dumpleton
favors worker mpm.</p>
<pre>sudo apt-get install libapache2-mod-wsgi
sudo apt-get install apache2-mpm-worker</pre>
<h4>Create .wsgi application file</h4>
<p>My virtualenv is located at <code>/srv/python-environments/saltycrane</code>.
My Django settings files is at <code>/srv/SaltyCrane/iwiwdsmi/settings.py</code>.
</p>
<p><code>/srv/SaltyCrane/saltycrane.wsgi</code>:</p>
<pre class="python">import os
import sys
import site
site.addsitedir('/srv/python-environments/saltycrane/lib/python2.5/site-packages')
os.environ['DJANGO_SETTINGS_MODULE'] = 'iwiwdsmi.settings'
sys.path.append('/srv/SaltyCrane')
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()</pre>
<h4>Edit Apache's httpd.conf file</h4>
<p>I went back and forth between using embedded mode or daemon mode.
I've ended up with embedded mode for now since it seems to use a tad less
memory and is supposed to be a little bit faster. However, Graham Dumpleton seems to
recommend daemon mode for people on VPSs. I may change my mind again later.
To use daemon mode, I just need to uncomment the <code>WSGIDaemonProcess</code>
and <code>WSGIProcessGroup</code> lines.
I have <code>StartServers</code> set to 1 because I can only afford to have
one Apache process running.
This is assuming nginx is proxying requests to apache. For more
on my nginx setup, see <a href="/blog/2009/04/notes-using-nginx-mod_python-and-django/">
here</a>.</p>
<p>Edit <code>/etc/apache2/httpd.conf</code>:</p>
<pre>
<IfModule mpm_worker_module>
StartServers 1
ServerLimit 1
ThreadsPerChild 5
ThreadLimit 5
MinSpareThreads 5
MaxSpareThreads 5
MaxClients 5
MaxRequestsPerChild 500
</IfModule>
KeepAlive Off
NameVirtualHost 127.0.0.1:8080
Listen 8080
<VirtualHost 127.0.0.1:8080>
ServerName www.saltycrane.com
# WSGIDaemonProcess saltycrane.com processes=1 threads=5 display-name=%{GROUP}
# WSGIProcessGroup saltycrane.com
WSGIScriptAlias / /srv/SaltyCrane/saltycrane.wsgi
</VirtualHost>
<VirtualHost 127.0.0.1:8080>
ServerName supafu.com
# WSGIDaemonProcess supafu.com processes=1 threads=5 display-name=%{GROUP}
# WSGIProcessGroup supafu.com
WSGIScriptAlias / /srv/Supafu/supafu.wsgi
</VirtualHost>
<VirtualHost 127.0.0.1:8080>
ServerName handsoncards.com
# WSGIDaemonProcess handsoncards.com processes=1 threads=5 display-name=%{GROUP}
# WSGIProcessGroup handsoncards.com
WSGIScriptAlias / /srv/HandsOnCards/handsoncards.wsgi
</VirtualHost>
</pre>
<h4>Restart Apache</h4>
<pre>sudo /etc/init.d/apache2 restart</pre>
Install WordPress 2.8.4 on Ubuntu 9.04 Jaunty
2009-10-20T17:11:10-07:00https://www.saltycrane.com/blog/2009/10/install-wordpress-284-ubuntu-904-jaunty/<p>Since we're using <a href="http://wordpress.org/">WordPress</a> at work, I decided to install WordPress on my local machine for testing and educational purposes. Here are my notes. The versions of stuff are: WordPress 2.8.4, Ubuntu 9.04 Jaunty Jackalope, MySQL 5.1, PHP 5.2.6, Apache 2.2.11 with prefork mpm.</p>
<h4>Install prerequisites</h4>
<ul>
<li>Install Apache and PHP
<pre>sudo apt-get install php5
sudo apt-get install php5-mysql</pre>
</li>
<li>Install MySQL Server
<pre>sudo apt-get install mysql-server</pre>
<p>Set a password for the MySQL root user when prompted.</p>
</li>
</ul>
<h4>Download Wordpress code</h4>
<pre>cd /var/www
sudo wget http://wordpress.org/wordpress-2.8.4.tar.gz
sudo tar zxvf wordpress-2.8.4.tar.gz</pre>
<h4>Create MySQL database and user</h4>
<pre>mysql -uroot -p</pre>
<p>Enter the password you created above.</p>
<pre>CREATE DATABASE wordpress;
CREATE USER 'wp_user'@'localhost' IDENTIFIED BY 'wp_password';
GRANT ALL PRIVILEGES ON wordpress.* TO 'wp_user'@'localhost';
\q</pre>
<h4>Edit WordPress wp-config.php file</h4>
<ul>
<li>Copy the sample file
<pre>cd /var/www/wordpress
sudo cp wp-config-sample.php wp-config.php</pre>
</li>
<li>Edit the following lines in <code>/var/www/wordpress/wp-config.php</code>
<pre>/** The name of the database for WordPress */
define('DB_NAME', 'wordpress');
/** MySQL database username */
define('DB_USER', 'wp_user');
/** MySQL database password */
define('DB_PASSWORD', 'wp_password');
/** MySQL hostname */
define('DB_HOST', 'localhost');</pre>
</li>
</ul>
<h4>Set up Apache virtual host</h4>
<ul>
<li>Edit <code>/etc/apache2/sites-available/wordpress</code>
<pre>
ServerName localhost
<VirtualHost *:80>
DocumentRoot /var/www/wordpress
ErrorLog /var/log/apache2/wordpress.error.log
</VirtualHost>
</pre>
</li>
<li>Symlink to <code>sites-enabled</code>
<pre>sudo ln -s /etc/apache2/sites-available/wordpress /etc/apache2/sites-enabled/wordpress</pre>
</li>
<li>Remove default virtual host from <code>sites-enabled</code>
<pre>sudo rm /etc/apache2/sites-enabled/000-default</pre>
</li>
</ul>
<h4>Restart Apache and view your Wordpress site</h4>
<ul>
<li><pre>sudo /etc/init.d/apache2 restart</pre></li>
<li>Go to <a href="http://localhost">http://localhost</a> in your browser.
You should get the WordPress Welcome page.
</li>
</ul>
<h4>References</h4>
<ul>
<li><a href="http://acassis.wordpress.com/2008/10/30/installing-wordpress-on-ubuntu/">
http://acassis.wordpress.com/2008/10/30/installing-wordpress-on-ubuntu/</a></li>
<li><a href="https://help.ubuntu.com/community/WordPress">
https://help.ubuntu.com/community/WordPress</a></li>
<li><a href="http://codex.wordpress.org/Installing_WordPress">
http://codex.wordpress.org/Installing_WordPress</a></li>
</ul>
Notes on installing Scalr on Ubuntu Jaunty
2009-10-13T11:14:15-07:00https://www.saltycrane.com/blog/2009/10/notes-installing-scalr-ubuntu-jaunty/<p>Here are my notes on installing Scalr. I am using an Alestic Ubuntu
9.04-jaunty-base-20091011 image. Here are my references:</p>
<ul>
<li><a href="http://wiki.scalr.net/Installation">http://wiki.scalr.net/Installation</a></li>
<li><a href="http://abraham.taherivand.net/2009/08/scalr-1-1-0-getting-in-touch-part-i/">http://abraham.taherivand.net/2009/08/scalr-1-1-0-getting-in-touch-part-i/</a></li>
<li><a href="http://blog.getasysadmin.com/2008/11/how-to-install-scalr-on-ubuntu-810-ec2.html">http://blog.getasysadmin.com/2008/11/how-to-install-scalr-on-ubuntu-810-ec2.html</a></li>
</ul>
<h4>Install prerequisites</h4>
<pre><code>apt-get update
apt-get upgrade
apt-get install php5
apt-get install php5-mysql
apt-get install php5-mcrypt
apt-get install php5-mhash
apt-get install php5-snmp
# apt-get install bind9
</code></pre>
<h4>Install php ssh2 extension</h4>
<p><a href="http://www.how2pc.co.il/blog/2009/08/how-to-install-a-php-ssh2-extension-on-ubuntu-server-9-04/">http://www.how2pc.co.il/blog/2009/08/how-to-install-a-php-ssh2-extension-on-ubuntu-server-9-04/</a></p>
<pre><code>apt-get install php5-dev php5-cli php-pear build-essential
apt-get install libssh2-1-dev
pecl install -f ssh2
echo ‘extension=ssh2.so’ > /etc/php5/conf.d/ssh2.ini
</code></pre>
<h4>Install snmp</h4>
<pre><code>apt-get install snmp
# apt-get install snmpd
</code></pre>
<h4>Test environment (optional)</h4>
<pre><code>/etc/init.d/apache2 restart
cd /var/www
wget http://scalr.googlecode.com/files/testenvironment.php
</code></pre>
<p>Visit <a href="http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/testenvironment.php">http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/testenvironment.php</a>
in your browser. You should get the follwing error:</p>
<pre><code>Warning: require_once(/var/www/../src/prepend.inc.php) [function.require-once]: failed to open stream: No such file or directory in /var/www/testenvironment.php on line 93
Fatal error: require_once() [function.require]: Failed opening required '/var/www/../src/prepend.inc.php' (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/testenvironment.php on line 93
</code></pre>
<p>This error means extensions were set up correctly.</p>
<h4>Install Scalr code</h4>
<pre><code>cd /var
wget http://scalr.googlecode.com/files/scalr-1.1.0.tar.gz
tar zxvf scalr-1.1.0.tar.gz
chown -R www-data:www-data /var/scalr/app/cache /var/scalr/app/cron/cron.pid /var/scalr/app/etc/.passwd
chmod 700 -R /var/scalr/app/cache /var/scalr/app/cron/cron.pid /var/scalr/app/etc/.passwd
mv www www_old
ln -s /var/scalr/app/www /var/www
# chmod a+rX -R /var/www
</code></pre>
<h4>Setup MySQL database</h4>
<pre><code>apt-get install mysql-server
mysqladmin -p create scalr
mysql -p scalr < /var/scalr/sql/database.sql
</code></pre>
<p>Edit <code>/var/scalr/app/etc/config.ini</code> with your database settings.</p>
<h4>Configure account</h4>
<ul>
<li>Copy cert-XXXXXXXXXXXX.pem and pk-XXXXXXXXXXXX.pem into <code>/var/scalr/app/etc</code></li>
<li>Go to <a href="http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/login.php">http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/login.php</a></li>
<li>Login with admin/admin</li>
<li>Change the following settings at Settings > Core settings (<a href="http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/settings_core.php">http://ec2-XX-XXX-XXX-XX.compute-1.amazonaws.com/settings_core.php</a>)
<ul>
<li>Password</li>
<li>Email</li>
<li>Account ID: (AWS account ID)</li>
<li>Key name: (xxxxxxxxxx part of cert-xxxxxxxxxx.pem or pk-xxxxxxxxxx.pem file)</li>
<li>Access Key: (Secret Access Key)</li>
<li>Access Key ID: (Access Key ID)</li>
</ul></li>
</ul>
<h4>Set up cron jobs</h4>
<p>To do. See other references.</p>
Notes on Python Fabric 0.9b1
2009-10-04T22:36:28-07:00https://www.saltycrane.com/blog/2009/10/notes-python-fabric-09b1/<p><a href="http://fabfile.org">Fabric</a> is a Python package used for
deploying websites or generally running commands on a remote server.
I first used Fabric about a <a href="/blog/2008/09/notes-python-deployment-using-fabric/">
year ago</a> and thought it was great. Since then, Fabric has procured a
<a href="http://lists.gnu.org/archive/html/fab-user/2009-04/msg00041.html">
new maintainer</a>, a <a href="http://www.nongnu.org/fab/">new domain</a>,
and a few <a href="http://git.fabfile.org/cgit.cgi/fabric/refs/">new revisions</a>.
</p>
<p>Here are my notes on installing the latest stable version (0.9b1) on Ubuntu Jaunty and
running a simple example.</p>
<h4>Install Fabric 0.9b1</h4>
<ul>
<li>Install Easy Install & pip
<pre>sudo apt-get install python-setuptools python-dev build-essential</pre>
<pre>sudo easy_install -U pip</pre>
</li>
<li><p>Install Fabric</p>
<p>Note: According to the Fabric website, the latest version of the prerequisite Python library,
<a href="http://www.lag.net/paramiko/">Paramiko</a> has a bug, so it is recommended
to install the previous version, 1.7.4, instead. This can be accomplished by creating
a requirements file for pip:</p>
<pre>http://www.lag.net/paramiko/download/paramiko-1.7.4.tar.gz
http://git.fabfile.org/cgit.cgi/fabric/snapshot/fabric-0.9b1.tar.gz</pre>
<p>To install, use the <code>pip install</code> command with the <code>-r</code> option
and the path to your requirements file. For convenience, you can install Fabric
using my requirements file:
</p>
<pre>sudo pip install -r http://www.saltycrane.com/site_media/code/fabric-requirements.txt</pre>
</li>
</ul>
<h4>Using Fabric</h4>
<ul>
<li>Create a file called fabfile.py in ~/myproject:
<pre class="python">from __future__ import with_statement # needed for python 2.5
from fabric.api import env, run
def ec2():
env.hosts = ['ec2-65-234-55-183.compute-1.amazonaws.com']
env.user = 'saltycrane'
env.key_filename = '/path/to/my/id_ssh_keyfile'
def ps_apache():
run('ps -e -O rss,pcpu | grep apache')</pre>
</li>
<li>Run it
<pre>cd ~/myproject
fab ec2 ps_apache</pre>
<p>Results:</p>
<pre>[ec2-65-234-55-183.compute-1.amazonaws.com] run: ps -e -O rss,pcpu | grep apache
[ec2-65-234-55-183.compute-1.amazonaws.com] err: stdin: is not a tty
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 3571 10996 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5047 28352 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5048 27756 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5049 23752 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5050 27344 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5055 27344 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5166 28404 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 5167 27900 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 9365 1208 0.0 S ? 00:00:00 /bin/bash -l -c ps -e -O rss,pcpu | grep apache
Done.
Disconnecting from ec2-65-234-55-183.compute-1.amazonaws.com... done.</pre>
</li>
</ul>
<h4>List of available <code>env</code> options</h4>
<p>I extracted this list from
<a href="http://git.fabfile.org/cgit.cgi/fabric/tree/fabric/state.py?id=aabdc44f73063fe3787b00ce2119ba43886cc007"><code>state.py</code> (0.9b1)</a>.
Or view the <a href="http://git.fabfile.org/cgit.cgi/fabric/tree/fabric/state.py">tip version</a>
</p>
<pre class="python">env.reject_unknown_hosts = True # reject unknown hosts
env.disable_known_hosts = True # do not load user known_hosts file
env.user = 'username' # username to use when connecting to remote hosts
env.password = 'mypassword' # password for use with authentication and/or sudo
env.hosts = ['host1.com', 'host2.com'] # comma-separated list of hosts to operate on
env.roles = ['web'] # comma-separated list of roles to operate on
env.key_filename = 'id_rsa' # path to SSH private key file. May be repeated.
env.fabfile = '../myfabfile.py' # name of fabfile to load, e.g. 'fabfile.py' or '../other.py'
env.warn_only = True # warn, instead of abort, when commands fail
env.shell = '/bin/sh' # specify a new shell, defaults to '/bin/bash -l -c'
env.rcfile = 'myfabconfig' # specify location of config file to use
env.hide = ['everything'] # comma-separated list of output levels to hide
env.show = ['debug'] # comma-separated list of output levels to show
env.version = '1.0'
env.sudo_prompt = 'sudo password:'
env.use_shell = False
env.roledefs = {'web': ['www1', 'www2', 'www3'],
'dns': ['ns1', 'ns2'],
}
env.cwd = 'mydir'</pre>
<h4>How to check the status code of a command</h4>
<p>To check the return code of your command, set the <code>env.warn_only</code> option to True and check the
<code>return_code</code> attribute of object returned from <code>run()</code>.
For example:
</p>
<pre class="python">def ec2():
env.hosts = ['ec2-65-234-55-183.compute-1.amazonaws.com']
env.user = 'saltycrane'
env.key_filename = '/path/to/my/id_ssh_keyfile'
env.warn_only = True
def getstatus():
output = run('ls non_existent_file')
print 'output:', output
print 'failed:', output.failed
print 'return_code:', output.return_code</pre>
<pre>fab ec2 getstatus</pre>
<pre>[ec2-65-234-55-183.compute-1.amazonaws.com] run: ls non_existent_file
[ec2-65-234-55-183.compute-1.amazonaws.com] err: ls: cannot access non_existent_file: No such file or directory
Warning: run() encountered an error (return code 2) while executing 'ls non_existent_file'
output:
failed: True
return_code: 2
Done.
Disconnecting from ec2-65-234-55-183.compute-1.amazonaws.com... done.</pre>
<h4>Links</h4>
<ul>
<li><a href="http://git.fabfile.org/cgit.cgi/fabric/tree/">Browse Fabric source code</a></li>
<li><a href="http://lists.gnu.org/archive/html/fab-user/">Fabric mailing list</a></li>
<li><a href="http://docs.fabfile.org/0.9/usage.html">Using Fabric documentation</a></li>
</ul>
<ul>
<li><a href="http://morethanseven.net/2009/07/27/fabric-django-git-apache-mod_wsgi-virtualenv-and-p/">
Fabric, Django, Git, Apache, mod_wsgi, virtualenv and pip deployment</a> <em>(uses
a previous version of Fabric)</em>
</li>
<li><a href="http://gist.github.com/158177">Fabfile.py from the above article
updated for Fabric 0.9/1.0</a>
</li>
</ul>
<h4>Other notes</h4>
<ul>
<li><p>Error message: <code>paramiko.SSHException: Channel closed.</code></p>
<p>Try using Paramiko version 1.7.4 instead of 1.7.5.
See <a href="http://www.mail-archive.com/fab-user@nongnu.org/msg00844.html">
http://www.mail-archive.com/fab-user@nongnu.org/msg00844.html</a>.
</p>
</li>
<li>How to check the version of Paramiko:
<pre>$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import paramiko
>>> paramiko.__version__
'1.7.5 (Ernest)'</pre>
</li>
<li><p>Error message: <code>Fatal error: No existing session</code></p>
<p>This occurred when I used the wrong username.</p>
</li>
</ul>
Fuse error on Scalr
2009-09-25T12:11:25-07:00https://www.saltycrane.com/blog/2009/09/fuse-error-scalr/<p>I got the following error while trying to use s3fs on one of Scalr's Ubuntu EC2 instances:</p>
<pre>fuse: device not found, try 'modprobe fuse' first</pre>
<p>Trying <code>modprobe fuse</code>, I got:</p>
<pre>FATAL: Module fuse not found.
FATAL: Error running install command for fuse</pre>
<p>The solution to this problem is here: <a href="http://groups.google.com/group/scalr-discuss/web/mount-an-s3-bucket-to-your-instance">http://groups.google.com/group/scalr-discuss/web/mount-an-s3-bucket-to-your-instance</a>. The Scalr AMI does not include fuse.ko kernel module, so I needed to install it:</p>
<pre>mkdir -p /lib/modules/2.6.16-xenU/kernel/fs/fuse
cd /lib/modules/2.6.16-xenU/kernel/fs/fuse
rm -f fuse.ko
curl -O http://www.persistentfs.com/extras/ec2-linux-2.6.16-xenU/fuse.ko
/sbin/depmod
modprobe fuse
echo fuse >>/etc/modules</pre>
Using psycopg2 with virtualenv on Ubuntu <del>Jaunty</del>Maverick
2009-07-31T22:56:10-07:00https://www.saltycrane.com/blog/2009/07/using-psycopg2-virtualenv-ubuntu-jaunty/<p><em>Update 2009-11-02</em>: Well I am dumb-- psycopg2 can be installed with
pip/easy_install. The reason I got the error before was because I didn't have
the required dependencies installed. On Ubuntu, I used <code>apt-get build-dep</code>.
So, here's the summary:</p>
<p><em>Update 2009-11-11</em>: My update doesn't work. See comments
<a href="#c5746">#4</a> and <a href="#c5750">#5</a> below.</p>
<p><em>Update 2010-05-17</em>: Here is what finally worked for me using
Python 2.6 on Ubuntu 10.04 Lucid Lynx using virtualenv 1.4.8 and pip 0.7.1. Thanks to <a href="#c9520">Daniel</a>
for the final piece of the solution.
</p>
<p><em>Update 2011-05-13</em>: Looks like the mx+virtualenv issue has been
<a href="http://is.gd/JcwElp">reported</a> and
<a href="https://github.com/dvarrazzo/psycopg/commit/af424821b70299704449337904d3ab7fb1b40751">fixed</a> to be released in 2.4.2.
via <a href="http://twitter.com/#!/psycopg/status/68992875530162176">@psycopg's tweet</a>
</p>
<p><em>Update 2011-11-05</em>: psycopg2 2.4.2 has been released with the mxDateTime fix
so the install is very easy now.</p>
<h4>Install dependencies</h4>
<pre class="console">$ sudo apt-get build-dep python-psycopg2</pre>
<h4>Install pyscopg2 in a virtualenv</h4>
<pre class="console">$ virtualenv --no-site-packages myenv
$ source myenv/bin/activate
$ pip install psycopg2 </pre>
<br>
<p>No longer needed:
<del>$ easy_install -i http://downloads.egenix.com/python/index/ucs4/ egenix-mx-base </del>
</p>
<br><br>
<p><b>*** THE REST OF THIS POST DESCRIBES MY INITIAL OUTDATED SOLUTION. ***</b></p>
<p>I <a href="/blog/2009/05/notes-using-pip-and-virtualenv-django/">
recently</a> starting using <a href="http://pypi.python.org/pypi/virtualenv">
virtualenv</a> for creating isolated Python environments. Virtualenv has made
it easy to manage different Python configurations for different websites, but I
was slowed down a bit trying to use <a href="http://pypi.python.org/pypi/psycopg2/">
psycopg2</a>, the Python-PostgreSQL database adapter, with virtualenv after upgrading
to Ubuntu 9.04 Jaunty Jackalope.
</p>
<p>
Currently, virtualenv (1.3.3) doesn't find Ubuntu Jaunty's Python packages.
In Ubuntu Jaunty, the default Python version changed from 2.5 to 2.6. More importantly,
the <code>site-packages</code> directory was changed to <code>dist-packages</code>.
Prior to Ubuntu 9.04, Ubuntu's Python packages, such as <code>python-psycopg2</code>,
were installed to <code>/usr/lib/python2.5/site-packages</code>. Now Ubuntu's Python
packages are installed to <code>/usr/lib/python2.6/dist-packages</code>. (See
<a href="http://groups.google.com/group/python-virtualenv/browse_thread/thread/79c685d103c805c7">
this discussion</a> at the virtualenv group for more information.)
</p>
<p>As a result of this change, virtualenv (as of 1.3.3) doesn't find Ubuntu's
Python packages installed using <code>apt-get</code>. My solution
was to create symlinks to the desired packages and egg-info files in
<code>site-packages</code>. I'm not sure if this is the proper way to handle this.
If there is a better solution, please let me know. One advantage of using this
method is that I don't need to clutter my virtualenv with all the packages that
have accumulated in my global <code><del>site-packages</del>dist-packages</code>.
</p>
<h4>Install easy_install, pip, and virtualenv</h4>
<pre>sudo apt-get install python-setuptools python-dev build-essential
sudo easy_install -U pip
sudo pip install -U virtualenv</pre>
<h4>Install Ubuntu's psycopg2 package</h4>
<pre>sudo apt-get install python-psycopg2</pre>
<h4>Symlink the psycopg2 (and mxDateTime) files</h4>
<pre>sudo mkdir /usr/lib/python2.6/site-packages
sudo ln -s /usr/lib/python2.6/dist-packages/psycopg2 /usr/lib/python2.6/site-packages
sudo ln -s /usr/lib/python2.6/dist-packages/psycopg2-2.0.8.egg-info /usr/lib/python2.6/site-packages
sudo ln -s /usr/lib/python2.6/dist-packages/mx /usr/lib/python2.6/site-packages</pre>
<h4>Create a virtualenv</h4>
<pre>virtualenv myenv</pre>
<h4>Check what's available</h4>
<pre>pip freeze -E myenv</pre>
<p>Results:</p>
<pre>psycopg2==2.0.8
wsgiref==0.1.2</pre>
<br><br>
<p>Note: you might wonder why I didn't do a <code>pip install -E myenv psycopg2</code>.
I tried this, but got an error. Maybe psycopg2 doesn't support pip/easy_install?
Here is my error message:</p>
<pre style="overflow: scroll; height: 70px">Downloading/unpacking psycopg2
Downloading psycopg2-2.0.11.tar.gz (255Kb): 255Kb downloaded
Running setup.py egg_info for package psycopg2
error: No such file or directory
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
error: No such file or directory
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in ./pip-log.txt
Complete output from command temp/bin/python /usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py install -E temp psycopg2 temp ___VENV_RESTART___:
----------------------------------------
Traceback (most recent call last):
File "/usr/local/bin/pip", line 3, in <module>
pip.main()
File "/usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py", line 926, in main
return command.main(initial_args, args[1:], options)
File "/usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py", line 258, in main
restart_in_venv(options.venv, site_packages, complete_args)
File "/usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py", line 1009, in restart_in_venv
call_subprocess([python, file] + args + [base, '___VENV_RESTART___'])
File "/usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py", line 3643, in call_subprocess
% (command_desc, proc.returncode))
pip.InstallationError: Command temp/bin/python /usr/local/lib/python2.6/dist-packages/pip-0.4-py2.6.egg/pip.py install -E temp psycopg2 temp ___VENV_RESTART___ failed with error code 1</pre>
Install wmii snapshot 20090703 on Ubuntu Jaunty
2009-07-13T09:30:34-07:00https://www.saltycrane.com/blog/2009/07/install-wmii-snapshot-20090703-ubuntu-jaunty/<p>On Friday I installed the latest snapshot of <a href="http://wmii.suckless.org/">wmii</a>,
the lightweight, tiling, dynamic, scriptable, keyboard navigatable window manager.
(See my <a href="/blog/tag/wmii/">wmii tag</a> for more posts about wmii.)
Wmii orangizes windows into columns.
Traditionally there has been three modes for each
column: default, stack, and max. "Default" shows all windows in a column.
"Stack" shows
only one window at a time with the title bars of the other windows showing.
"Max" shows only one window without any other titlebars. I usually use two columns
each in stack mode. To navigate windows, I use Mod4+j/k to move among windows in a
column and Mod4+h/l to move between columns. This has worked well for me, but sometimes
I wanted to display two windows in a column while keeping the rest of the windows hidden.
Thankfully, the latest snapshot of wmii now includes a new mode which solves my problem.
It is a hybrid of the "default" and "stack" modes. It is called "defalut-max" and it
allows for multiple stacks of windows per column. There is also a "default+max" mode
which is like a hybrid of the old "default" and "max" modes. Additionally these modes
now apply to the floating layer as well.
</p>
<p>Another really cool thing about the latest wmii snapshot is that it now includes
a Python library to interface with the wmii filesystem and a wmiirc written in
Python. This should make it a lot easier to script wmii. This is exciting stuff.
Thanks to Kris Maglione and the wmii contributors.
</p>
<p>Here are my notes for installing wmii snapshot 20090703
on Ubuntu 9.04 Jaunty.
</p>
<h4>Install prerequisites</h4>
<ul>
<li>
<pre>$ sudo apt-get install build-essential xorg-dev</pre>
<pre>$ sudo apt-get install dwm-tools</pre>
</li>
<li>This is so it can find the freetype headers.
<pre>$ sudo ln -s /usr/include/freetype2/freetype /usr/include/freetype</pre>
</li>
</ul>
<h4>Install wmii</h4>
<ul>
<li>Download wmii+ixp-snap20090703.tgz from the
<a href="http://www.suckless.org/wmii/">wmii homepage</a>
and save it to <code>~/incoming</code>.
</li>
<li>Unpack
<pre>$ cd ~/incoming
$ tar -zxvf wmii+ixp-snap20090703.tgz</pre>
</li>
<li>make config
<pre>$ cd ~/incoming/wmii+ixp-snap20090703
$ make config</pre>
Accept all defaults except set <code>PREFIX=/home/eliot/lib/wmii_20090703</code>
<br><br>
</li>
<li>make & make install
<pre>$ make
$ make install</pre>
</li>
<li>Create links in my <code>~/bin</code> directory:
<pre>$ cd ~/bin
$ ln -s ~/lib/wmii_20090703/bin/* .</pre>
</li>
</ul>
<h4>Use the fun Python wmiirc</h4>
<ul>
<li>Copy the Python files to <code>~/.wmii-hg</code>. This includes the <code>pygmi</code> directory, the <code>pyxp</code> directory, and the two wmiirc files.
<pre>$ cp -r ~/incoming/wmii+ixp-snap20090703/alternative_wmiircs/python/* ~/.wmii-hg/</pre>
</li>
<li>Edit <code>~/.wmii-hg/wmiirc.py</code> as desired.</li>
</ul>
<h4>Run wmii</h4>
<ul>
<li>Create <code>~/.xsession</code>:
<pre>xmodmap ~/.Xmodmap
gnome-screensaver&
gnome-power-manager&
nm-applet --sm-disable&
urxvt&
until wmii; do
true
done</pre>
</li>
<li>Log out of the current window manager and then select X client script
as the session and log in.</li>
</ul>
<h4>Install <code>slock</code> screen locker</h4>
<ul>
<li>Download from <a href="http://tools.suckless.org/slock">http://tools.suckless.org/slock</a></li>
<li>
<pre class="console">$ tar zxvf slock-0.9.tar.gz
$ cd slock-0.9
$ sudo make clean install </pre>
</li>
<li>Running slock:
Simply invoke the 'slock' command. To get out of it, enter your password.
</li>
</li>
</ul>
<h4>Freetype header error</h4>
<p>The first time around, I got the following error:</p>
<pre style="overflow: scroll; height: 80px;">In file included from /usr/include/X11/Xft/Xft.h:39,
from ../include/x11.h:9,
from cmd/wmii9menu.c:49:
/usr/include/ft2build.h:56:38: error: freetype/config/ftheader.h: No such file or directory
In file included from ../include/x11.h:9,
from cmd/wmii9menu.c:49:
/usr/include/X11/Xft/Xft.h:40:10: error: #include expects "FILENAME" or <FILENAME>
In file included from ../include/x11.h:9,
from cmd/wmii9menu.c:49:
/usr/include/X11/Xft/Xft.h:60: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘_XftFTlibrary’
/usr/include/X11/Xft/Xft.h:94: error: expected specifier-qualifier-list before ‘FT_UInt’
/usr/include/X11/Xft/Xft.h:101: error: expected specifier-qualifier-list before ‘FT_UInt’
/usr/include/X11/Xft/Xft.h:198: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/X11/Xft/Xft.h:303: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/X11/Xft/Xft.h:362: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘XftLockFace’
/usr/include/X11/Xft/Xft.h:401: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/X11/Xft/Xft.h:407: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/X11/Xft/Xft.h:416: error: expected declaration specifiers or ‘...’ before ‘FT_UInt’
/usr/include/X11/Xft/Xft.h:417: error: expected declaration specifiers or ‘...’ before ‘FT_UInt’
/usr/include/X11/Xft/Xft.h:426: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘XftCharIndex’
/usr/include/X11/Xft/Xft.h:459: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
make[1]: *** [wmii9menu.o] Error 1
make: *** [dall] Error 2</pre>
<p>Per <a href="http://qtopia.net/modules/newbb_plus/archive.php?forum=8&topic_id=43:">this
discussion</a>, I solved the problem by creating a symbolic link:</p>
<pre>$ sudo ln -s /usr/include/freetype2/freetype /usr/include/freetype</pre>
Notes on C++ development with Emacs on Ubuntu Linux
2009-07-08T17:28:02-07:00https://www.saltycrane.com/blog/2009/07/notes-c-development-emacs-ubuntu-linux/<p>
Here are my notes on setting up an Emacs C++ development environment.
I'm using <a href="http://gcc.gnu.org/">GNU g++</a> (4.3.3),
<a href="http://www.gnu.org/software/make/">GNU make</a> (3.81),
<a href="http://www.gnu.org/software/gdb/">GNU gdb</a> (3.8-debian),
and <a href="http://www.gnu.org/software/emacs/">GNU emacs</a> (23.0.92.1 built from CVS 2009-04-22)
on <a href="http://www.ubuntu.com/">Ubuntu Linux</a> (9.04 Jaunty Jackalope).
</p>
<h4>Install tools</h4>
<p>If you don't already have <code>g++</code>, <code>make</code>, and <code>gdb</code>, install them:</p>
<pre>sudo apt-get install g++ make gdb</pre>
<h4>Create files</h4>
<ul>
<li>Create a project directory:
<pre>mkdir helloworld</pre>
</li>
<li>Create a source file <code>helloworld/hello.cpp</code>:
<pre>#include <iostream>
int main(void)
{
char greeting[] = "Hello World!";
std::cout << greeting << std::endl;
return 0;
}</pre>
</li>
<li>Create a makefile <code>helloworld/Makefile</code>:
<pre class="Makefile"># Makefile for GNU g++
CC=g++
CFLAGS=-g -Wall
all: hello
hello: hello.cpp
$(CC) $(CFLAGS) -o hello hello.cpp
clean:
rm hello
</pre>
</li>
</ul>
<h4>Compile within Emacs</h4>
<p>See also: <a href="http://www.gnu.org/software/emacs/manual/html_mono/emacs.html.gz#Building">
32 Compiling and Testing Programs</a> in the GNU Emacs manual.</p>
<ul>
<li>Open <code>hello.cpp</code> in Emacs</li>
<li>
<pre>M-x compile RET</pre>
<pre>make -k RET</pre>
<p><em>Note:</em> The default compile command is <code>make -k</code>.
You can change the default command by setting the <code>compile-command</code>
variable in your <code>.emacs</code> file.</p>
</li>
<li>Useful commands in the compilation window:
<ul>
<li><code>C-x `</code>: Go to next error in the code</li>
<li><code>M-n</code>: Go to next error message</li>
<li><code>M-p</code>: Go to previous error message</li>
<li><code>RET</code>: Go to the source code for the current error message</li>
</ul>
</li>
</ul>
<h4>Run using Emacs shell</h4>
<ul>
<li>
<pre>M-! ./hello RET</pre>
<p>You should see the output in the minibuffer.</p>
</li>
</ul>
<h4>Debug with gdb in Emacs</h4>
<p>For more information see:
<a href="http://www.gnu.org/software/emacs/manual/html_mono/emacs.html.gz#Debuggers">
32.6 Running Debuggers Under Emacs</a> in the GNU Emacs manual.</p>
<ul>
<li>To show multiple debugging windows such as breakpoints, locals, etc.,
set the <code>gdb-many-windows</code> variable in <code>~/.emacs</code>:
<pre>(setq gdb-many-windows t)</pre>
Restart Emacs.<br><br>
</li>
<li>Start the debugger. While visiting <code>helloworld/hello.cpp</code>:
<pre>M-x gdb RET --annotate=3 hello RET</pre>
</li>
<li>Set a breakpoint by clicking in the left margin at the desired location.</li>
<li>Run the debugger by typing <code>run</code> at the <code>(gdb)</code> prompt
in the <code>*gud-hello*</code> buffer.
</li>
<li>In the GUD buffer, use the following commands:
<ul>
<li><code>C-c C-s</code> Step into</li>
<li><code>C-c C-n</code> Stev over</li>
<li><code>C-c C-p</code> Evaluate the expression at point.</li>
<li><code>C-c C-r</code> Continue</li>
<li><code>C-c C-u</code> Continue until current line</li>
</ul>
<br>
</li>
<li>When finished, type <code>quit</code> at the <code>(gdb)</code> prompt.</li>
</ul>
<h4>Documentation:</h4>
<ul>
<li><a href="http://gcc.gnu.org/onlinedocs/gcc-4.4.0/gcc/">GNU gcc 4.4</a></li>
<li><a href="http://www.gnu.org/software/make/manual/make.html">GNU make</a></li>
<li><a href="http://www.gnu.org/software/emacs/manual/">GNU Emacs manuals</a></li>
<li><a href="http://sourceware.org/gdb/current/onlinedocs/gdb_toc.html">GNU gdb</a></li>
</ul>
<h4>References</h4>
<ul>
<li><em>Practical C++ Programming, 2nd Edition</em> by Steve Oualine (2004)</li>
<li><em>Learning GNU Emacs, 3rd Edition</em> by Debra Cameron et al. (2004)</li>
<li><a href="http://www.linuxjournal.com/article/5765">Emacs: the Free Software IDE</a>
at the Linux Journal (2002)</li>
</ul>
Notes on using pip and virtualenv with Django
2009-05-06T23:18:37-07:00https://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/<p>I have been using
<a href="/blog/2008/08/somewhere-your-python-path/">
a symlinking method</a> to install Python packages up to this point. To better
handle dependencies and multiple versions I have wanted to switch over to
<a href="http://pypi.python.org/pypi/pip">pip</a> and
<a href="http://pypi.python.org/pypi/virtualenv">virtualenv</a>.
Pip is a better alternative to
<a href="http://peak.telecommunity.com/DevCenter/EasyInstall">Easy Install</a>
and virtualenv is a tool to create isolated Python environments.
I have wanted to use pip and virtualenv for a long time now. Finally, today,
I took my first steps and created an environment with the Python packages
required for this blog. My notes are below. (I am running Ubuntu
<del>Intrepid</del><del>Karmic</del>Maverick and Python <del>2.5</del>2.6.)
A lot of my notes on virtualenv are taken from
<a href="http://arthurkoziel.com/2008/10/22/working-virtualenv/">
Arthur Koziel's excellent tutorial</a>.
</p>
<p><em>Update 2012-03-14:</em> Updated examples for pip 1.1 and virtualenv 1.7. As of pip 1.1,
the <code>-E</code> option is removed. As of virtualenv 1.7, the <code>--no-site-packages</code>
has become the default and is deprecated. Use <code>--system-site-packages</code> if you
want to include system site packages (the old default behavior). Examples are run on Ubuntu
10.10 Maverick Meerkat.
</p>
<h4 id="install-easy-install">Install Easy Install</h4>
<p>If you don't already have Easy Install, it can be installed as follows:</p>
<pre class="console">$ sudo apt-get install python-setuptools python-dev build-essential</pre>
<h4 id="install-pip">Install pip</h4>
<p>In most cases it is not necessary to install pip because it is included with virtualenv.</p>
<pre class="console">$ sudo easy_install -U pip
install_dir /usr/local/lib/python2.6/dist-packages/
Searching for pip
Reading http://pypi.python.org/simple/pip/
Reading http://pip.openplans.org
Reading http://www.pip-installer.org
Best match: pip 1.1
Downloading http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gz#md5=62a9f08dd5dc69d76734568a6c040508
Processing pip-1.1.tar.gz
Running pip-1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-i_5nEU/pip-1.1/egg-dist-tmp-ytHsCZ
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
Adding pip 1.1 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip-2.6 script to /usr/local/bin
Installed /usr/local/lib/python2.6/dist-packages/pip-1.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip</pre>
<h4 id="install-virtualenv">Install virtualenv</h4>
<pre class="console">$ sudo easy_install -U virtualenv
install_dir /usr/local/lib/python2.6/dist-packages/
Searching for virtualenv
Reading http://pypi.python.org/simple/virtualenv/
Reading http://www.virtualenv.org
Reading http://virtualenv.openplans.org
Best match: virtualenv 1.7.1.2
Downloading http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.7.1.2.tar.gz#md5=3be8a014c27340f48b56465f9109d9fa
Processing virtualenv-1.7.1.2.tar.gz
Running virtualenv-1.7.1.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-td3AoM/virtualenv-1.7.1.2/egg-dist-tmp-7dJVfO
warning: no previously-included files matching '*.*' found under directory 'docs/_templates'
Adding virtualenv 1.7.1.2 to easy-install.pth file
Installing virtualenv script to /usr/local/bin
Installed /usr/local/lib/python2.6/dist-packages/virtualenv-1.7.1.2-py2.6.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv</pre>
<h4 id="create-virtualenv">Create a virtual environment</h4>
<p>This command creates the isolated virtual environment.</p>
<p><em>Update 2010-04-13:</em> Updated to use
<a href="http://pypi.python.org/pypi/distribute">distribute</a> because
it is the <a href="http://s3.pixane.com/pip_distribute.png">new hotness</a>.
</p>
<pre class="console">$ cd /tmp</pre>
<pre class="console">$ virtualenv --distribute myenv
New python executable in myenv/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.</pre>
<h4 id="create-virtualenv-with-a-different-version-of-python">Create a virtualenv with a different version of Python</h4>
<p>This creates a virtualenv that uses Python 2.7 instead of the default Python 2.6.</p>
<pre class="console">$ virtualenv --distribute --python=/usr/bin/python2.7 myenv-py27
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in myenv-py27/bin/python2.7
Also creating executable in myenv-py27/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.</pre>
<h4 style="color:#888888" id="clear-pythonpath">Clear the PYTHONPATH variable</h4>
<p style="color:#888888">I don't know if this is necessary, but I had a problem with the akismet
module when this was set. Maybe I did something wrong, but when I cleared
PYTHONPATH, it worked.
</p>
<pre style="color:#888888">$ export PYTHONPATH=</pre>
<h4 id="install-yolk">Install a package (Yolk) in the new virtual environment</h4>
<p>Activate the virtual environment and install Yolk inside it.
(<a href="http://pypi.python.org/pypi/yolk">Yolk</a> is a tool that
lists Python packages.)
</p>
<pre class="console">$ source /tmp/myenv/bin/activate
$ pip install yolk
Downloading/unpacking yolk
Downloading yolk-0.4.3.tar.gz (86Kb): 86Kb downloaded
Running setup.py egg_info for package yolk
warning: no files found matching '*.txt' under directory 'tests'
warning: no files found matching '*.conf' under directory 'docs'
warning: no files found matching '*.css_t' under directory 'docs'
warning: no files found matching 'indexsidebar.html' under directory 'docs'
warning: no files found matching 'tests/test_cli.py'
Requirement already satisfied (use --upgrade to upgrade): distribute in ./myenv/lib/python2.6/site-packages/distribute-0.6.24-py2.6.egg (from yolk)
Installing collected packages: yolk
Running setup.py install for yolk
warning: no files found matching '*.txt' under directory 'tests'
warning: no files found matching '*.conf' under directory 'docs'
warning: no files found matching '*.css_t' under directory 'docs'
warning: no files found matching 'indexsidebar.html' under directory 'docs'
warning: no files found matching 'tests/test_cli.py'
Installing yolk script to /tmp/myenv/bin
Successfully installed yolk
Cleaning up...</pre>
<h4 id="activate-virtualenv">Use the virtual environment</h4>
<ul>
<li>Activate the virtual environment
<pre class="console">$ source /tmp/myenv/bin/activate</pre>
</li>
<li>Run yolk to list your installed packages:
<pre class="console">$ yolk -l
Python - 2.6.6 - active development (/usr/lib/python2.6/lib-dynload)
distribute - 0.6.24 - active
pip - 1.1 - active
wsgiref - 0.1.2 - active development (/usr/lib/python2.6)
yolk - 0.4.3 - active</pre>
</li>
<li>Deactivate the environment
<pre class="console">$ deactivate</pre>
</li>
<li>Try running yolk
<pre class="console">$ yolk -l
yolk: command not found</pre>
</li>
</ul>
<h4 id="install-django">Install Django in the virtual environment</h4>
<pre class="console">$ source /tmp/myenv/bin/activate
$ pip install Django
Downloading/unpacking Django
Downloading Django-1.3.1.tar.gz (6.5Mb): 6.5Mb downloaded
Running setup.py egg_info for package Django
Installing collected packages: Django
Running setup.py install for Django
changing mode of build/scripts-2.6/django-admin.py from 644 to 755
changing mode of /tmp/myenv/bin/django-admin.py to 755
Successfully installed Django
Cleaning up...</pre>
<h4 id="pip-freeze">Create a requirements file from existing library versions using pip freeze</h4>
<br>
<p>The <code>pip freeze</code> command allows you to take a snapshot of the
exact versions of all your Python libraries. For more information,
see the documentation on
<a href="http://www.pip-installer.org/en/latest/requirements.html#freezing-requirements">pip freeze</a>.
</p>
<pre class="console">$ source /tmp/myenv/bin/activate
$ pip freeze > /tmp/requirements.txt</pre>
<pre class="console">$ cat /tmp/requirements.txt
Django==1.3.1
distribute==0.6.24
wsgiref==0.1.2
yolk==0.4.3</pre>
<h4 id="pip-uninstall">Uninstall a package</h4>
<pre class="console">$ source /tmp/myenv/bin/activate
$ pip uninstall Django
Uninstalling Django:
/tmp/myenv/bin/django-admin.py
/tmp/myenv/lib/python2.6/site-packages/Django-1.3.1-py2.6.egg-info
/tmp/myenv/lib/python2.6/site-packages/django
Proceed (y/n)? y
Successfully uninstalled Django</pre>
<span id="requirements"></span>
<h4 id="pip-requirements">Install libraries based on a requirements file</h4>
<p>Once you have a requirements file, you can use pip to install the exact versions
of the libraries specified in your requirements file. For more information,
see the documentation on <a href="http://www.pip-installer.org/en/latest/requirements.html">pip
requirements files</a>.
Here's my requirements file for this blog,
<code>/tmp/saltycrane-requirements.txt</code>:</p>
<pre>psycopg2==2.4.2
Django==1.3.1
Markdown==2.0
http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.0.7a.tar.gz
Pygments==1.3.1
Twisted==10.0.0
http://django-tagging.googlecode.com/files/django-tagging-0.3.1.tar.gz
-e hg+http://bitbucket.org/ubernostrum/django-contact-form/#egg=django-contact-form
-e hg+http://bitbucket.org/jezdez/akismet/#egg=akismet
Fabric==1.3.1</pre>
<p>Here's the command to install using my reqirements file:
</p>
<pre class="console" style="height: 600px; overflow: auto">$ source /tmp/myenv/bin/activate
$ pip install -r /tmp/saltycrane-requirements.txt
Downloading/unpacking http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.0.7a.tar.gz (from -r /tmp/saltycrane-requirements.txt (line 7))
Downloading BeautifulSoup-3.0.7a.tar.gz
Running setup.py egg_info for package from http://www.crummy.com/software/BeautifulSoup/download/3.x/Beaut
ifulSoup-3.0.7a.tar.gz
Downloading/unpacking http://django-tagging.googlecode.com/files/django-tagging-0.3.1.tar.gz (from -r /tmp/saltycrane-requirements.txt (line 11))
Downloading django-tagging-0.3.1.tar.gz
Running setup.py egg_info for package from http://django-tagging.googlecode.com/files/django-tagging-0.3.1.tar.gz
Downloading/unpacking psycopg2==2.4.2 (from -r /tmp/saltycrane-requirements.txt (line 1))
Downloading psycopg2-2.4.2.tar.gz (667Kb): 667Kb downloaded
Running setup.py egg_info for package psycopg2
no previously-included directories found matching 'doc/src/_build'
Downloading/unpacking Django==1.3.1 (from -r /tmp/saltycrane-requirements.txt (line 4))
Downloading Django-1.3.1.tar.gz (6.5Mb): 6.5Mb downloaded
Running setup.py egg_info for package Django
Downloading/unpacking Markdown==2.0 (from -r /tmp/saltycrane-requirements.txt (line 5))
Downloading Markdown-2.0.zip (93Kb): 93Kb downloaded
Running setup.py egg_info for package Markdown
Downloading/unpacking Pygments==1.3.1 (from -r /tmp/saltycrane-requirements.txt (line 8))
Downloading Pygments-1.3.1.tar.gz (1.1Mb): 1.1Mb downloaded
Running setup.py egg_info for package Pygments
Downloading/unpacking Twisted==10.0.0 (from -r /tmp/saltycrane-requirements.txt (line 9))
Downloading Twisted-10.0.0.tar.bz2 (2.6Mb): 2.6Mb downloaded
Running setup.py egg_info for package Twisted
Obtaining django-contact-form from hg+http://bitbucket.org/ubernostrum/django-contact-form/#egg=django-contact-form (from -r /tmp/saltycrane-requirements.txt (line 12))
Cloning hg http://bitbucket.org/ubernostrum/django-contact-form/ to ./myenv/src/django-contact-form
Running setup.py egg_info for package django-contact-form
Obtaining akismet from hg+http://bitbucket.org/jezdez/akismet/#egg=akismet (from -r /tmp/saltycrane-requirements.txt (line 13))
Cloning hg http://bitbucket.org/jezdez/akismet/ to ./myenv/src/akismet
Running setup.py egg_info for package akismet
Downloading/unpacking Fabric==1.3.1 (from -r /tmp/saltycrane-requirements.txt (line 15))
Downloading Fabric-1.3.1.tar.gz (167Kb): 167Kb downloaded
Running setup.py egg_info for package Fabric
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no files found matching 'fabfile.py'
Downloading/unpacking zope.interface (from Twisted==10.0.0->-r /tmp/saltycrane-requirements.txt (line 9))
Downloading zope.interface-3.8.0.tar.gz (111Kb): 111Kb downloaded
Running setup.py egg_info for package zope.interface
Downloading/unpacking ssh>=1.7.8 (from Fabric==1.3.1->-r /tmp/saltycrane-requirements.txt (line 15))
Downloading ssh-1.7.13.tar.gz (790Kb): 790Kb downloaded
Running setup.py egg_info for package ssh
Requirement already satisfied (use --upgrade to upgrade): distribute in ./myenv/lib/python2.6/site-packages/distribute-0.6.24-py2.6.egg (from zope.interface->Twisted==10.0.0->-r /tmp/saltycrane-requirements.txt (line 9))
Downloading/unpacking pycrypto>=2.1,!=2.4 (from ssh>=1.7.8->Fabric==1.3.1->-r /tmp/saltycrane-requirements.txt (line 15))
Downloading pycrypto-2.5.tar.gz (426Kb): 426Kb downloaded
Running setup.py egg_info for package pycrypto
Installing collected packages: psycopg2, Django, Markdown, Pygments, Twisted, django-contact-form, akismet, Fabric, BeautifulSoup, django-tagging, zope.interface, ssh, pycrypto
Running setup.py install for psycopg2
building 'psycopg2._psycopg' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.6/psycopg/psycopgmodule.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/green.c -o build/temp.linux-x86_64-2.6/psycopg/green.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/pqpath.c -o build/temp.linux-x86_64-2.6/psycopg/pqpath.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/utils.c -o build/temp.linux-x86_64-2.6/psycopg/utils.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/bytes_format.c -o build/temp.linux-x86_64-2.6/psycopg/bytes_format.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/connection_int.c -o build/temp.linux-x86_64-2.6/psycopg/connection_int.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.6/psycopg/connection_type.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/cursor_int.c -o build/temp.linux-x86_64-2.6/psycopg/cursor_int.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/cursor_type.c -o build/temp.linux-x86_64-2.6/psycopg/cursor_type.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/lobject_int.c -o build/temp.linux-x86_64-2.6/psycopg/lobject_int.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/lobject_type.c -o build/temp.linux-x86_64-2.6/psycopg/lobject_type.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/notify_type.c -o build/temp.linux-x86_64-2.6/psycopg/notify_type.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/xid_type.c -o build/temp.linux-x86_64-2.6/psycopg/xid_type.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_asis.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_binary.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_binary.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_datetime.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_datetime.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_list.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_list.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pboolean.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_pboolean.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pdecimal.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_pdecimal.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pint.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_pint.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pfloat.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_pfloat.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_qstring.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_qstring.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/microprotocols.c -o build/temp.linux-x86_64-2.6/psycopg/microprotocols.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/microprotocols_proto.c -o build/temp.linux-x86_64-2.6/psycopg/microprotocols_proto.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/typecast.c -o build/temp.linux-x86_64-2.6/psycopg/typecast.o -Wdeclaration-after-statement
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.2 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x08040B -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.6/mx -I/usr/include/python2.6 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_mxdatetime.c -o build/temp.linux-x86_64-2.6/psycopg/adapter_mxdatetime.o -Wdeclaration-after-statement
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/psycopg/psycopgmodule.o build/temp.linux-x86_64-2.6/psycopg/green.o build/temp.linux-x86_64-2.6/psycopg/pqpath.o build/temp.linux-x86_64-2.6/psycopg/utils.o build/temp.linux-x86_64-2.6/psycopg/bytes_format.o build/temp.linux-x86_64-2.6/psycopg/connection_int.o build/temp.linux-x86_64-2.6/psycopg/connection_type.o build/temp.linux-x86_64-2.6/psycopg/cursor_int.o build/temp.linux-x86_64-2.6/psycopg/cursor_type.o build/temp.linux-x86_64-2.6/psycopg/lobject_int.o build/temp.linux-x86_64-2.6/psycopg/lobject_type.o build/temp.linux-x86_64-2.6/psycopg/notify_type.o build/temp.linux-x86_64-2.6/psycopg/xid_type.o build/temp.linux-x86_64-2.6/psycopg/adapter_asis.o build/temp.linux-x86_64-2.6/psycopg/adapter_binary.o build/temp.linux-x86_64-2.6/psycopg/adapter_datetime.o build/temp.linux-x86_64-2.6/psycopg/adapter_list.o build/temp.linux-x86_64-2.6/psycopg/adapter_pboolean.o build/temp.linux-x86_64-2.6/psycopg/adapter_pdecimal.o build/temp.linux-x86_64-2.6/psycopg/adapter_pint.o build/temp.linux-x86_64-2.6/psycopg/adapter_pfloat.o build/temp.linux-x86_64-2.6/psycopg/adapter_qstring.o build/temp.linux-x86_64-2.6/psycopg/microprotocols.o build/temp.linux-x86_64-2.6/psycopg/microprotocols_proto.o build/temp.linux-x86_64-2.6/psycopg/typecast.o build/temp.linux-x86_64-2.6/psycopg/adapter_mxdatetime.o -lpq -o build/lib.linux-x86_64-2.6/psycopg2/_psycopg.so
no previously-included directories found matching 'doc/src/_build'
Running setup.py install for Django
changing mode of build/scripts-2.6/django-admin.py from 644 to 755
changing mode of /tmp/myenv/bin/django-admin.py to 755
Running setup.py install for Markdown
changing mode of build/scripts-2.6/markdown.py from 644 to 755
changing mode of /tmp/myenv/bin/markdown.py to 755
Running setup.py install for Pygments
Installing pygmentize script to /tmp/myenv/bin
Running setup.py install for Twisted
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c conftest.c -o conftest.o
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c conftest.c -o conftest.o
building 'twisted.runner.portmap' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c twisted/runner/portmap.c -o build/temp.linux-x86_64-2.6/twisted/runner/portmap.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/twisted/runner/portmap.o -o build/lib.linux-x86_64-2.6/twisted/runner/portmap.so
building 'twisted.protocols._c_urlarg' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c twisted/protocols/_c_urlarg.c -o build/temp.linux-x86_64-2.6/twisted/protocols/_c_urlarg.o
twisted/protocols/_c_urlarg.c: In function ‘unquote’:
twisted/protocols/_c_urlarg.c:41: warning: ‘tmp’ may be used uninitialized in this function
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/twisted/protocols/_c_urlarg.o -o build/lib.linux-x86_64-2.6/twisted/protocols/_c_urlarg.so
building 'twisted.test.raiser' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c twisted/test/raiser.c -o build/temp.linux-x86_64-2.6/twisted/test/raiser.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/twisted/test/raiser.o -o build/lib.linux-x86_64-2.6/twisted/test/raiser.so
building 'twisted.python._epoll' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c twisted/python/_epoll.c -o build/temp.linux-x86_64-2.6/twisted/python/_epoll.o
twisted/python/_epoll.c: In function ‘__pyx_f_6_epoll_5epoll___dealloc__’:
twisted/python/_epoll.c:168: warning: label ‘__pyx_L1’ defined but not used
twisted/python/_epoll.c: In function ‘__pyx_f_6_epoll_5epoll_wait’:
twisted/python/_epoll.c:432: warning: label ‘__pyx_L7’ defined but not used
twisted/python/_epoll.c:430: warning: label ‘__pyx_L6’ defined but not used
twisted/python/_epoll.c: In function ‘__pyx_tp_new_6_epoll_epoll’:
twisted/python/_epoll.c:508: warning: unused variable ‘p’
twisted/python/_epoll.c: In function ‘__pyx_tp_dealloc_6_epoll_epoll’:
twisted/python/_epoll.c:513: warning: unused variable ‘p’
twisted/python/_epoll.c: In function ‘__pyx_tp_traverse_6_epoll_epoll’:
twisted/python/_epoll.c:528: warning: unused variable ‘p’
twisted/python/_epoll.c:527: warning: unused variable ‘e’
twisted/python/_epoll.c: In function ‘__pyx_tp_clear_6_epoll_epoll’:
twisted/python/_epoll.c:533: warning: unused variable ‘p’
twisted/python/_epoll.c: At top level:
twisted/python/_epoll.c:32: warning: ‘__Pyx_UnpackItem’ declared ‘static’ but never defined
twisted/python/_epoll.c:33: warning: ‘__Pyx_EndUnpack’ declared ‘static’ but never defined
twisted/python/_epoll.c:34: warning: ‘__Pyx_PrintItem’ declared ‘static’ but never defined
twisted/python/_epoll.c:35: warning: ‘__Pyx_PrintNewline’ declared ‘static’ but never defined
twisted/python/_epoll.c:37: warning: ‘__Pyx_ReRaise’ declared ‘static’ but never defined
twisted/python/_epoll.c:38: warning: ‘__Pyx_Import’ declared ‘static’ but never defined
twisted/python/_epoll.c:39: warning: ‘__Pyx_GetExcValue’ declared ‘static’ but never defined
twisted/python/_epoll.c:40: warning: ‘__Pyx_ArgTypeTest’ declared ‘static’ but never defined
twisted/python/_epoll.c:41: warning: ‘__Pyx_TypeTest’ declared ‘static’ but never defined
twisted/python/_epoll.c:42: warning: ‘__Pyx_GetStarArgs’ declared ‘static’ but never defined
twisted/python/_epoll.c:43: warning: ‘__Pyx_WriteUnraisable’ declared ‘static’ but never defined
twisted/python/_epoll.c:45: warning: ‘__Pyx_ImportType’ declared ‘static’ but never defined
twisted/python/_epoll.c:46: warning: ‘__Pyx_SetVtable’ declared ‘static’ but never defined
twisted/python/_epoll.c:47: warning: ‘__Pyx_GetVtable’ declared ‘static’ but never defined
twisted/python/_epoll.c:48: warning: ‘__Pyx_CreateClass’ declared ‘static’ but never defined
twisted/python/_epoll.c:50: warning: ‘__Pyx_InitStrings’ declared ‘static’ but never defined
twisted/python/_epoll.c:51: warning: ‘__Pyx_InitCApi’ declared ‘static’ but never defined
twisted/python/_epoll.c:52: warning: ‘__Pyx_ImportModuleCApi’ declared ‘static’ but never defined
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/twisted/python/_epoll.o -o build/lib.linux-x86_64-2.6/twisted/python/_epoll.so
building 'twisted.python._initgroups' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c twisted/python/_initgroups.c -o build/temp.linux-x86_64-2.6/twisted/python/_initgroups.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/twisted/python/_initgroups.o -o build/lib.linux-x86_64-2.6/twisted/python/_initgroups.so
changing mode of build/scripts-2.6/mktap from 644 to 755
changing mode of build/scripts-2.6/tap2deb from 644 to 755
changing mode of build/scripts-2.6/tap2rpm from 644 to 755
changing mode of build/scripts-2.6/trial from 644 to 755
changing mode of build/scripts-2.6/pyhtmlizer from 644 to 755
changing mode of build/scripts-2.6/tapconvert from 644 to 755
changing mode of build/scripts-2.6/manhole from 644 to 755
changing mode of build/scripts-2.6/twistd from 644 to 755
changing mode of build/scripts-2.6/ckeygen from 644 to 755
changing mode of build/scripts-2.6/cftp from 644 to 755
changing mode of build/scripts-2.6/conch from 644 to 755
changing mode of build/scripts-2.6/tkconch from 644 to 755
changing mode of build/scripts-2.6/mailmail from 644 to 755
changing mode of build/scripts-2.6/lore from 644 to 755
changing mode of /tmp/myenv/bin/mktap to 755
changing mode of /tmp/myenv/bin/ckeygen to 755
changing mode of /tmp/myenv/bin/cftp to 755
changing mode of /tmp/myenv/bin/conch to 755
changing mode of /tmp/myenv/bin/tap2deb to 755
changing mode of /tmp/myenv/bin/tap2rpm to 755
changing mode of /tmp/myenv/bin/lore to 755
changing mode of /tmp/myenv/bin/trial to 755
changing mode of /tmp/myenv/bin/pyhtmlizer to 755
changing mode of /tmp/myenv/bin/tkconch to 755
changing mode of /tmp/myenv/bin/tapconvert to 755
changing mode of /tmp/myenv/bin/manhole to 755
changing mode of /tmp/myenv/bin/twistd to 755
changing mode of /tmp/myenv/bin/mailmail to 755
Running setup.py develop for django-contact-form
Creating /tmp/myenv/lib/python2.6/site-packages/django-contact-form.egg-link (link to .)
Adding django-contact-form 0.3 to easy-install.pth file
Installed /tmp/myenv/src/django-contact-form
Running setup.py develop for akismet
Creating /tmp/myenv/lib/python2.6/site-packages/akismet.egg-link (link to .)
Adding akismet 0.1.5 to easy-install.pth file
Installed /tmp/myenv/src/akismet
Running setup.py install for Fabric
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no files found matching 'fabfile.py'
Installing fab script to /tmp/myenv/bin
Running setup.py install for BeautifulSoup
Running setup.py install for django-tagging
Running setup.py install for zope.interface
building 'zope.interface._zope_interface_coptimizations' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.6/src/zope/interface/_zope_interface_coptimizations.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/zope/interface/_zope_interface_coptimizations.o -o build/lib.linux-x86_64-2.6/zope/interface/_zope_interface_coptimizations.so
Skipping installation of /tmp/myenv/lib/python2.6/site-packages/zope/__init__.py (namespace package)
Installing /tmp/myenv/lib/python2.6/site-packages/zope.interface-3.8.0-py2.6-nspkg.pth
Running setup.py install for ssh
Running setup.py install for pycrypto
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for __gmpz_init in -lgmp... no
checking for __gmpz_init in -lmpir... no
checking whether mpz_powm is declared... no
checking whether mpz_powm_sec is declared... no
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for inttypes.h... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking wchar.h usability... yes
checking wchar.h presence... yes
checking for wchar.h... yes
checking for inline... inline
checking for int16_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for int8_t... yes
checking for size_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for uint8_t... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for memmove... yes
checking for memset... yes
configure: creating ./config.status
config.status: creating src/config.h
warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash._MD2' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/MD2.c -o build/temp.linux-x86_64-2.6/src/MD2.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/MD2.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_MD2.so
building 'Crypto.Hash._MD4' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/MD4.c -o build/temp.linux-x86_64-2.6/src/MD4.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/MD4.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_MD4.so
building 'Crypto.Hash._SHA256' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/SHA256.c -o build/temp.linux-x86_64-2.6/src/SHA256.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/SHA256.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_SHA256.so
building 'Crypto.Hash._SHA224' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/SHA224.c -o build/temp.linux-x86_64-2.6/src/SHA224.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/SHA224.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_SHA224.so
building 'Crypto.Hash._SHA384' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/SHA384.c -o build/temp.linux-x86_64-2.6/src/SHA384.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/SHA384.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_SHA384.so
building 'Crypto.Hash._SHA512' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/SHA512.c -o build/temp.linux-x86_64-2.6/src/SHA512.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/SHA512.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_SHA512.so
building 'Crypto.Hash._RIPEMD160' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -DPCT_LITTLE_ENDIAN=1 -Isrc/ -I/usr/include/python2.6 -c src/RIPEMD160.c -o build/temp.linux-x86_64-2.6/src/RIPEMD160.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/RIPEMD160.o -o build/lib.linux-x86_64-2.6/Crypto/Hash/_RIPEMD160.so
building 'Crypto.Cipher.AES' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/AES.c -o build/temp.linux-x86_64-2.6/src/AES.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/AES.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/AES.so
building 'Crypto.Cipher.ARC2' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/ARC2.c -o build/temp.linux-x86_64-2.6/src/ARC2.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/ARC2.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/ARC2.so
building 'Crypto.Cipher.Blowfish' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/Blowfish.c -o build/temp.linux-x86_64-2.6/src/Blowfish.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/Blowfish.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/Blowfish.so
building 'Crypto.Cipher.CAST' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/CAST.c -o build/temp.linux-x86_64-2.6/src/CAST.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/CAST.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/CAST.so
building 'Crypto.Cipher.DES' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -Isrc/libtom/ -I/usr/include/python2.6 -c src/DES.c -o build/temp.linux-x86_64-2.6/src/DES.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/DES.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/DES.so
building 'Crypto.Cipher.DES3' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -Isrc/libtom/ -I/usr/include/python2.6 -c src/DES3.c -o build/temp.linux-x86_64-2.6/src/DES3.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/DES3.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/DES3.so
building 'Crypto.Cipher.ARC4' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/ARC4.c -o build/temp.linux-x86_64-2.6/src/ARC4.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/ARC4.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/ARC4.so
building 'Crypto.Cipher.XOR' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/XOR.c -o build/temp.linux-x86_64-2.6/src/XOR.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/XOR.o -o build/lib.linux-x86_64-2.6/Crypto/Cipher/XOR.so
building 'Crypto.Util.strxor' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/strxor.c -o build/temp.linux-x86_64-2.6/src/strxor.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/strxor.o -o build/lib.linux-x86_64-2.6/Crypto/Util/strxor.so
building 'Crypto.Util._counter' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.6 -c src/_counter.c -o build/temp.linux-x86_64-2.6/src/_counter.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/src/_counter.o -o build/lib.linux-x86_64-2.6/Crypto/Util/_counter.so
Successfully installed psycopg2 Django Markdown Pygments Twisted django-contact-form akismet Fabric BeautifulSoup django-tagging zope.interface ssh pycrypto
Cleaning up...</pre>
<h4 id="virtualenv-mod_python">Use virtualenv with Django and mod_python</h4>
<p>Here is how to use a virtualenv with
<a href="http://www.modpython.org/">mod_python</a>.
This is taken from
<em><a href="http://mydjangoblog.com/2009/03/30/django-mod_python-and-virtualenv/">
Django, mod_python and virtualenv</a></em>.
For more information, see the virtualenv documentation on
<a href="http://virtualenv.openplans.org/#using-virtualenv-without-bin-python">using
virtualenv without bin/python</a>.
</p>
<ul>
<li>Create a file <code>/srv/SaltyCrane/myvirtualdjango.py</code>:
<pre class="python">activate_this = "/srv/python-environments/saltycrane/bin/activate_this.py"
execfile(activate_this, dict(__file__=activate_this))
from django.core.handlers.modpython import handler</pre>
</li>
<li>Edit your <code>httpd.conf</code>
<pre> <Location "/">
SetHandler python-program
PythonHandler myvirtualdjango
SetEnv DJANGO_SETTINGS_MODULE iwiwdsmi.settings
PythonPath "['/srv/SaltyCrane',] + sys.path"
PythonDebug Off
</Location></pre>
</li>
</ul>
<h4 id="virtualenv-mod_wsgi">Use virtualenv with Django and mod_wsgi</h4>
<p><em>Added 2009-09-27:</em>
Here is how I set up my virtualenv with
<a href="http://code.google.com/p/modwsgi/">mod_wsgi</a>.
To use the the packages in my virtualenv,
I used <code>site.addsitedir</code> at the
top of my .wsgi application file. You may also want to set the WSGIPythonHome
variable in your httpd.conf file (outside of any VirtualHost sections). For detailed information
on using mod_wsgi with virtualenv, see the
<a href="http://code.google.com/p/modwsgi/wiki/VirtualEnvironments">Virtual
Environments section</a> of the modwsgi project documentation.
</p>
<ul>
<li><code>/srv/SaltyCrane/saltycrane.wsgi</code>:
<pre>import os
import sys
<b>import site
site.addsitedir('/srv/python-environments/saltycrane/lib/python2.5/site-packages')</b>
os.environ['DJANGO_SETTINGS_MODULE'] = 'iwiwdsmi.settings'
sys.path.append('/srv/SaltyCrane')
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()</pre>
</li>
<li><code>httpd.conf</code>:
<pre> WSGIScriptAlias / /srv/SaltyCrane/saltycrane.wsgi</pre>
</li>
</ul>
<h4 id="background">Background / Discussion</h4>
<p> Below are some links to
some essential information on pip, virtualenv, and Python packaging
(mostly from the creator of pip and virtualenv, Ian Bicking) and some further
discussion on the state of Python packaging. <em>(Updated 2012-11-17)</em>
</p>
<ul>
<li><a href="http://blog.ianbicking.org/2007/10/10/workingenv-is-dead-long-live-virtualenv/">
Workingenv is dead, long live Virtualenv!</a> by Ian Bicking (2007-10-10)
</li>
<li><a href="http://groups.google.com/group/django-developers/msg/5407cdb400157259">
mailing list post explaining a number of package management technologies</a>
by Kevin Teague (2008-09-14)
</li>
<li><a href="http://www.coactivate.org/projects/topp-engineering/blog/2008/09/24/pyinstall-a-new-hope/">
pyinstall: A New Hope</a> by Ian Bicking (2008-09-24)
</li>
<li><a href="http://blog.ianbicking.org/2008/10/01/pyinstall-pybundles/">
pyinstall pybundles</a> by Ian Bicking (2008-10-01)
</li>
<li><a href="http://blog.ianbicking.org/2008/10/28/pyinstall-is-dead-long-live-pip/">
pyinstall is dead, long live pip!</a> by Ian Bicking (2008-10-28)
</li>
<li><a href="http://www.b-list.org/weblog/2008/dec/14/packaging/">
On packaging</a> by James Bennett (2008-12-14)
<li><a href="http://blog.ianbicking.org/2008/12/14/a-few-corrections-to-on-packaging/">
A Few Corrections To "On Packaging"</a> by Ian Bicking (2008-12-14)
</li>
<li><a href="http://www.b-list.org/weblog/2008/dec/15/pip/">
Why I like pip</a> by James Bennett (2008-12-15)
</li>
<li><a href="http://blog.ianbicking.org/2008/12/16/using-pip-requirements/">
Using pip Requirements</a> by Ian Bicking (2008-12-16)
</li>
<li><a href="http://guide.python-distribute.org/">
The Hitchhiker’s Guide to Packaging</a> (Documentation for distribute) (<em>Added 2010-04-13:</em>)
</li>
<li>
<a href="http://blog.ziade.org/2012/09/10/dear-django-help-python-packaging/">Dear Django, help Python Packaging</a>
by Tarek Ziadé (2012-09-10)
</li>
<li>
Brett Cannon's summary of <a href="https://plus.google.com/u/0/115362263245161504841/posts/UcBQK8P4jw3">various PEPs regarding packaging</a> (2012-09-11)</a>
</li>
<li>
<a href="http://blog.ziade.org/2012/09/12/dear-pyramid-help-python-packaging/">Dear Pyramid, help Python Packaging</a>
by Tarek Ziadé (2012-09-12)
</li>
<li>
<a href="https://twitter.com/tarek_ziade/status/269749074872373248/photo/1">The Chronology of Python Packaging Part 1</a>,
<a href="https://twitter.com/tarek_ziade/status/269754203868643328/photo/1">Part 2</a> by Tarek Ziadé (2012-11-17)
</li>
</ul>
Notes on using nginx with mod_python and Django
2009-04-28T00:26:11-07:00https://www.saltycrane.com/blog/2009/04/notes-using-nginx-mod_python-and-django/<p>Here are my notes on setting up
<a href="http://nginx.net/">nginx</a> as a reverse proxy with
Apache, mod_python, and Django on Ubuntu Intrepid. Nginx is used
to serve my static media files while all other requests are passed
on to the Apache/mod_python/Django web server.
</p>
<p>I realize
<a href="http://code.google.com/p/modwsgi/">mod_wsgi</a> has become the
<a href="http://simonwillison.net/2009/Apr/1/modwsgi/">preferred</a>
<a href="http://collingrady.wordpress.com/2009/01/06/mod_python-versus-mod_wsgi/">
way</a> to deploy Django, but I'm a little behind the times and am still
using mod_python. I hope to switch to mod_wsgi soon.
</p>
<p>I <a href="http://www.saltycrane.com/blog/2008/12/card-store-project-4-notes-using-amazons-cloudfront/">
have been using</a> Amazon's
<a href="http://aws.amazon.com/cloudfront/">CloudFront</a> service for
delivering my static media files. As far as I can tell, it has worked well.
My main reason for switching to nginx is so I can skip the extra step of
uploading to Amazon. Regarding my concern about the memory footprint of nginx,
it looks like it is using around 5mb with my two process configuration.
</p>
<p>My configuration parameters are shown below. I'm running two sites, SaltyCrane and
HandsOnCards on a <a href="http://www.slicehost.com/">Slicehost</a> 256mb plan.
</p>
<table width=550>
<tr>
<td><b>Description</b></td>
<td><b>HandsOnCards</b></td>
<td><b>SaltyCrane</b></td>
</tr>
<tr>
<td>Redirection</td>
<td><code class="code2">http://www.handsoncards.com</code> is redirected to <code class="code2">http://handsoncards.com</code></td>
<td><code class="code2">http://saltycrane.com</code> is redirected to <code class="code2">http://www.saltycrane.com</code></td>
</tr>
<tr>
<td>Static media filesystem path</td>
<td><code class="code2">/srv/HandsOnCards/handsoncards/static/</code></td>
<td><code class="code2">/srv/SaltyCrane/iwiwdsmi/media/</code></td>
</tr>
<tr>
<td>Static media web path</td>
<td><code class="code2">/site_media/</code></td>
<td><code class="code2">/site_media/</code></td>
</tr>
<tr>
<td>Django settings.py file location</td>
<td><code class="code2">/srv/HandsOnCards/handsoncards/settings.py</code></td>
<td><code class="code2">/srv/SaltyCrane/iwiwdsmi/settings.py</code></td>
</tr>
<tr>
<td>Additional Python packages path</td>
<td><code class="code2">/srv/python-packages</code></td>
<td><code class="code2">/srv/python-packages</code></td>
</tr>
</table>
<h4>Install nginx</h4>
<p>Some recommend installing nginx from source, but I took the easier route
and used Ubuntu's package manager.</p>
<pre>sudo apt-get install nginx</pre>
<h4>Nginx configuration</h4>
<p>Edit <code>/etc/nginx/nginx.conf</code>:</p>
<pre class="nginx">user www-data www-data;
worker_processes 2;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
sendfile on;
tcp_nopush on;
keepalive_timeout 3;
tcp_nodelay off;
gzip on;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml
application/xml application/xml+rss text/javascript;
server {
listen 80;
server_name www.handsoncards.com;
rewrite ^/(.*) http://handsoncards.com/$1 permanent;
}
server {
listen 80;
server_name handsoncards.com;
access_log /var/log/nginx/handsoncards.com.access.log;
error_log /var/log/nginx/handsoncards.com.error.log;
location / {
proxy_pass http://127.0.0.1:8080/;
include /etc/nginx/proxy.conf;
}
location /site_media/ {
alias /srv/HandsOnCards/handsoncards/static/;
expires 24h;
}
}
server {
listen 80;
server_name saltycrane.com;
rewrite ^/(.*) http://www.saltycrane.com/$1 permanent;
}
server {
listen 80;
server_name www.saltycrane.com;
access_log /var/log/nginx/saltycrane.com.access.log;
error_log /var/log/nginx/saltycrane.com.error.log;
location / {
proxy_pass http://127.0.0.1:8080/;
include /etc/nginx/proxy.conf;
}
location /site_media/ {
alias /srv/SaltyCrane/iwidsmi/media/;
expires 24h;
}
}
}
</pre>
<p>Edit <code>/etc/nginx/proxy.conf</code>:</p>
<pre class="nginx">proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffers 32 4k;</pre>
<p>Restart Nginx</p>
<pre>sudo /etc/init.d/nginx restart</pre>
<h4>Install Apache</h4>
<p>I already had Apache and mod_python installed, but in case you don't:</p>
<pre>apt-get install apache2 apache2-mpm-prefork
apt-get install libapache2-mod-python</pre>
<h4>Apache Configuration</h4>
<p>Edit <code>/etc/apache2/httpd.conf</code>:</p>
<pre>MaxClients 2
MaxRequestsPerChild 350
KeepAlive Off
NameVirtualHost 127.0.0.1:8080
Listen 8080
<VirtualHost 127.0.0.1:8080>
ServerName www.saltycrane.com
<Location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE iwiwdsmi.settings
PythonPath "['/srv/SaltyCrane', '/srv/python-packages'] + sys.path"
PythonDebug Off
</Location>
</VirtualHost>
<VirtualHost 127.0.0.1:8080>
ServerName handsoncards.com
<Location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE handsoncards.settings
PythonPath "['/srv/HandsOnCards', '/srv/python-packages'] + sys.path"
PythonDebug Off
</Location>
</VirtualHost></pre>
<p>Edit <code>/etc/apache2/ports.conf</code> and comment out the following two lines:</p>
<pre>#NameVirtualHost *:80
#Listen 80</pre>
<p>Restart Apache</p>
<pre>sudo /etc/init.d/apache2 restart</pre>
<h4>Add Django's reverse proxy middleware</h4>
<p>Edit your <code>settings.py</code> file to include
<code class="code2">django.middleware.http.SetRemoteAddrFromForwardedFor</code> in
<code>MIDDLEWARE_CLASSES</code>. This allows your Django application to use the real
IP address of the client instead of <code>127.0.0.1</code> from your nginx proxy. It
sets Django's <code>request.META['REMOTE_ADDR']</code>
to be <code>request.META['HTTP_X_FORWARDED_FOR']</code> which we set above in nginx's
<code>proxy.conf</code>. For more information, see the
<a href="http://docs.djangoproject.com/en/dev/ref/middleware/#reverse-proxy-middleware">
Middleware reference</a> in the Django documentation.
</p>
<pre>MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.middleware.http.SetRemoteAddrFromForwardedFor',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
)</pre>
<h4>References</h4>
<ul>
<li><a href="http://bart.whahay.net/blog/2009/04/06/setting-up-nginx-django.html">
http://bart.whahay.net/blog/2009/04/06/setting-up-nginx-django.html</a> (my main reference)</li>
<li><a href="http://articles.slicehost.com/2009/3/5/ubuntu-intrepid-nginx-configuration">
http://articles.slicehost.com/2009/3/5/ubuntu-intrepid-nginx-configuration</a> (general nginx configuration)</li>
<li><a href="http://articles.slicehost.com/2009/3/6/ubuntu-intrepid-nginx-virtual-hosts-1">
http://articles.slicehost.com/2009/3/6/ubuntu-intrepid-nginx-virtual-hosts-1</a> (nginx virtual hosts configuration)</li>
<li><a href="http://forum.slicehost.com/comments.php?DiscussionID=2964">
http://forum.slicehost.com/comments.php?DiscussionID=2964</a> (explains the
difference between <code>root</code> and <code>alias</code> in nginx.conf.)</li>
</ul>
<h4>Errors</h4>
<ul>
<li>A 502 Bad Gateway from Nginx probably means there is something wrong with your Apache setup</li>
<li>An Internal Server Error from Apache means something is probably wrong with your Django
setup.</li>
</ul>
Scripting wmii column widths with Python
2009-04-12T23:05:12-07:00https://www.saltycrane.com/blog/2009/04/scripting-wmii-column-widths-python/<p>I mentioned in my previous post on
<a href="http://www.saltycrane.com/blog/2009/03/using-wmii-gnome/">
using wmii with Gnome</a> that I had written a script for resizing
the column widths in
<a href="http://wmii.suckless.org/">wmii</a>. This is the followup post.
Note, I am using the 20080520 snapshot of wmii. This doesn't work with wmii
3.6 (as Marco commented below).
</p>
<p>To incrementally change window sizes, I use the following in my
<code>~/.wmii-3.5/wmiirc</code> file:</p>
<pre> Key $MODKEY-y
# shrink horizontally
wmiir xwrite /tag/sel/ctl grow sel sel right -10
wmiir xwrite /tag/sel/ctl grow sel sel left -10
Key $MODKEY-u
# grow horizontally
wmiir xwrite /tag/sel/ctl grow sel sel right 10
wmiir xwrite /tag/sel/ctl grow sel sel left 10
Key $MODKEY-i
# shrink vertically
wmiir xwrite /tag/sel/ctl grow sel sel down -10
wmiir xwrite /tag/sel/ctl grow sel sel up -10
Key $MODKEY-o
# grow vertically
wmiir xwrite /tag/sel/ctl grow sel sel down 10
wmiir xwrite /tag/sel/ctl grow sel sel up 10
Key $MODKEY-Shift-y
# shrink horizontally
wmiir xwrite /tag/sel/ctl grow sel sel right -2
wmiir xwrite /tag/sel/ctl grow sel sel left -2
Key $MODKEY-Shift-u
# grow horizontally
wmiir xwrite /tag/sel/ctl grow sel sel right 2
wmiir xwrite /tag/sel/ctl grow sel sel left 2
Key $MODKEY-Shift-i
# shrink vertically
wmiir xwrite /tag/sel/ctl grow sel sel down -2
wmiir xwrite /tag/sel/ctl grow sel sel up -2
Key $MODKEY-Shift-o
# grow vertically
wmiir xwrite /tag/sel/ctl grow sel sel down 2
wmiir xwrite /tag/sel/ctl grow sel sel up 2</pre>
<p>In addition to incrementally changing column widths, I wanted to
be able to switch to predetermined column width ratios with a keyboard
shortcut. For example, I wanted to be able to set the column widths
at a 20/80 ratio, a 40/60 ratio, a 50/50 ratio, a 60/40 ratio, and so on.
So I hacked a Python script to do this. <del>It is pretty ugly because
I first grow the window by a set amount, measure the change in size,
then grow it again to the correct width. If anyone knows of a better
way to do this, please let me know.</del> I'm posting my solution here
in case anyone else wanted to do the same thing and got stuck.
<del><em>(Note, this script only works with two columns)</em></del>
<p><em>UPDATE 2009-12-21: I just learned from the
<a href="http://wmii.googlecode.com/hg/doc/wmii.pdf">new wmii documentation</a>
that I can specify a grow amount in pixels by suffixing it with "px".
This means I no longer have to perform the ugly, extra grow-then-measure
step in my script. I'm not sure if this is a newly added change or if it
is just newly documented. I am now using wmii 3.9b1. I have updated the
script below to use the new method. Also, the script now works with more
than two columns. I kept the old method for reference.</em></p>
</p>
<pre class="python">#!/usr/bin/env python
import os
import re
import sys
class Wmii:
"""
wmiir xwrite /tag/sel/ctl grow col row side increment
col: column number of the window to grow
row: row number of the window to grow
side: the side to grow. one of left, right, up, or down
increment: the number of pixels to grow. use a positive number to grow larger
and a negative number to grow smaller
"""
def set_column_widths(self, width_list):
"""Use the 'grow' command to set the column widths to those specified.
Widths are specified in percentages.
"""
total_width_perc = sum([float(width) for width in width_list])
for i, width_perc in enumerate(width_list[:-1]):
self.read_current_col_widths()
total_width_px = float(sum(self.curr_colwidths))
new_width_px = float(width_perc) / total_width_perc * total_width_px
grow_amount_px = int(round(new_width_px - self.curr_colwidths[i]))
self.xwrite("/tag/sel/ctl grow %d 1 right %dpx" % (i+1, grow_amount_px))
def read_current_col_widths(self):
"""'wmiir read /tag/sel/index' and set the attribute, self.curr_colwidths.
self.curr_colwidths is a list of the width (ints) (in pixels) of each
column in the view.
"""
lines = self.read("/tag/sel/index")
self.curr_colwidths = []
for line in lines:
match = re.search(r"# [^~]+ \d+ (\d+)", line)
if match:
self.curr_colwidths.append(int(match.group(1)))
print self.curr_colwidths
def xwrite(self, path_and_value):
"""Use the xwrite form."""
cmd = "wmiir xwrite %s" % path_and_value
print cmd
os.system(cmd)
def read(self, path):
"""Return a list of the lines returned by "wmii read path" """
return os.popen4("wmiir read " + path)[1].readlines()
if __name__ == "__main__":
w = Wmii()
w.set_column_widths(sys.argv[1:])</pre>
<b>Old method (for reference):</b>
<pre class="python" style="overflow: auto; height: 100px">#!/usr/bin/env python
import os
import re
import sys
class Wmii:
"""
wmiir xwrite /tag/sel/ctl grow col row side increment
col: column number of the window to grow
row: row number of the window to grow
side: the side to grow. one of left, right, up, or down
increment: the number of pixels to grow. use a positive number to grow larger
and a negative number to grow smaller
"""
def __init__(self):
pass
def set_column_widths(self, width0, width1):
"""Use the 'grow' command to set the column widths to those specified.
Widths are specified in percentages.
Currently only works with 2 columns.
"""
self.determine_pixels_per_grow_horiz()
new_width0 = sum(self.curr_colwidths) * (float(width0) /
(float(width0)+float(width1)))
grow_amount = int(round((new_width0-self.curr_colwidths[0]) /
self.pixels_per_grow_increment))
self.xwrite("/tag/sel/ctl grow 1 1 right %d" % grow_amount)
def determine_pixels_per_grow_horiz(self):
"""Try growing by an increment of 1 and record the number of pixels changed.
"""
self.read_current_col_widths()
prev_colwidth0 = self.curr_colwidths[0]
self.xwrite("/tag/sel/ctl grow 1 1 right 1")
self.read_current_col_widths()
self.pixels_per_grow_increment = self.curr_colwidths[0] - prev_colwidth0
def read_current_col_widths(self):
"""'wmiir read /tag/sel/index' and set the attribute, self.curr_colwidths.
self.curr_colwidths is a list of the width (ints) (in pixels) of each
column in the view.
"""
lines = self.read("/tag/sel/index")
self.curr_colwidths = []
for line in lines:
match = re.search(r"# [^~]+ \d+ (\d+)", line)
if match:
self.curr_colwidths.append(int(match.group(1)))
print self.curr_colwidths
def read_current_column_number(self):
"""'wmiir read /tag/sel/ctl' and set the attribute, self.curr_col."""
lines = self.read("/tag/sel/ctl")
self.curr_col = re.split(" ", lines[1])[1]
print "curr_col = %s" % self.curr_col
def xwrite(self, path_and_value):
"""Use the xwrite form."""
cmd = "wmiir xwrite %s" % path_and_value
print cmd
os.system(cmd)
def read(self, path):
"""Return a list of the lines returned by "wmii read path" """
return os.popen4("wmiir read " + path)[1].readlines()
if __name__ == "__main__":
w = Wmii()
w.set_column_widths(sys.argv[1], sys.argv[2])</pre>
<p>I named the script <code>wmii.py</code>, made it executable,
and put it on my path. Then I modified my <code>wmiirc</code>:</p>
<pre> Key $MODKEY-q
wmii.py 20 80
Key $MODKEY-w
wmii.py 40 60
Key $MODKEY-e
wmii.py 50 50
Key $MODKEY-r
wmii.py 60 40
Key $MODKEY-t
wmii.py 80 20</pre>
<p>Hope that's helpful to someone. Let me know if you've written some cool
python wmii scripts.</p>
Ubuntu notes
2009-03-25T22:14:25-07:00https://www.saltycrane.com/blog/2009/03/ubuntu-notes/<h4>Boot process</h4>
<ul>
<li>What are the stages of the boot process?<br>
<ol>
<li>BIOS</li>
<li>Master Boot Record</li>
<li>Bootloader (LILO, GRUB)</li>
<li>Kernel</li>
<li>init</li>
<li>Run Levels</li>
</ol>
<br>
</li>
<li>Where is the GRUB menu list?<br>
<code>/boot/grub/menu.lst</code>
<br><br>
</li>
<li>How do I get rid of Ubuntu splash screen / progress bar and show scrolling text?<br>
Remove <code>quiet</code> and <code>splash</code> from "kernel" line in
<code>/boot/grub/menu.lst</code>
<br><br>
</li>
</ul>
See also:<br>
<a href="http://en.wikipedia.org/wiki/Linux_startup_process">
http://en.wikipedia.org/wiki/Linux_startup_process</a><br>
<a href="http://www.debianhelp.co.uk/boot.htm">
http://www.debianhelp.co.uk/boot.htm</a><br><br>
<h4>Run levels</h4>
<ul>
<li>What are the run levels for Ubuntu?<br>
1: single user<br>
2: default<br>
3-5: same as 2<br>
6: reboot
<br><br>
</li>
<li>Where can I find the default startup programs for run level 2?<br>
<code>/etc/rc2.d</code>
<br><br>
</li>
<li>How do I update the symbolic links in <code>/etc/rc?.d</code>?<br>
Use <code>update-rc.d</code>. For help, <code>man update-rc.d</code>.
<br><br>
</li>
<li>Where is the default run level set in Ubuntu?<br>
<code>/etc/event.d/rc-default</code>
<br><br>
</li>
</ul>
<h4>Xsessions</h4>
<ul>
<li>Where is another location for startup programs?<br>
<code>/etc/X11/Xsession.d</code>
<br><br>
</li>
</ul>
<h4>Restarting</h4>
<ul>
<li>What are some alternatives to rebooting?
<ol>
<li>Restart the display manager, gdm<br>
<code>sudo /etc/init.d/gdm restart</code> or <code>CTRL+ALT+BACKSPACE</code>
</li>
<li>Switch to runlevel 1 then back to runlevel 2<br>
<code>sudo telinit 1</code> then <code>telinit 2</code> (this didn't work for me;
I wasn't able to get to runlevel 1.)
</li>
<li>This article has a lot of good tips including Alt+SysReq+"<b>R</b>aising <b>S</b>kinny <b>E</b>lephants <b>I</b>s <b>U</b>tterly <b>B</b>oring" -
<a href="http://www.linuxclues.com/articles/09.htm">http://www.linuxclues.com/articles/09.htm</a>
</li>
</ol>
<br>
</li>
<li>How can I reboot?<br>
<code>reboot</code> or <code>CTRL+ALT+DEL</code>
<br><br>
</li>
</ul>
<h4>Keeping Ubuntu time up to date</h4>
<p>I would think Ubuntu would install something to keep the system clock from
drifting, but it seems they haven't. There are at least 2 options: ntpdate and
ntpd. Ntpdate is used from a cron script to sync the time once a day. Ntpd runs
as a daemon and ajusts the time continually but it costs resources. On my machine,
using <code>ps -e -O rss | grep ntpd</code>, I got 1.3 MB of memory use.
Here is how to install ntpd:
</p>
<pre>sudo apt-get install ntp</pre>
<p>The ntp config file is located at: <code>/etc/ntp.conf</code></p>
<p>Reference: <a href="https://help.ubuntu.com/9.04/serverguide/C/NTP.html">
https://help.ubuntu.com/9.04/serverguide/C/NTP.html</a></p>
<h4>My /etc/apt/sources.list</h4>
<p>This sources.list includes security, updates, and proposed (but not backports) and source for the main, restricted, universe, and multiverse repositories.</p>
<pre>deb http://archive.ubuntu.com/ubuntu/ jaunty main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jaunty main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ jaunty-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jaunty-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ jaunty-updates restricted main universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jaunty-updates restricted main universe multiverse
deb http://archive.ubuntu.com/ubuntu/ jaunty-proposed restricted main multiverse universe
deb-src http://archive.ubuntu.com/ubuntu/ jaunty-proposed restricted main multiverse universe</pre>
<h4>My filesystem changes</h4>
<ul>
<li>For wmii:
<pre>sudo ln -s /usr/include/freetype2/freetype /usr/include/freetype</pre>
</li>
</ul>
<h4>System config file changes:</h4>
<p>These changes apply to Ubuntu Karmic</p>
<ul>
<li>Disable xsplash before session load:
Comment out the following from <code>/etc/gdm/PreSession/Default</code>:
<pre>if [ -x '/usr/bin/xsplash' ];
then
/usr/bin/xsplash --daemon
fi</pre>
</li>
<li>To run ~/.xsession from gdm:
Create a '.desktop' file in '/usr/share/xsessions' [owned by root:root with permissions 644] whose 'Exec' entry is '/etc/X11/Xsession'.
</li>
</ul>
</ul>
<h4>How to show what you've installed with apt-get</h4>
<p>Haven't found a way that I'm happy with yet. Here are some options:</p>
<ul>
<li><pre>sudo apt-get install apt-show-versions
apt-show-versions</pre>
</li>
<li><pre>dpkg --get-selections</pre>
</li>
<li><pre>dpkg -l </pre></li>
<li>
<pre>cat /var/log/dpkg.log | grep installed </pre>
</li>
</ul>
<h4>Sound troubleshooting</h4>
<ul>
<li>Make sure speakers are plugged in / power strip on. (For some reason, the power to my speakers keeps going out.</li>
<li>Run gnome-volume-control. Make sure volume is up and not muted. (For some reason on Karmic this resets to mute on restart.)</li>
</ul>
<h4>Skype microphone troubleshooting</h4>
<p>Running Skype 2.1.0.47-1 on Ubuntu Karmic</p>
<ul>
<li>Run gnome-volume-control. Select Microphone 1. Raise the volume up to almost Max.</li>
<li>(Testing can be done with gnome-sound-recorder.)</li>
</ul>
<h4>Flash</h4>
<ul>
<li>Karmic w/ Firefox 3.5.5
<p>Can't click in Flash - <a href="http://ubuntuforums.org/showthread.php?t=1314910">http://ubuntuforums.org/showthread.php?t=1314910</a>
</p>
</li>
</ul>
<h4>Emacs keybindings</h4>
<p>Copy <code>/usr/share/themes/Emacs/gtk-2.0-key/gtkrc</code> to <code>~/.gtkrc-2.0</code>. Edit as necessary.</p>
<h4>(Hardware) System Information</h4>
<pre>sudo apt-get install sysinfo</pre>
<h4>custom xsession</h4>
<p>Edit <code>/usr/share/xsessions/user.desktop</code></p>
<pre>[Desktop Entry]
Name=~/.xsession
Comment=User's .xsession
Exec=/etc/X11/Xsession
TryExec=/etc/X11/Xsession
Icon=
Type=Application</pre>
<h4>Can't create test file error</h4>
<p>Trying to move datadir from /var/lib/mysql to /mnt/mysql-data on EC2</p>
<p><a href="http://ubuntuforums.org/showthread.php?t=782224">
http://ubuntuforums.org/showthread.php?t=782224</a>
</p>
<p>Had to edit <code>/etc/apparmor.d/usr.sbin.mysqld</code></p>
<h4>Troubleshooting X freezes</h4>
<ul>
<li><a href="https://wiki.ubuntu.com/X/Troubleshooting/Freeze">Troubleshooting X freezes, hangs, and lockups</a></li>
<li><a href="https://wiki.ubuntu.com/X/Troubleshooting/HalBreaksKeyboardAndMouse">Troubleshooting keyboard and mouse failure due to HAL breakage</a></li>
</ul>
<h4>Thinkpad W510 suspend</h4>
<a href="http://math.umons.ac.be/an/W510/#ACPI">http://math.umons.ac.be/an/W510/#ACPI</a><br>
<a href="https://bugs.launchpad.net/ubuntu/+source/linux/+bug/522998">https://bugs.launchpad.net/ubuntu/+source/linux/+bug/522998</a><br>
For Ubuntu 10.10:<br>
<code>/etc/pm/config.d/unload_modules</code><br>
<pre>SUSPEND_MODULES="xhci_hcd"</pre>
<p>Note: in 10.04, I had "xhci" instead of "xhci_hcd"</p>
<h4 id="cisco-vpn">Cisco VPN</h4>
<pre class="console">$ sudo apt-get install network-manager-vpnc </pre>
<ul>
<li>Click on network manager icon in gnome-panel</li>
<li>VPN Connections -> Cofigure VPN...</li>
<li>Import</li>
<li>Select file from my company</li>
<li>Set user name</li>
<li>Apply</li>
<li>Close</li>
<li><b>Restart newtork-manager</b>
<pre class="console">$ sudo /etc/init.d/network-manager restart </pre>
</li>
<li>Click on network manager icon in gnome-panel</li>
<li>VPN Connections -> My Company</li>
<li>Enter password</li>
<li>Done</li>
</ul>
<h4 id="vpn-only-its-network">VPN change to gain access to important stuff on our network</h4>
<pre> - click network icon
- "VPN Connections" > "Configure VPN..."
- highlight "My Company VPN"
- "Edit"
- "IPv4 Settings" tab
- "Routes..."
- uncheck "Use this connection only for resources on its network"
- "OK"
- "Apply"
- "Close"</pre>
<h4 id="vpnc">When the above vpn setup didn't work with my Verizon mifi</h4>
<p>...using <code>vpnc</code> from the command line worked:
<a href="http://www.debuntu.org/how-to-connect-to-a-cisco-vpn-using-vpnc">http://www.debuntu.org/how-to-connect-to-a-cisco-vpn-using-vpnc</a>
See also: <a href="http://www.chicagotech.net/routingissuesonvpn.htm">http://www.chicagotech.net/routingissuesonvpn.htm</a>.
</p>
<p>/etc/vpnc/mycompany.conf:</p>
<pre>IPSec gateway vpn.abc.mycompany.com
IPSec ID MyCompanyVPN
IPSec secret mygrouppassword
#IKE Authmode hybrid
Xauth username myusername</pre>
<p>To use it, I run:</p>
<pre class="console">$ sudo vpnc mycompany </pre>
<p>To disconnect</p>
<pre class="console">$ sudo vpnc-disconnect </pre>
<h4 id="vpnc-no-response">vpnc: no response from target error</h4>
<pre>vpnc: no response from target</pre>
<p>I added the following line to /etc/vpnc/mycompany.conf:</p>
<pre>NAT Traversal Mode cisco-udp</pre>
<p>Connection really slow and nothing happens: disabled encryption.</p>
<p>Debugging:</p>
<pre class="console">$ sudo vpnc --debug 2 --no-detach mycompany </pre>
<p>Help:</p>
<pre class="console">$ man vpnc </pre>
<h4 id="vpnc-timeout">Prevent vpnc from periodically disconnecting due to timeout</h4>
<p>See <a href="http://superuser.com/questions/309723/vpnc-it-keeps-disconnecting-after-24-minutes">http://superuser.com/questions/309723/vpnc-it-keeps-disconnecting-after-24-minutes</a></p>
<p>Start vpnc with the <code>--dpd-idle 0</code> option. e.g.:</p>
<pre class="console">$ sudo vpnc --debug 2 --dpd-idle 0 mycompany </pre>
<h4 id="set-timezone">Set the timezone on a server</h4>
<p>run as root</p>
<pre class="console"># dpkg-reconfigure tzdata </pre>
<h4 id="set-hostname">Set the hostname on a server</h4>
<p>run as root</p>
<pre class="console"># echo "plato" > /etc/hostname
# hostname -F /etc/hostname </pre>
<p>Edit /etc/hosts:</p>
<pre>127.0.0.1 localhost.localdomain localhost
12.34.56.78 plato.example.com plato</pre>
<h4 id="claws-mail-html">HTML messages in Claws Mail</h4>
<ul>
<li>Install extra plugins
<pre class="console">$ sudo apt-get install claws-mail-extra-plugins </pre></li>
<li>Load the Fancy HTML plugin<br>
Configuration -> Plugins.. -> Load... -> fancy.so</li>
<li>Configure preferences<br>
Configuration -> Preferences... -> Message View/Text Options, under "HTML messages", uncheck the first box and check the second and third boxes.<br>
Configuration -> Preferences... -> Plugins/Fance, check "Auto-Load images" and uncheck "Block external content"
</li>
</ul>
<h4 id="claws-mail-folders">How to set the folders for Sent, Drafts, and Trash</h4>
<p>Ensure that all folders (e.g. "Sent Items", "Deleted Items") are available:</p>
<ul>
<li>Configuration -> Preferences for current account... -> Receive</li>
<li>Uncheck "Show subscribed folders only"</li>
</ul>
<p>Set the folders:</p>
<ul>
<li>Configuration -> Preferences for current account... -> Advanced</li>
<li>Select the appropriate folders</li>
<li>OK</li>
</ul>
<h4 id="claws-mail-keyboard-shortcuts">Set custom keyboard shortcuts for Claws Mail</h4>
<p>You can also modify each menu shortcut by pressing any key(s) when focusing the mouse pointer on the item.
</p>
<h4 id="notification-location">Change the location of the popup notification</h4>
<a href="http://askubuntu.com/questions/130124/how-can-i-make-notifications-appear-on-a-specific-monitor">
http://askubuntu.com/questions/130124/how-can-i-make-notifications-appear-on-a-specific-monitor</a>
<pre>$ sudo apt-get install dconf-tools
$ dconf-editor & </pre>
<p>apps -> notify-osd. Change multihead-mode to "focus-follow"</p>
<h4 id="suspend-command-line">How to suspend from the command line</h4>
<a href="http://manpages.ubuntu.com/manpages/precise/man8/pm-action.8.html">
http://manpages.ubuntu.com/manpages/precise/man8/pm-action.8.html</a>
<pre class="console">$ sudo pm-suspend </pre>
<h4 id="switching-ttys">Switching TTYs</h4>
<p>The default TTY on Ubuntu is TTY7</p>
<p>How to determine which TTY you're currently on:</p>
<pre class="console">$ sudo fgconsole
7</pre>
<p>How to switch to TTY1 from the command line<p>
<pre class="console">$ sudo chvt 1 </pre>
<p>How to switch to TTY1 by pressing a key combo:<p>
<p>CTRL + LEFT ALT + F1</p>
<p>How to switch to TTY7 by pressing a key combo:<p>
<p>CTRL + LEFT ALT + F7</p>
<h4 id="install-claws-mail-ubuntu-precise">Install Claws Mail on Ubuntu 12.04</h4>
<p>The claws-mail package included in Ubuntu 12.04 has a bug that doesn't allow
you to click any buttons in one of the Filtering dialogs. See
<a href="http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2624">
http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2624</a>.
To fix, install the PPA from the Claws Mail team:
<a href="https://launchpad.net/~claws-mail/+archive/ppa">https://launchpad.net/~claws-mail/+archive/ppa</a>
</p>
<pre class="console">$ sudo add-apt-repository ppa:claws-mail/ppa </pre>
<pre class="console">$ sudo apt-get update </pre>
<pre class="console">$ sudo apt-get install claws-mail </pre>
<p>That did not work. So install from source:</p>
<pre>
1. sudo apt-get build-dep claws-mail
2. download claws mail source
3. untar
4. ./configure --prefix=/home/eliot/lib/claws-mail-3.8.1
5. make
6. make install
</pre>
<p>That didn't work either.</p>
<p>To get filtering working, I copied the filtering rules file: <code>~/.claws-mail/matcherrc</code>.
Here is an example file:</p>
<pre>[#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/BP]
enabled rulename "Mark as read" all mark_as_read
[#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/INBOX]
enabled rulename "Archive" age_greater 90 move "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Archive/Inbox"
[#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Serverconfig]
enabled rulename "Archive" age_greater 90 move "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Archive/Serverconfig"
[preglobal]
[postglobal]
[filtering]
disabled rulename "Test (copy to)" all copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/BP" set_tag "inbox"
enabled rulename "Serverconfig (copy to)" from matchcase "serverconfig@mycompany.com" copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Serverconfig"
enabled rulename "Jira (copy to)" from matchcase "jira@mycompany.com" copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Jira"
enabled rulename "Confluence (copy to)" from matchcase "confluence@mycompany.com" copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Confluence"
enabled rulename "Jenkins (move to)" from matchcase "jenkins@mycompany.com" move "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Jenkins"
disabled rulename "Mark non-inbox messages as read" ~tag matchcase "inbox" mark_as_read
enabled rulename "Build Announce" from matchcase "Build Annouce <no-reply@mycompany.com>" move "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Build Announce"
enabled rulename "Database builder" from matchcase "\"database_builder@mycompany.com\" <database_builder@mycompany.com>" copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Database builder"
enabled rulename "Daily News (copy to)" from matchcase "dailynews@mycompany.com" copy "#imap/saltycrane@mycompany.com@owa013.msoutlookonline.net/Daily News"</pre>
<h4 id="printer-settings">Printer Settings from the command line Ubuntu 12.04</h4>
<pre class="console">$ system-config-printer & </pre>
<p>Ref: <a href="http://askubuntu.com/questions/147019/how-can-i-learn-the-command-which-will-start-an-application-in-unitys-system-s">http://askubuntu.com/questions/147019/how-can-i-learn-the-command-which-will-start-an-application-in-unitys-system-s</a></p>
<h4 id="command-line-settings">Settings from the command line Ubuntu 12.04</h4>
<pre class="console">$ gnome-control-center & </pre>
<h4 id="resize-image-command-line">How to resize an image from the command line</h4>
<p>Install imagemagick</p>
<pre class="console">$ sudo apt-get install imagemagick </pre>
<p>Resize an image</p>
<pre class="console">$ convert 2012-10-31_14-49-38_807.jpg -resize 50% 2012-10-31_14-49-38_807-halfsize.jpg </pre>
<h4 id="determine-network-adapter">How to determine what type of (wireless) network adapter (card) you have</h4>
<p>Reference: <a href="http://ubuntuforums.org/showthread.php?t=1734728">http://ubuntuforums.org/showthread.php?t=1734728</a></p>
<pre class="console">$ sudo lshw -c network </pre>
<h4 id="network-tools">Network Tools</h4>
<pre>
mtr - mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.
ntop web based tool - ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
nmap - scan your server for open ports.
iptraf
netstat
ss
traceroute
</pre>
<h4 id="nvidia-settings">nvidia-settings 'BadValue (integer parameter out of range for operation)'</h4>
<p>I got the following error when trying to use <code>nvidia-settings</code> after
an <code>apt-get dist-upgrade</code>:
</p>
<pre>The program 'nvidia-settings' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadValue (integer parameter out of range for operation)'.
(Details: serial 739 error_code 2 request_code 139 minor_code 25)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)</pre>
<p>Googling told me that the new version of nvidia-settings was broken.
<code>dpkg</code> told me that I had version 331 installed:
</p>
<pre class="console">$ dpkg -l | grep nvidia
ii nvidia-304 304.116-0ubuntu0.0.1 NVIDIA binary Xorg driver, kernel module and VDPAU library
ii nvidia-settings 331.20-0ubuntu0.0.1 Tool for configuring the NVIDIA graphics driver</pre>
<p><code>apt-cache</code> told me there were 3 versions available:</p>
<pre class="console">$ apt-cache showpkg nvidia-settings | tail -5
Provides:
331.20-0ubuntu0.0.1 -
304.88-0ubuntu0.0.2 -
295.33-0ubuntu1 -
Reverse Provides: </pre>
<p>I installed the 304 version and it worked:</p>
<pre class="console">$ sudo apt-get install nvidia-settings=304.88-0ubuntu0.0.2 </pre>
<h4 id="apt-get-version">How to install a specific version of a package with apt-get and prevent it from being upgraded with apt-get update</h4>
<br>
<p>Install specific version:</p>
<pre class="console">$ sudo apt-get install nvidia-settings=304.88-0ubuntu0.0.2 </pre>
<p>Prevent it from being upgraded:</p>
<pre class="console">$ sudo apt-mark hold nvidia-settings </pre>
<h4 id="install-emacs">Install emacs 24 on Ubuntu 12.04</h4>
<a href="http://askubuntu.com/questions/297170/how-to-install-emacs-24-3-on-ubuntu">http://askubuntu.com/questions/297170/how-to-install-emacs-24-3-on-ubuntu</a>
<pre>$ sudo add-apt-repository ppa:cassou/emacs
$ sudo apt-get update
$ sudo apt-get install emacs24 </pre>
<h4 id="ctrl-space-emacs">Control-Space doesn't work in Emacs in Ubuntu 14.04</h4>
<a href="http://askubuntu.com/questions/243639/ctrlspace-has-been-bound-to-invoke-some-input-method-and-does-not-work-in-ema">http://askubuntu.com/questions/243639/ctrlspace-has-been-bound-to-invoke-some-input-method-and-does-not-work-in-ema</a>
<ul>
<li><pre>$ unity-control-center </pre></li>
<li>Go to Keyboard -> Shortcuts -> Typing</li>
<li>Remove Control+Space keybinding</li>
</ul>
<h4 id="ports">List commands using ports</h4>
<pre class="console">$ lsof -i :5001
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python 11851 eliot 3u IPv4 845825 0t0 TCP *:5001 (LISTEN)</pre>
<h4 id="cisco-anyconnect-vpn">Cisco AnyConnect VPN</h4>
<p>Install Openconnect:</p>
<pre class="console">$ sudo apt-get install openconnect </pre>
<p>Connect:</p>
<pre class="console">$ sudo openconnect vpn.mycompany.com </pre>
<h4 id="disable-alt-and-super-key">How to disable the ALT and SUPER keys in Unity 15.04</h4>
<pre class="console">$ sudo apt-get install compizconfig-settings-manager </pre>
<pre>
1. Desktop
2. Ubuntu Unity Plugin
3. General
4. Key to show the menu bar while pressed. <- Disable it.
5. Launcher
6. Key to show the Dash, Launcher and Help Overlay. <- Disable it.
</pre>
<h4>Install Flash</h4>
<a href="http://askubuntu.com/questions/531672/how-to-install-flash-payer-in-ubuntu-14-04-lts">http://askubuntu.com/questions/531672/how-to-install-flash-payer-in-ubuntu-14-04-lts</a>
Using wmii with Gnome
2009-03-19T11:15:17-07:00https://www.saltycrane.com/blog/2009/03/using-wmii-gnome/<style>
.super {
vertical-align: super;
font-size: 85%;
}
.fn {
font-size: 85%;
}
</style>
<p>
<a href="http://glyph.twistedmatrix.com/2009/01/you-got-your-windowmaker-in-my-peanut.html">
Thanks to Glyph</a>, I can now use
<a href="http://wmii.suckless.org/">wmii</a> as the
<a href="http://en.wikipedia.org/wiki/Window_manager">window manager</a> for
<a href="http://www.gnome.org/">Gnome</a>. I like wmii because it makes good use
of screen real estate, is keyboard-navigatable, is
scriptable<a class="super" name="tt1" href="#ss1">1</a>, and uses
minimal resources<a class="super" name="tt2" href="#ss2">2</a>.
</p>
<p>
It is possible to use
<a href="http://live.gnome.org/GnomePanel">gnome-panel</a> within wmii-- just press
<code><em>MOD+P gnome-panel RET</em></code>. And you can manually fine tune your
<code>.xinitrc</code> or <code>.xsession</code> by adding your required stuff
such as <code>gnome-power-manager</code> and <code>nm-applet</code>. (This is what
I had been doing (and may continue to do depending on how this full on Gnome thing
works out).)
</p>
<p>
If, for some strange reason (e.g.
you want to use an annoying<a class="super" name="tt3" href="#ss3">3</a>
<a href="http://www.adobe.com/products/air/">Adobe Air</a> application), you want
to use the full Gnome desktop with wmii, that is possible also. To do this,
start up Gnome and go to <em>System, Preferences, Sessions</em>. Uncheck
<em>Window Manager</em>, then click <em>Add</em> and fill in the path
to your wmii executable. Glyph also suggests running <code>gconf-editor</code>
and turning off <em>/apps/nautilus/preferences/show_desktop</em>. After logging
out and logging in, you should now have wmii as the window manager for Gnome.
</p>
<p>Note, I've only used this Gnome+wmii configuration about a day, so I'm not sure if
I will keep it or not. I have found
<a href="http://www.tweetdeck.com/">TweetDeck</a> will load up but has some disappearing
act problems. <a href="http://www.twitterlocal.net/">TwitterLocal</a> on the
other hand, seems to run fine.
</p>
<p><em>Update 2010-07-18:</em> I only used the full Gnome+wmii configuration for a couple days before going back to pure wmii. However, I am retrying this configuration with Ubuntu 10.04 Lucid again since I have a more powerful computer now and I need extra non-hacker type stuff to just work without trying to figure out which program provides it... So it looks like Ubuntu 10.04 no longer has the Sessions item in the Preferences menu. I tried using <code>gconf-editor</code> and changin <code>/desktop/gnome/session/required_components/windowmanager</code> from <code>gnome-wm</code> to the path to my wmii executable but that didn't work. If you know how to do it, let me know.
</p>
<hr>
<ol class="fn">
<li>
<a name="ss1"></a>Yes I have written one script for wmii. It allows me to change column widths
using the keyboard instead of the mouse. It's a bit of a hack, but maybe I will
make another post for that sometime. <em>Update 2009-04-12: <a href="/blog/2009/04/scripting-wmii-column-widths-python/">here is that post</a></em> <a href="#tt1">↩</a>
</li>
<li>
<a name"ss2"></a> Of course, using Gnome with wmii negates this advantage. When I bought my Dell
530N with 1GB of RAM, I had planned to purchase more RAM separately because it was
so much cheaper. I suppose I ought to do that sometime. Of course at work, my
computer had only half a gig of RAM and I was running out of memory continually.
<a href="#tt2">↩</a>
</li>
<li>
<a name="ss3"></a> I call Air annoying because it only works in Gnome or KDE (and to add another
alliterative adjective). Otherwise, I don't know much about it.
<a href="#tt3">↩</a>
</li>
</ol>
Card store project #5: Redirecting my www-prefixed domain to my non-www-prefixed domain
2008-12-28T16:53:07-08:00https://www.saltycrane.com/blog/2008/12/card-store-project-5-redirecting-my-www-prefixed-domain-my-non-www-prefixed-domain/<p>For search engine optimization and analytics, I wanted to make
<a href="http://www.handsoncards.com/">http://www.handsoncards.com/</a>
permanently redirect to <a href="http://handsoncards.com/">http://handsoncards.com/</a>.
So I started Googling. It took me a while to figure this one out... The first few articles I read
suggested using Rewrite directives in my <code>.htaccess</code> file but this didn't work for me
(maybe because I'm using mod_python and Django?). Then I found I could use
Rewrite in my <code>httpd.conf</code> file, but I got an infinite redirect loop.
Finally, I found the solution from <a href="http://www.webmasterworld.com/apache/3217088.htm">
this discussion</a>. Here's what I did. I'm using Django, mod_python, and
Apache on Ubuntu at Slicehost.
</p>
<ul>
<li>Enable mod_rewrite. Logged in as root:
<pre>a2enmod rewrite</pre>
</li>
<li>Edit <code>/etc/apache2/httpd.conf</code>:
<pre><VirtualHost *>
ServerName handsoncards.com
ServerAlias www.handsoncards.com
#
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.handsoncards\.com
RewriteRule (.*) http://handsoncards.com$1 [R=301,L]
#
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE handsoncards.settings
PythonPath "['/srv/HandsOnCards', '/srv/python-packages'] + sys.path"
PythonDebug Off
</VirtualHost></pre>
</li>
<li>Restart Apache:
<pre>/etc/init.d/apache2 restart</pre>
</li>
</ul>
Postgres backup with cron
2008-12-27T12:36:36-08:00https://www.saltycrane.com/blog/2008/12/postgres-backup-cron/<p>So I accidentally deleted my Postgres database on my public server instead
of my local server last night. Luckily, I was still in the testing phase.
I had been meaning to figure out how to backup Postgres but had been putting
it off. Now I've been provided the proper motivation.</p>
<p>The PostgreSQL documentation has a whole
<a href="http://www.postgresql.org/docs/8.3/interactive/backup.html">
chapter devoted to backup</a>. There it describes three methods of
backup: SQL dump, file system level backup, and continuous archiving.
I chose SQL dump.</p>
<h4>Manual SQL dump</h4>
<p>To do a manual dump, here is what I did. (As you might guess,
handsoncards_db is my database name.)</p>
<pre>su postgres
pg_dump handsoncards_db > /tmp/testdump
exit</pre>
<h4>Schedule backups with cron</h4>
<p>To perform backups at regular intervals I used cron.
Logged in as root, I created a file
<code>handsoncards_db_backup</code> in <code>/etc/cron.d</code>.
(Note, I'm running Ubuntu Intrepid. Cron will automatically start
running this new job without a restart.)
<pre>
# m h dom mon dow user command
45 3 * * * postgres pg_dump handsoncards_db > /srv/backups/handsoncards_db_backup
</pre>
<p>This will create a backup at 3:45am every day.
Be sure to put a newline at the end of the file.</p>
<p>Finally I created the backup directory and made postgres the owner.</p>
<pre>mkdir -p /srv/backups
chown postgres:postgres /srv/backups</pre>
<h4>Restore a database from the dump file</h4>
<p>If necessary, delete the old database. Then create a new databse and restore
from the dump file.</p>
<pre>su postgres
psql template1</pre>
<pre>CREATE DATABASE handsoncards_db OWNER django_user ENCODING 'UTF8';
\q</pre>
<pre>psql handsoncards_db < /srv/backups/handsoncards_db_backup
exit</pre>
<p>NOTE: if you get a permission denied error when trying to restore, check the
Unix permissions on the backup file and all the parent directories.</p>
Card store project #2: Installing Satchmo, Django, PostgreSQL, and Apache on Ubuntu at Slicehost
2008-12-11T21:21:22-08:00https://www.saltycrane.com/blog/2008/12/card-store-project-2-installing-satchmo-django-postgresql-and-apache-ubuntu-slicehost/<p>As I mentioned in my
<a href="http://www.saltycrane.com/blog/2008/12/card-store-project-1-plans/">previous
post</a>, I'm planning to set up an e-commerce site using the
<a href="http://www.satchmoproject.com/">Satchmo</a> shopping cart framework.
Satchmo is built on the
<a href="http://www.satchmoproject.com/">Django</a> web framework which is
written in the
<a href="http://www.python.org/">Python</a> programming language. Satchmo
has a lot of <a href="http://www.satchmoproject.com/docs/svn/features.html">features</a>
built in which means it saves you a lot of work implementing them yourself.
Check out <a href="http://www.youtube.com/watch?v=d42a4g650Ws">this video
introduction to Satchmo</a> at this year's DjangoCon for more information.</p>
<p>After reading <a href="http://groups.google.com/group/satchmo-users/browse_thread/thread/b453d6453b6a28cf?pli=1">
this discussion</a> on the Satchmo mailing list, I decided to use
<a href="http://www.slicehost.com/">Slicehost</a> for hosting my site.
Their cheapest plan provides 256MB of RAM for $20/month.
Here are my notes on setting up Satchmo and Django with
<a href="http://www.postgresql.org/">PostgreSQL</a>,
<a href="http://httpd.apache.org/">Apache</a>,
and <a href="http://www.modpython.org/">mod_python</a> on
<a href="http://www.ubuntu.com/">Ubuntu</a> Intrepid at Slicehost.
It seems <a href="http://www.lighttpd.net/">lighttpd</a> and
<a href="http://nginx.net/">nginx</a> are popular lightweight alternatives to
Apache on VPS setups. I do not know too much about this, but may explore these
in the future.
</p>
<p>Note, I don't describe how to setup a Satchmo project below. Maybe I will
write another post about that later... For more information, see the
<a href="http://www.satchmoproject.com/docs/rel/0.8.1/new_installation.html">
Satchmo Installation documentation</a>. <em>Update 2008-12-12: I did write
another blog post-- see <a href="http://www.saltycrane.com/blog/2008/12/card-store-project-3-installing-satchmo-part-2/">
Installing Satchmo, part 2</a>.</em>
</p>
<h4>Setup Slicehost</h4>
<ul>
<li>Sign up for an account at <a href="http://www.slicehost.com/">http://www.slicehost.com/</a>
I chose Ubuntu Intrepid for my operating system and named my slice "toad".
</li>
<li>I followed the following two excellent Slicehost tutorials:
<a href="http://articles.slicehost.com/2008/4/25/ubuntu-hardy-setup-page-1">Ubuntu Setup Part 1</a>
and <a href="http://articles.slicehost.com/2008/4/25/ubuntu-hardy-setup-page-2">Part 2</a>.
</li>
<li>An important step is setting the system locale. Since I'm in the United States,
I used the following commands (run as root):
<pre>locale-gen en_US.UTF-8
update-locale LANG=en_US.UTF-8</pre>
</li>
</ul>
<h4>Install apache, mod_python, postgresql, and postgres python bindings</h4>
<ul>
<li>Run as root (or use sudo):
<pre>apt-get update
apt-get upgrade
apt-get dist-upgrade
apt-get install apache2 apache2-mpm-prefork
apt-get install libapache2-mod-python
apt-get install postgresql
apt-get install python-psycopg2</pre>
</li>
</ul>
<h4>Configure apache</h4>
<ul>
<li>I used the following Slicehost tutorials:
<a href="http://articles.slicehost.com/2008/4/25/ubuntu-hardy-installing-apache-and-php5">Install Apache</a>,
<a href="http://articles.slicehost.com/2008/4/28/ubuntu-hardy-apache-configuration-1">Configure Apache Part 1</a>,
<a href="http://articles.slicehost.com/2008/4/28/ubuntu-hardy-apache-configuration-2">Configure Apache Part 2</a>
</li>
<li>Of particular note, when installing apache, I got this warning:
<pre>apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName</pre>
Fix it by setting the ServerName in <code>/etc/apache2/apache2.conf</code>:
<pre>ServerName yourdomain.com
ServerTokens Prod</pre>
See the <a href="http://httpd.apache.org/docs/2.2/mod/core.html#servername">Apache
docs</a> for more information on ServerName.
</li>
</ul>
<h4>Install Django 1.0.2</h4>
<ul>
<li>Run the following as root:
<pre>cd /srv
wget http://www.djangoproject.com/download/1.0.2/tarball/
tar zxvf Django-1.0.2-final.tar.gz
mkdir python-packages
cd python-packages
ln -s ../Django-1.0.2-final/django</pre>
</li>
</ul>
<h4>Test Django (optional)</h4>
<ul>
<li>Run the following as root:
<pre>export PYTHONPATH=/srv/python-packages
/srv/Django-1.0.2-final/django/bin/django-admin.py startproject testdjango</pre>
</li>
<li>Edit <code>/etc/apache2/httpd.conf</code>:
<pre><location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE testdjango.settings
PythonPath "['/srv', '/srv/python-packages'] + sys.path"
PythonDebug On
</location></pre>
</li>
<li>Restart apache:
<pre>/etc/init.d/apache2 restart</pre>
Navigate to your slice's IP address in your browser and you
should see the Django "It worked!" page.
</li>
</ul>
<h4>Install Satchmo</h4>
<p>Satchmo requires a number of external Python packages besides
Django. Unfortunately, there isn't one standard way of installing
Python packages. I decided to install most of them using Ubuntu's
APT package management and one of them using
<a href="http://peak.telecommunity.com/DevCenter/EasyInstall">easy_install</a>,
and one of them from source.
Alternatively, I could have installed most of the packages with
easy_install, and one of them with APT (and one of them from source).
Or, I could have done it another way... sigh.
</p>
<p>Note, I use the directory <code>/srv/python-packages</code> to
store symbolic links to my needed python packages. Then I add this
directory to my Python Path in the PYTHONPATH environment variable or in
the Apache <code>httpd.conf</code> file. See my post,
<a href="http://www.saltycrane.com/blog/2008/08/somewhere-your-python-path/">
Somewhere on your Python path</a> for more information.
</p>
<ul>
<li>Install Satchmo prerequisites. Run the following commands as root:
<br><b>Install APT packages:</b>
<pre>apt-get update
apt-get upgrade
apt-get install python-crypto
apt-get install python-yaml
apt-get install python-imaging
apt-get install python-reportlab
apt-get install python-trml2pdf</pre>
<br><b>Install django-registration using easy_install:</b>
<pre>apt-get install python-setuptools python-dev build-essential
easy_install django-registration</pre>
<br><b>Install comment_utils source:</b>
<pre>apt-get install subversion
mkdir -p /srv/python-packages/dist
cd /srv/python-packages/dist
svn co http://django-comment-utils.googlecode.com/svn/trunk/comment_utils/
mv comment_utils comment_utils_rev92
cd /srv/python-packages
ln -s dist/comment_utils_rev92 comment_utils</pre>
</li>
<li>Install Satchmo 0.8. Run the following as root: <em>Update 2008-12-12:
I changed this to use release 0.8 instead of the SVN trunk version.</em>
<pre>cd /srv
wget http://www.satchmoproject.com/snapshots/satchmo-0.8.tgz
tar zxvf satchmo-0.8.tgz
cd /srv/python-packages
ln -s ../satchmo-0.8/satchmo</pre>
To install the SVN trunk version instead,
<pre>cd /srv
svn co svn://satchmoproject.com/satchmo/trunk
mv trunk satchmo_revXXXX
cd /srv/python-packages
ln -s ../satchmo_revXXXX/satchmo</pre>
</li>
</ul>
<h4 id="postgres">Setup PostgreSQL</h4>
<p>I followed the
<a href="http://www.punteney.com/writes/setting-django-slicehost-ubuntu-hardy-postgres-apa/">instructions
at Punteney.com</a> for setting up Django and Postgres on Slicehost.
</p>
<ul>
<li>If you didn't already install Postgres above, run the following as root:
<pre>apt-get install postgresql
apt-get install python-psycopg2</pre>
</li>
<li>Change the password for the "postgres" Unix user. (run as root or use sudo)
<pre>passwd -d postgres
su postgres -c passwd</pre>
</li>
<li>Change password for "postgres" user in the database.
(From Punteney's article, "It's convenient
to have the two passwords match, but not required.")
<pre>su postgres -c psql template1</pre>
<pre>ALTER USER postgres WITH PASSWORD 'postgres_user_password';
template1=\q</pre>
</li>
<li>Create django user:
<pre>su postgres
createuser -P django_user</pre>
Punteney says to answer no to all here.
Remember the password to put in Django settings.py file. (For this
example, I set the password to <code>my_password</code>.)
</li>
<li>Create a database. Still <code>su</code>'ed as the "postgres" user:
<pre>psql template1</pre>
<pre>CREATE DATABASE django_db OWNER django_user ENCODING 'UTF8';
\q</pre>
<pre>exit</pre>
(exit from postgres su)
</li>
<li>Configure access to the database. Edit
<code>/etc/postgresql/8.3/main/pg_hba.conf</code>:
<pre>local all postgres ident sameuser
local django_db django_user md5</pre>
</li>
<li>Restart the postgres server:
<pre>/etc/init.d/postgresql-8.3 restart</pre>
</li>
<li>In your Django project, use the following database settings in
<code>settings.py</code>:
<pre>DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'django_db'
DATABASE_USER = 'django_user'
DATABASE_PASSWORD = 'my_password'
DATABASE_HOST = ''
DATABASE_PORT = ''</pre>
</li>
<li>Run <code>syncdb</code>:
<pre>cd /srv/yourproject
python manage.py syncdb</pre>
</li>
</ul>
<h4>Setup a domain name</h4>
<ul>
<li>Register for a domain name. Google for "domain registration" for
options.
</li>
<li>Follow the
<a href="http://articles.slicehost.com/2007/10/24/creating-dns-records">
Slicehost directions</a> for creating DNS records. This includes
setting the nameservers at your domain registration service to use
Slicehost's nameservers (ns1.slicehost.net, ns2.slicehost.net, ns3.slicehost.net).
</li>
</ul>
Card store project #1: Plans
2008-12-08T11:09:35-08:00https://www.saltycrane.com/blog/2008/12/card-store-project-1-plans/<p>I'm planning to start an online greeting card / stationary store with my wife.
My wife (and possibly my sister
and other friends) will create designs for the cards, while
I'll run the website and manage the business. I'll admit, I don't know
the first thing about running a business, but, for a number of reasons, I still
want to try. If you have any advice on running an online business, I'd appreciate
any feedback. Here are some of my plans for the store. I've already started on some
of them and will write more details as I complete them.
</p>
<h4>General plans</h4>
<p>My general plan is to have my wife, and possibly other artists, draw
card designs using colored pencils, ink, and other media. I'll scan
the designs, lay out the cards, and print them. We had thought about doing
custom, hand-made cards for each order, but my wife doesn't have a lot of
time, and I didn't want to burden her with making tens of millions of cards
when our store hits the big time tomorrow.
I plan to make the cards
on-demand, with quick turn around times by automating as much of the process
as possible. In the beginning we'll do things manually, because I don't
expect many orders. Then later, I'll add <a href="http://www.python.org/">Python</a>
scripts,
<a href ="http://twistedmatrix.com/trac/">Twisted</a>
Perspective Broker communication, and custom robotic cutting, folding, and
mail delivery. Finally, I plan to <a href="http://en.wikipedia.org/wiki/Pinky_and_the_Brain">
take over the world</a>.
</p>
<h4>Desktop publishing plans</h4>
<p>A degree in Art, or Graphic Design would probably
be a whole lot more useful than my degree in Engineering. If I did have one of those
degrees, I would probably be doing most of my work on a Mac with Photoshop,
Illustrator, InDesign, etc. But, since I'm a geeky engineer and programmer,
I'm going to use Linux, the <a href="http://www.gimp.org/">Gimp</a>,
and deal with all the hardware/software
incompatibilities that come with it. Besides Gimp,
I plan to use <a href="http://www.scribus.net/">Scribus</a>, a pretty cool
open-source desktop publishing program. For printing, I've already purchased
a <a href="http://www.office.xerox.com/printers/color-printers/phaser-8560/enus.html">
Xerox Phaser 8560DN</a> solid-ink printer. I've done some preliminary
printing, and there has been some pain, as my wife can attest, but overall
it is a pretty cool printer that works well with Linux. The biggest hurdle,
I think, will be figuring out the ICC profiles / color management stuff.
I'll post my Linux Phaser 8560 setup notes when I have everything in order.
</p>
<h4>Business-related plans</h4>
<p>I plan to run the business as a <a href="http://en.wikipedia.org/wiki/Sole_proprietorship">
sole proprietorship</a> since it is the simplest of the
<a href="http://www.sba.gov/smallbusinessplanner/start/chooseastructure/START_FORMS_OWNERSHIP.html">
business structure options</a>. As far as I can tell from preliminary reading,
I will need to get a city business license, a sales tax license, a state
employer identification number, and probably a
<a href="http://entrepreneurs.about.com/od/businessstructure/a/doingbusinessas.htm">
ficticious business name</a>. For more information, see the
<a href="http://www.sba.gov/smallbusinessplanner/index.html">
U.S. Small Business Administration guide</a>.
</p>
<h4>Website plans</h4>
<p>Regarding website plans, I need to do a lot of work to learn the front end
web development such as design, CSS, and Javascript. On the back end, I have
a little more experience, though, again, I still have a lot to learn. Here are
my website technical plans.
</p>
<ul>
<li><a href="http://www.satchmoproject.com/">Satchmo</a>:
Satchmo is an open source e-commerce framework built
on Python and
<a href="http://www.djangoproject.com/">Django</a>.
It is <a href="http://www.satchmoproject.com/blog/">actively
developed</a>, has a number of
<a href="http://www.satchmoproject.com/docs/svn/features.html">features</a>,
and, since it is written in Python/Django, is easily hackable should I have
custom needs.</li>
<li><a href="http://www.slicehost.com/">Slicehost</a> hosting: I originally thought
I could use <a href="http://www.webfaction.com/">Webfaction</a> shared hosting since
it has served me well for this blog. However, after reading
<a href="http://groups.google.com/group/satchmo-users/browse_thread/thread/b453d6453b6a28cf">
this discussion on Satchmo hosting</a>, I realized I needed more RAM and
decided to go with VPS hosting at Slicehost.</li>
<li>I plan
to use <a href="http://aws.amazon.com/s3/">Amazon S3</a> for media storage.
I gained experience with this service at work. Also, after reading
Adrian Holovaty's (somewhat dated) <a href="http://holovaty.com/blog/archive/2006/04/07/0927">
post on using Django and S3 for media files</a>, I implemented S3 storage
for this blog. With the beta release of <a href="http://aws.amazon.com/cloudfront/">
Amazon CloudFront</a>, performance should be even better.
</li>
<li>Other technologies: I'll stick with <a href="http://www.ubuntu.com/">Ubuntu</a>
for the
server OS. I'll start with <a href="http://www.apache.org/">Apache</a> for the
web server, since it recommended by Django. I may explore other options if
memory becomes a problem. Though I have slightly more experience with
<a href="http://www.mysql.com/">MySQL</a> from work, I plan to use
<a href="http://www.postgresql.org/">PostgreSQL</a> for the database,
again, since it is
recommended by Django and seems to be a better designed database. I don't know
very much about caching, but I plan to look into
<a href="http://www.danga.com/memcached/">memcached</a>,
<a href="http://nginx.net/">nginx</a>, and
<a href="http://www.squid-cache.org/">squid-cache</a>.
</li>
</ul>
<br>
<p>Well, those are my plans. I am starting off bright-eyed. Hopefully, the trials
of the road ahead will not wreck my spirit. Again, feedback is appreciated! Thanks.
</p>
Creating remote server nicknames with .ssh/config
2008-11-20T15:01:35-08:00https://www.saltycrane.com/blog/2008/11/creating-remote-server-nicknames-sshconfig/<p>Using the <code>~/.ssh/config</code> file is an easy way to give your
remote machines nicknames and reduce the number of keystrokes needed
to login with <code>ssh</code>, <code>rsync</code>,
<code>hg push</code>/<code>pull</code>/<code>clone</code>, access
files via <a href="http://www.gnu.org/software/tramp/">Emacs Tramp</a>
(Transparent Remote (file) Access, Multiple Protocol),
or use any other SSH-based tool. You can also set other ssh options such
as <code>IdentityFile</code>, <code>Port</code>, or <code>CompressionLevel</code>.
For more information and a full list of options, check out the man page for <code>ssh_config</code> or
<a href="http://kimmo.suominen.com/docs/ssh/#config">this article</a>
by Kimmo Suominen.
</p>
<p>Here is part of my <code>~/.ssh/config</code> file. It defines the
nicknames turk, tyran, tuna, and tally for some EC2 servers I've been
working with.</p>
<pre>Host turk
User root
HostName ec2-67-202-21-122.compute-1.amazonaws.com
Host tuna
User root
HostName ec2-75-101-178-62.compute-1.amazonaws.com
Host tyran
User root
HostName ec2-67-202-43-207.compute-1.amazonaws.com
Host tally
User root
HostName ec2-67-202-59-207.compute-1.amazonaws.com</pre>
<p>Now, wherever I would normally have typed
<code>root@ec2-67-202-21-122.compute-1.amazonaws.com</code>,
I can just type <code>turk</code>. Here are some examples.</p>
<h4>SSH login</h4>
<p>Old way:</p>
<pre>ssh root@ec2-67-202-21-122.compute-1.amazonaws.com</pre>
<p>New way:</p>
<pre>ssh turk</pre>
<h4>rsync</h4>
<p>Old way:</p>
<pre>rsync -avz myproject root@ec2-67-202-21-122.compute-1.amazonaws.com:/srv</pre>
<p>New way:</p>
<pre>rsync -avz myproject turk:/srv</pre>
<h4>Mercurial</h4>
<p>Old way:</p>
<pre>hg push ssh://root@ec2-67-202-21-122.compute-1.amazonaws.com//srv/myproject</pre>
<p>New way:</p>
<pre>hg push ssh://turk//srv/myproject</pre>
<h4>Emacs Tramp</h4>
<p>To use your <code>~/.ssh/config</code> with Emacs Tramp, you will need
something like the following in your <code>.emacs</code>:</p>
<pre>(tramp-set-completion-function "ssh"
'((tramp-parse-sconfig "/etc/ssh_config")
(tramp-parse-sconfig "~/.ssh/config")))</pre>
<p>Old way:</p>
<pre>C-x C-f /root@ec2-67-202-21-122.compute-1.amazonaws.com:/srv/myproject/myfile.py</pre>
<p>New way:</p>
<pre>C-x C-f /turk:/srv/myproject/myfile.py</pre>
<h4>scp</h4>
<p>Old way:</p>
<pre>scp etc/.screenrc root@ec2-67-202-21-122.compute-1.amazonaws.com:/root</pre>
<p>New way:</p>
<pre>scp etc/.screenrc turk:/root</pre>
Installing beanstalkd and pybeanstalk on Ubuntu
2008-10-10T10:06:25-07:00https://www.saltycrane.com/blog/2008/10/installing-beanstalkd-and-pybeanstalk-ubuntu/<p><a href="http://nikolay.com">Nikolay</a> pointed me at
<a href="http://xph.us/software/beanstalkd/">beanstalkd</a>,
a lightweight, <a href="http://en.wikipedia.org/wiki/Message_queue">message
queue</a> service partially inspired by the popular caching system,
<a href="http://www.danga.com/memcached/">memcached</a>. It features a
blocking "reserve" call so workers don't need to poll for new jobs.
However, some might miss the data persistence since the queue is stored
in memory.
beanstalkd has client
libraries in Erlang, Perl, PHP, Python, and Ruby. Of course, I will use
the Python version, <a href="http://code.google.com/p/pybeanstalk/">pybeanstalk</a>.
Simon Willison also <a href="http://simonwillison.net/2008/May/12/beanstalkd/">
found beanstalkd interesting</a>. (Credit to him
for some of the words above.)
</p>
<p><em>Update 2008-10-13:</em> For a helpful example on using
beanstalkd and pybeanstalk see
<a href="http://parand.com/say/index.php/2008/10/12/beanstalkd-python-basic-tutorial/">
Parand's beanstalkd tutorial</a>.
</p>
<p>Here is how I installed the beanstalkd server and pybeanstalk client on
Ubuntu. I found no Ubuntu package for beanstalkd, so I installed from source.
</p>
<h5>Install beanstalkd</h5>
<ul>
<li>Install prerequistes
<pre>$ sudo apt-get install libevent1 libevent-dev</pre>
</li>
<li>Download
<pre>$ cd ~/lib
$ wget http://xph.us/software/beanstalkd/rel/beanstalkd-1.0.tar.gz</pre>
</li>
<li>Unpack
<pre>$ tar zxvf beanstalkd-1.0.tar.gz</pre>
</li>
<li>Make
<pre>$ cd beanstalkd-1.0
$ make</pre>
</li>
<li>Print help
<pre>$ ./beanstalkd -h</pre>
</li>
</ul>
<h5>Install PyYAML</h5>
<ul>
<li>Download
<pre>$ wget http://pyyaml.org/download/pyyaml/PyYAML-3.06.tar.gz</pre>
</li>
<li>Unpack
<pre>$ tar zxvf PyYAML-3.06.tar.gz</pre>
</li>
<li>Put <code>PyYAML-3.06/lib/yaml</code>
<a href="http://www.saltycrane.com/blog/2008/08/somewhere-your-python-path/">
somewhere on your Python path</a> or run <code>python setup.py</code>.
</li>
</ul>
<h5>Install pybeanstalk</h5>
<ul>
<li>Download
<pre>$ wget http://pybeanstalk.googlecode.com/files/pybeanstalk-0.11.1.tar.gz</pre>
</li>
<li>Unpack
<pre>$ tar zxvf pybeanstalk-0.11.1.tar.gz</pre>
</li>
<li>Put <code>pybeanstalk-0.11.1/beanstalk</code> somewhere on your Python path
or run <code>python setup.py</code>
</li>
</ul>
<h5>Run beanstalkd server</h5>
<ul>
<li><pre>$ ~/lib/beanstalkd-1.0/beanstalkd -d -l 127.0.0.5 -p 11300</pre></li>
</ul>
<h5>Run test client</h5>
<ul>
<li>Create a file and run it:
<pre class="python">from beanstalk import serverconn
from beanstalk import job
SERVER = '127.0.0.5'
PORT = 11300
# setup connection
connection = serverconn.ServerConn(SERVER, PORT)
connection.job = job.Job
# produce data
for i in range(5):
print 'put data: %d' % i
data = job.Job(data=str(i), conn=connection)
data.Queue()
# consume data
while True:
j = connection.reserve()
print 'got data: %s' % j.data
j.Finish()</pre>
Results:
<pre>put data: 0
put data: 1
put data: 2
put data: 3
put data: 4
got data: 0
got data: 1
got data: 2
got data: 3
got data: 4</pre>
</li>
</ul>
Installing Python 2.6 from source on Ubuntu Hardy
2008-10-02T15:42:03-07:00https://www.saltycrane.com/blog/2008/10/installing-python-26-source-ubuntu-hardy/<p><a href="http://www.python.org/download/releases/2.6/">Python
2.6</a> was released yesterday! This version aims to
smooth the transition from Python 2.5 to Python 3.0 which is planned
for release soon (currently available as a
<a href="http://www.python.org/download/releases/3.0/">release candidate</a>).
Python 3.0 will be break backwards compatibility with
the 2.x series. Python 2.6 is backwards compatible with 2.5.
All the backwards compatible features of 3.0 have been backported to
2.6.</p>
<p> One of the new 2.6 features I'm particularly intersted in is
the new <a href="http://docs.python.org/dev/library/multiprocessing.html">
multiprocessing module</a> which has a similar interface to
the <a href="http://docs.python.org/dev/library/threading.html">
threading module</a>, but it uses processes instead of threads.
This avoids the limitations imposed by the Global Interpreter Lock
in a multi-threaded Python program. Yet it still has the nice
communications and management features like <code>Pipe()</code>s,
<code>Queue</code>s, <code>Pool</code>s, etc. I didn't plan to
focus so much on the multiprocessing module in this post-- I just want
to document my install notes on Ubuntu Linux. For all the new features
in 2.6 see <a href="http://docs.python.org/whatsnew/2.6.html">What's
New in Python 2.6</a>. (It is a long list).</p>
<ul>
<li>Download the Python 2.6 compressed source tarball
<pre>$ cd incoming
$ wget http://www.python.org/ftp/python/2.6/Python-2.6.tgz</pre>
</li>
<li>Unpack
<pre>$ tar zxvf Python-2.6.tgz</pre>
</li>
<li>Read the README at <code>~/incoming/Python-2.6/README</code><br><br>
</li>
<li>Install prerequisites (Disclaimer: I know nothing about libraries,
packages, dependencies, etc. This is what I did-- I am not sure
if I grabbed the correct prerequisites or not.)
<pre>$ sudo apt-get install build-essential
$ sudo apt-get install libncursesw5-dev
$ sudo apt-get install libreadline5-dev
$ sudo apt-get install libssl-dev
$ sudo apt-get install libgdbm-dev
$ sudo apt-get install libbz2-dev
$ sudo apt-get install libc6-dev
$ sudo apt-get install libsqlite3-dev
$ sudo apt-get install tk-dev</pre>
</li>
<li>Configure. I am installing to <code>~/lib/python2.6</code>.
<pre>$ cd Python-2.6
$ ./configure --prefix=/home/sofeng/lib/python2.6</pre>
</li>
<li>Make
<pre>$ make</pre>
Note I got the following message:
<pre>Failed to find the necessary bits to build these modules:
bsddb185 sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.</pre>
I was not able to find the Ubuntu packages for these.<br><br>
</li>
<li>Try it out <em>(Optional)</em>
<pre>$ ./python</pre>
<pre>Python 2.6 (r26:66714, Oct 2 2008, 15:32:46)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> </pre>
</li>
<li>Test <em>(Optional)</em>
<pre>$ make test</pre>
I got the following status:
<pre>327 tests OK.
33 tests skipped:
test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_gl test_imgfile test_kqueue test_linuxaudiodev test_macos
test_macostools test_normalization test_ossaudiodev test_pep277
test_py3kwarn test_scriptpackages test_socketserver test_startfile
test_sunaudiodev test_timeout test_urllib2net test_urllibnet
test_winreg test_winsound test_zipfile64
Those skips are all expected on linux2.</pre>
</li>
<li>Install
<pre>$ make install</pre>
</li>
<li>Finally, I added <code>~/lib/python/bin/python2.6</code> to my <code>PATH</code>
</li>
</ul>
<br>
<h5>Notes</h5>
<p>The first time I ran, I got this message:</p>
<pre>Failed to find the necessary bits to build these modules:
_hashlib _ssl bsddb185
bz2 gdbm readline
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.</pre>
<p>I think I found Ubuntu packages for some of them:</p>
<ul>
<li>_hashlib and _ssl in libssl-dev</li>
<li>bz2 in libbz2-dev</li>
<li>gdbm in libgdbm-dev</li>
<li>readline in libreadline5-dev</li>
</ul>
Installing Emacs 23 from <del>CVS</del>Bazaar source on Ubuntu <del>Hardy</del>Karmic
2008-10-01T17:10:10-07:00https://www.saltycrane.com/blog/2008/10/installing-emacs-23-cvs-ubuntu-hardy/<p>I have been using the <code>emacs-snapshot</code> package in Ubuntu Hardy.
However, when I tried to use <a href="http://www.gnu.org/software/tramp/">
Tramp</a>, I got an error message: <code>Variable binding depth exceeds
max-specpdl-size</code>. I couldn't find out how to fix this, and I
didn't want to use <a href="http://lists.gnu.org/archive/html/emacs-devel/2008-09/msg00213.html">
Emacs 22.3</a> because it doesn't have Xft (anti-aliased fonts),
so I decided to live on the bleeding edge and install Emacs 23
from CVS. Besides the INSTALL and INSTALL.CVS files, I also used
<a href="http://theblackdragon.wordpress.com/2006/04/27/installing-gnu-emacs-23-from-cvs/">
theBlackDragon's article</a> for reference.
</p>
<p><em>Update 2008-10-09:</em> I just found that Romain Francoise maintains an <a href="http://emacs.orebokech.com/">emacs-snapshot Debian package</a> and it has been
<a href="https://launchpad.net/~ubuntu-elisp/+archive">adapted for Ubuntu</a>.
This is an alternative to installing from source.</p>
<p><em>Update 2010-01-09:</em> Since Emacs has
<a href="http://lists.gnu.org/archive/html/emacs-devel/2009-12/msg00812.html">
switched from CVS to Bazaar</a>, I've updated these instructions to use
Bazaar. Also, I'm now running on Ubuntu 9.10 Karmic Koala instead of Ubuntu Hardy.</p>
<p><em>Update 2012-09-19:</em> Here are some additional Ubuntu 12.04 packages
I needed to install Emacs 24.2.</p>
<pre class="console">$ sudo apt-get install libgif-dev
$ sudo apt-get install libtiff4-dev
$ sudo apt-get install xaw3dg-dev
$ sudo apt-get install librsvg2-dev
$ sudo apt-get install libmagick++-dev
$ sudo apt-get install libgpm-dev
$ sudo apt-get install libgconf2-dev
$ sudo apt-get install libselinux1-dev
$ sudo apt-get install libm17n-dev
$ sudo apt-get install libotf-dev </pre>
<ul>
<li>Install Bazaar
<pre class="console">$ sudo apt-get install bzr</pre>
</li>
<li>Get the source code from the Bazaar repository
<pre class="console">$ cd ~/incoming
$ bzr branch http://bzr.savannah.gnu.org/r/emacs/trunk emacs_trunk</pre>
</li>
<li><p>Read the INSTALL and INSTALL.BZR files in <code>~/incoming/emacs_trunk</code></p>
</li>
<li>Install prerequisites:
<pre class="console">$ sudo apt-get install build-essential
$ sudo apt-get build-dep emacs23</pre>
<p>To see what is installed by <code>build-dep</code>, see the
<a href="http://ns2.canonical.com/source/karmic/emacs23">emacs23 karmic package
page</a></p>
</li>
<li>Configure. The argument <code>--prefix=/home/saltycrane/lib/emacs-bzr-20100210</code> means
I am installing Emacs in <code>/home/saltycrane/lib/emacs-bzr-20100210</code>.
<pre class="console">$ cd ~/incoming/emacs_trunk
$ ./configure --prefix=/home/saltycrane/lib/emacs-bzr-20100210</pre>
</li>
<li>Per the INSTALL.BZR file, I needed to do a <code>make bootstrap</code>
instead of <code>make</code> because some files, such as byte-compiled
lisp files are not stored in Bazaar. Note, this takes a long time (over 10 min for me).
<pre class="console">$ make bootstrap</pre>
</li>
<li>Make <em>(Optional)</em>
<pre class="console">$ make</pre>
</li>
<li>Test it <em>(Optional)</em>
<pre class="console">$ src/emacs -q</pre>
</li>
<li>Install
<pre class="console">$ make install</pre>
</li>
<li>Create symlinks (~/bin is already on my <code>PATH</code>)
<pre class="console">$ ln -s ~/lib/emacs-bzr-20100210 ~/lib/emacs
$ cd ~/bin
$ ln -s ../lib/emacs/bin/* .</pre>
<p>Alternatively, I could add <code>~/lib/emacs-bzr-20100210/bin</code> to my
<code>PATH</code>.</p>
</li>
</ul>
<p>Now I have "<a href="http://peadrop.com/blog/2007/01/06/pretty-emacs/">Pretty
Emacs</a>" with working
<a href="http://jeremy.zawodny.com/blog/archives/000983.html">Tramp
for remote file access</a>. It also has
<a href="http://lorentey.hu/project/emacs.html.hu">multi-tty</a> support
which <del>is supposed to be very cool (but I haven't tried it yet)</del>
is very cool (e.g. for displaying my running desktop emacs process on
my Android phone.)
</p>
<h5>Error messages</h5>
<ul>
<li>
<pre class="console">configure: error: You do not seem to have makeinfo >= 4.6, and your
source tree does not seem to have pre-built manuals in the `info' directory.
Either install a suitable version of makeinfo, or re-run configure
with the `--without-makeinfo' option to build without the manuals.</pre>
Solution:
<pre class="console">$ sudo apt-get install texinfo</pre>
</li>
<li>
<pre class="console">configure: error: The following required libraries were not found:
libjpeg libgif/libungif libtiff
Maybe some development libraries/packages are missing?
If you don't want to link with them give
--with-jpeg=no --with-gif=no --with-tiff=no
as options to configure</pre>
Solution:
<pre class="console">$ sudo apt-get install libjpeg-dev libgif-dev libtiff4-dev</pre>
</li>
<li>
<pre class="console">Warning: arch-dependent data dir (/home/saltycrane/lib/emacs-bzr/libexec/emacs/23.1.92/i686-pc-linux-gnu/) does not exist.
Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/site-lisp' does not exist.
Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/site-lisp' does not exist.
Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/lisp' does not exist.
Warning: Lisp directory `/home/saltycrane/lib/emacs-bzr/share/emacs/23.1.92/leim' does not exist.
Warning: Could not find simple.el nor simple.elc</pre>
<p>This happened because I originally used <code>./configure --prefix=/home/saltycrane/lib/emacs-bzr</code>
and then renamed <code>~/lib/emacs-bzr</code> to <code>~/lib/emacs-bzr-20100210</code>.
Solution: don't rename the directory.</p>
</li>
</ul>
Notes on Python deployment using Fabric
2008-09-28T00:24:21-07:00https://www.saltycrane.com/blog/2008/09/notes-python-deployment-using-fabric/<p>I found out about
<a href="http://www.nongnu.org/fab/">Fabric</a> via Armin Ronacher's article
<a href="http://lucumr.pocoo.org/cogitations/2008/07/17/deploying-python-web-applications/">
Deploying Python Web Applications</a>.
Fabric is a
<a href="http://www.capify.org/">Capistrano</a> inspired
deployment tool for the Python community. It is very simple
to use. There are 4 main commands: <code>local</code> is
almost like <code>os.system</code> because it runs a command
on the local machine, <code>run</code> and <code>sudo</code>
run a command on a remote machine as either a normal user
or as root, and <code>put</code> transfers a file to a remote
machine.</p>
<p>Here is a sample setup which displays information about
the Apache processes on my remote EC2 instance.
</p>
<ul>
<li><a href="http://www.saltycrane.com/blog/2007/01/how-to-install-easy-install-for-python/">
Install Easy Install</a></li>
<li>Install Fabric
<pre>$ sudo easy_install Fabric</pre></li>
<li>Create a file called <code>fabfile.py</code> located at <code>~/myproject</code>
<pre class="python">def ec2():
set(fab_hosts = ['ec2-65-234-55-183.compute-1.amazonaws.com'],
fab_user = 'sofeng',
fab_password = 'mypassword',)
def ps_apache():
run("ps -e -O rss,pcpu | grep apache")</pre>
Note: for security reasons, you can remove the password from the fabfile and
Fabric will prompt for it interactively. Per
<a href="http://www.nongnu.org/fab/user_guide.html">the documentation</a>,
Fabric also supports key-based authentication.<br><br>
</li>
<li>Run it
<pre>$ cd ~/myproject
$ fab ec2 ps_apache</pre>
Results:
<pre> Fabric v. 0.0.9, Copyright (C) 2008 Christian Vest Hansen.
Fabric comes with ABSOLUTELY NO WARRANTY; for details type `fab warranty'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `fab license' for details.
Running ec2...
Running ps_apache...
Logging into the following hosts as sofeng:
ec2-65-234-55-183.compute-1.amazonaws.com
[ec2-65-234-55-183.compute-1.amazonaws.com] run: ps -e -O rss,pcpu | grep apache
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2163 5504 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2520 15812 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2521 3664 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2522 3664 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2523 3664 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2524 3664 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2619 3664 0.0 S ? 00:00:00 /usr/sbin/apache2 -k start
[ec2-65-234-55-183.compute-1.amazonaws.com] out: 2629 1204 0.0 R ? 00:00:00 /bin/bash -l -c ps -e -O rss,pcpu | grep apache
Done.</pre>
</li>
</ul>
Notes on using EC2 command line tools
2008-09-27T20:52:48-07:00https://www.saltycrane.com/blog/2008/09/notes-using-ec2-command-line-tools/
<h5>Create AWS accounts</h5>
<ul>
<li>Create an AWS account at <a href="http://aws.amazon.com/">http://aws.amazon.com/</a>.</li>
<li>Create an AWS EC2 account at <a href="http://aws.amazon.com/ec2/">http://aws.amazon.com/ec2/</a>.
(You will need to enter a credit card number.)
</li>
</ul>
<h5>Create a X.509 Certificate</h5>
<p>Note: A X.509 Certificate is one type of Access Identifier. Access Identifiers
are used to <em>"identify yourself as the sender of a request to an AWS web service"</em>.
There are two types of access identifiers: AWS Access Key Identifiers and
X.509 Certificates. AWS Access Key Identifiers are supported by
all Amazon Web Services and X.509 Certificates are supported only by
Amazon's EC2 and SQS services (see <a href="http://aws-portal.amazon.com/gp/aws/developer/account/access-identifier-help.html">
here</a> for the chart). However, for some reason, the popular Java command
line tools for EC2 only support X.509 Certificates (and not AWS Access Key
Identifiers).</p>
<ul>
<li>From <a href="http://aws.amazon.com/account/">Your Account page</a>,
select <a href="http://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key">Access
Identifiers</a>.</li>
<li>In the "X.509 Certificate" section, click "Create New".
</li>
<li>Download both the "Private Key" file and the "X.509 Certificate" file
to the directory, <code>~/.ec2</code>. (The private key file will be
named something like pk-XXXXXXXXXXXXXXXXXXXXXX.pem and the X.509
Certificate file will be named something like
cert-XXXXXXXXXXXXXXXXXXXXXX.pem.)</li>
</ul>
<h5>Install Java</h5>
<p>The command line tools require Java version 5 or later. Only the
JRE is required.</p>
<ul>
<li><pre>$ sudo apt-get install sun-java6-jre</pre></li>
</ul>
<h5>Download Java Command-line Tools</h5>
<ul>
<li>Go to the <a href="http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88">
Amazon EC2 Command-Line Tools</a> library page, and
<a href="http://www.amazon.com/gp/redirect.html/ref=aws_rc_ec2tools?location=http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip&token=A80325AA4DAB186C80828ED5138633E3F49160D9">
Download the Amazon EC2 Command-Line Tools</a>.</li>
<li>Unzip the tools to <code>~/lib</code>
<pre>$ unzip ec2-api-tools.zip
mv ec2-api-tools-1.3-24159 ~/lib</pre>
</li>
</ul>
<h5>Define environment variables</h5>
<ul>
<li>Add the following lines to your <code>~/.bashrc</code> (or wherever
you set your environment variables).
<pre>export EC2_HOME=$HOME/lib/ec2-api-tools-1.3-24159
export JAVA_HOME=/usr
export EC2_PRIVATE_KEY=$HOME/.ec2/pk-XXXXXXXXXXXXXXXXXXXX.pem
export EC2_CERT=$HOME/.ec2/cert-XXXXXXXXXXXXXXXXXXXX.pem
export PATH=$PATH:$EC2_HOME/bin
</pre>
</li>
<li>Source your <code>.bashrc</code> or whichever file you used
<pre>$ source ~/.bashrc</pre>
</li>
</ul>
<h5>Test the command-line tools</h5>
<ul>
<li>Run the <code>ec2-describe-images</code> command to verify everything is working.
It should list all the Ubuntu 8.xx images from Alestic.
<pre>$ ec2-describe-images -a | grep alestic/ubuntu-8</pre>
Results:
<pre height='300px' style="height: 200px; overflow: auto">IMAGE ami-3a7c9953 alestic/ubuntu-8.04-hardy-base-20080419.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-75789d1c alestic/ubuntu-8.04-hardy-base-20080424.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-ce44a1a7 alestic/ubuntu-8.04-hardy-base-20080430.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-2048ad49 alestic/ubuntu-8.04-hardy-base-20080514.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-6a57b203 alestic/ubuntu-8.04-hardy-base-20080517.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-26bc584f alestic/ubuntu-8.04-hardy-base-20080628.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-179e7a7e alestic/ubuntu-8.04-hardy-base-20080803.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-c0fa1ea9 alestic/ubuntu-8.04-hardy-base-20080905.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-38d43051 alestic/ubuntu-8.04-hardy-base-20080922.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-1cd73375 alestic/ubuntu-8.04-hardy-base-20080924.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-337c995a alestic/ubuntu-8.04-hardy-desktop-20080419.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-4f789d26 alestic/ubuntu-8.04-hardy-desktop-20080424.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f744a19e alestic/ubuntu-8.04-hardy-desktop-20080430.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-1f4bae76 alestic/ubuntu-8.04-hardy-desktop-20080514.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-0e57b267 alestic/ubuntu-8.04-hardy-desktop-20080517.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-b5bc58dc alestic/ubuntu-8.04-hardy-desktop-20080628.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f39e7a9a alestic/ubuntu-8.04-hardy-desktop-20080803.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-44c4202d alestic/ubuntu-8.04-hardy-desktop-20080905.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f7d4309e alestic/ubuntu-8.04-hardy-desktop-20080922.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-88d733e1 alestic/ubuntu-8.04-hardy-desktop-20080924.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-bcbe5ad5 alestic/ubuntu-8.04-hardy-rightscale-20080701.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-27b95d4e alestic/ubuntu-8.04-hardy-rightscale-20080703.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-b1ea0ed8 alestic/ubuntu-8.04-hardy-rightscale-20080824.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-47c4202e alestic/ubuntu-8.04-hardy-rightscale-20080905.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f4d4309d alestic/ubuntu-8.04-hardy-rightscale-20080922.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-89d733e0 alestic/ubuntu-8.04-hardy-rightscale-20080924.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-dcbc58b5 alestic/ubuntu-8.10-intrepid-base-20080628.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-db9e7ab2 alestic/ubuntu-8.10-intrepid-base-20080804.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-9de105f4 alestic/ubuntu-8.10-intrepid-base-20080814.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-c3fa1eaa alestic/ubuntu-8.10-intrepid-base-20080905.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-3bd43052 alestic/ubuntu-8.10-intrepid-base-20080922.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-1ad73373 alestic/ubuntu-8.10-intrepid-base-20080924.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-b6bc58df alestic/ubuntu-8.10-intrepid-desktop-20080628.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-d69e7abf alestic/ubuntu-8.10-intrepid-desktop-20080804.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-d4e206bd alestic/ubuntu-8.10-intrepid-desktop-20080815.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-7dc22614 alestic/ubuntu-8.10-intrepid-desktop-20080908.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f5d4309c alestic/ubuntu-8.10-intrepid-desktop-20080922.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-b6d733df alestic/ubuntu-8.10-intrepid-desktop-20080924.manifest.xml 063491364108 available public i386 machine aki-a71cf9ce ari-a51cf9cc</pre>
</li>
</ul>
<h5>Generate a keypair</h5>
<p>In the second step, I generated a keypair as my X.509 Certificate. That was used
to identifiy myself to Amazon Web Services. Now I need to create another keypair
which is used to log into a running EC2 instance. (Note, there is exactly one
X.509 Certificate per user (i.e. AWS account), but a user can have many keypairs
used for logging into various EC2 instances.) See also the
<a href="http://docs.amazonwebservices.com/AWSEC2/2008-05-05/GettingStartedGuide/running-an-instance.html#generating-a-keypair">
Generating a keypair</a> section in the Getting Started Guide.
</p>
<ul>
<li>Generate the keypair. I named the keypair, <code>disco-keypair</code> because
I will use this keypair with EC2 instances used to try out
<a href="http://discoproject.org">Disco</a>.
<pre>$ ec2-add-keypair disco-keypair > ~/.ec2/id_rsa-disco-keypair
</pre>
</li>
<li>Set the permissions on the private key
<pre>chmod 600 ~/.ec2/id_rsa-disco-keypair</pre>
</li>
</ul>
<h5>Run an EC2 instance</h5>
<ul>
<li>Select an image to run. I used the <code>alestic/ubuntu-8.04-hardy-base-20080924</code>
image with image ID <code>ami-1cd73375</code>.
</li>
<li>Run the instance
<pre>$ ec2-run-instances -k disco-keypair ami-1cd73375</pre>
It should return something like:
<pre>RESERVATION r-568f5d3f 719606167433 default
INSTANCE i-339f3c5a ami-1cd73375 pending disco-keypair 0 m1.small 2008-09-28T00:50:35+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc</pre>
</li>
<li>Check the status of the running instance:
<pre>$ ec2-describe-instances</pre>
After a short period of time, it should return something like:
<pre>RESERVATION r-568f5d3f 719606167433 default
INSTANCE i-339f3c5a ami-1cd73375 ec2-75-101-200-13.compute-1.amazonaws.com ip-10-251-30-10.ec2.internal running disco-keypair 0 m1.small 2008-09-28T00:50:35+0000us-east-1c aki-a71cf9ce ari-a51cf9cc</pre>
Note the address <code>ec2-75-101-200-13.compute-1.amazonaws.com</code>. This
is the external address used to connect to the instance. Also note the instance
ID <code>i-339f3c5a</code>. This is needed to terminate the instance.
</li>
<li>Authorize access to the instance through ports 22 (ssh) and 80 (http)
<pre>$ ec2-authorize default -p 22
GROUP default
PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0</pre>
<pre>$ ec2-authorize default -p 80
GROUP default
PERMISSION default ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0</pre>
</li>
</ul>
<h5>SSH into instance</h5>
<ul>
<li>Use the address from the previous step to SSH into your instance:
<pre>$ ssh -i ~/.ec2/id_rsa-disco-keypair -l root ec2-75-101-200-13.compute-1.amazonaws.com</pre>
</li>
</ul>
<h5>Terminate the instance</h5>
<ul>
<li><pre>$ ec2-terminate-instance i-339f3c5a</pre>
which returns:
<pre>INSTANCE i-339f3c5a running shutting-down</pre>
</li>
<li>Running <code>ec2-describe-instances</code> shows that the
instance is terminated.
<pre>$ ec2-describe-instances
RESERVATION r-568f5d3f 719606167433 default
INSTANCE i-339f3c5a ami-1cd73375 terminated disco-keypair 0 m1.small 2008-09-28T00:50:35+0000 aki-a71cf9ce ari-a51cf9cc</pre>
</li>
</ul>
How to monitor an Apache web server using Monit
2008-09-22T20:47:22-07:00https://www.saltycrane.com/blog/2008/09/how-monitor-apache-web-server-using-monit/<p><a href="http://www.tildeslash.com/monit/">Monit</a>
is a tool that can monitor your Apache web server,
MySQL database, or other daemon process. It can restart
the service based on configurable conditions such as
CPU usage, memory usage, number of children, etc. It can log status to a file,
email status, and it has a web interface for monitoring or
restarting the service. Here are the steps I took to install and configure
the Monit tool on Ubuntu Hardy. It merely monitors the
status of my Apache web server and restarts it if it stops. It also
checks if the memory used by Apache is greater than 1 MB and logs
it in /var/log/monit.log.
For more configuration options, see the examples in the default
<code>/etc/monit/monitrc</code> file or the
<a href="http://www.tildeslash.com/monit/doc/manual.php#configuration_examples">
configuration examples in the monit documentation</a>. I also
found <a href="http://www.ubuntugeek.com/monitoring-ubuntu-services-using-monit.html">
Ubuntu Geek's guide</a> to be very helpful.
</p>
<ul>
<li>Install monit<pre>$ sudo apt-get install monit</pre></li>
<li>Edit the config file<pre>$ sudo nano /etc/monit/monitrc</pre>
Insert the following:
<pre># check services every 2 minutes
set daemon 120
# logging
set logfile /var/log/monit.log
# web interface
set httpd port 2812 and
use address localhost # only accept connection from localhost
allow localhost # allow localhost to connect to the server
allow admin:monit # require user ‘admin’ with password ‘monit’
# monitor apache
check process apache2 with pidfile /var/run/apache2.pid
start program = "/etc/init.d/apache2 start"
if totalmem > 1.0 MB for 2 cycles then alert</pre>
</li>
<li>Check the file syntax<pre>$ sudo monit -t</pre></li>
<li>Enable the service<pre>$ sudo nano /etc/default/monit</pre>
Change the following line:
<pre>startup=1</pre>
</li>
<li>Start monit<pre>$ sudo /etc/init.d/monit start</pre></li>
<li>Point your browser at <a href="http://localhost:2812">http://localhost:2812</a>
and log in using the user "admin" and the password "monit".
</li>
<li>Click on "apache2" and you can see information about the Apache process.
</li>
</ul>
Simple cron example
2008-09-15T16:55:19-07:00https://www.saltycrane.com/blog/2008/09/simple-cron-example/<p>Simple cron example (tested on Ubuntu):</p>
<ul>
<li>Edit your (user) crontab file
<pre>$ crontab -e</pre>
This will bring up your editor (nano by default in Ubuntu)
<br><br>
</li>
<li>Enter the following inside. This will append the current date to a log file every minute. The 6 fields of the crontab file are: minute, hour, day of month, month, day of week, command.
<pre>* * * * * /bin/date >> /tmp/cron_output
</pre>
<b>Be sure to put a blank line at the end of the file.</b><br>
<em>(NOTE 1: <code>>></code> only redirects STDOUT to a file. To redirect both STDOUT and
STDERR, use something like <code>/bin/date >> /tmp/cron_output 2>&1</code>)</em><br>
<em>(NOTE 2: If output is not redirected, cron will try to email the output to you.
To do this, a mail transfer agent such as <a href="http://en.wikipedia.org/wiki/Sendmail">sendmail</a>
or <a href="http://en.wikipedia.org/wiki/Postfix_(software)">postfix</a> must be installed.)</em><br>
<em>(NOTE 3 (added 2015-06-24): When I created my cron script in <code>/etc/cron.d</code> with Emacs using
<code>sudo::</code>, cron didn't pick up my script. When I created it with nano, cron picked it up.
It seems the cause is the permissions of the cron script. Emacs created the script with 664 permissions
while nano created the script with 644 permissions. When I changed the permissions to 644, it started
working. I am running Ubuntu 15.04. This <a href="http://askubuntu.com/a/251468/423429">Ask Ubuntu answer</a>
confirms that a 644 permission is problematic because it is considered insecure. See /var/log/syslog
for cron messages. The Ask Ubuntu page has a lot of other good tips:
<a href="http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work">
Reasons why crontab does not work</a>)
</em><br>
<br>
</li>
<li>Exit the editor. It should output:
<pre>crontab: installing new crontab</pre>
</li>
<li>Check that it is working:
<pre>tail -f /tmp/cron_output</pre>
You should see the date updated every minute on the minute (or close to it):
<pre>Tue Sep 16 23:58:01 PDT 2008
Tue Sep 16 23:59:01 PDT 2008
Wed Sep 17 00:00:01 PDT 2008
Wed Sep 17 00:01:01 PDT 2008
...
</pre>
</li>
</ul>
<p>
See also my post: <a href="/blog/2008/12/postgres-backup-cron/">Postgres backup with cron</a>
</p>
Notes on Django and MySql on Amazon's EC2
2008-08-30T03:08:25-07:00https://www.saltycrane.com/blog/2008/08/notes-django-and-mysql-amazons-ec2/<h5>Install Elasticfox</h5>
<p>Install the Elasticfox Firefox Extension for Amazon EC2:
<a href="http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609">
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609</a>
</p>
<h5>Set up Amazon EC2 accounts and Elasticfox</h5>
<p>Follow
<a href="http://arope99.blogspot.com/2008/05/getting-started-with-amazon-elastic.html">
Arope's instructions for setting up Amazon EC2 accounts
and Elasticfox</a>. I used the
alestic/ubuntu-8.04-hardy-base-20080628.manifest.xml machine
image.
</p>
<h5>view standard apache page</h5>
<p>In Elasticfox, right-click on your running instance and select
"Copy Public DNS Name to clipboard". Then, paste that address
in your browser. You should see Apache's "It works!" page.
</p>
<h5>ssh into instance</h5>
<p>In Elasticfox, right-click on your running instance and select
"SSH to Public Domain Name"</p>
<h5>install stuff</h5>
<p>Ubuntu Hardy has the following versions:</p>
<ul>
<li>Apache 2.2.8</li>
<li>Mod_python 3.3.1</li>
<li>MySql 5.0.51</li>
<li>Django 0.96.1</li>
</ul>
<br>
<p>On your remote instance, do the following.</p>
<pre># apt-get update
# apt-get install python-django
# apt-get install mysql-server
# apt-get install python-mysqldb
# apt-get install libapache2-mod-python</pre>
<p><em>Update 2008-09-09</em>: The
<a href="http://www.djangoproject.com/documentation/modpython/">Django mod_python
documentation</a> recommends using Apache's
<a href ="http://httpd.apache.org/docs/2.2/mod/prefork.html">prefork MPM</a> as opposed
to the <a href="http://httpd.apache.org/docs/2.2/mod/worker.html">worker MPM</a>. The
worker MPM was installed by default on my Alestic Ubuntu image so I uninstalled it and replaced it
with the prefork version.</p>
<pre># apt-get autoremove --purge apache2-mpm-worker
# apt-get install apache2-mpm-prefork</pre>
<p>To see your current version of Apache, run the command:
<code>apache2 -V</code></p>
<h5>create a django project</h5>
<pre># cd /srv
# django-admin startproject mysite</pre>
<h5>configure django mod_python</h5>
<p>See also Jeff Baier's article:
<a href="http://www.jeffbaier.com/2007/07/26/installing-django-on-an-ubuntu-linux-server/">
Installing Django on an Ubuntu Linux Server</a>
for more information.
</p>
<p>Edit <code>/etc/apache2/httpd.conf</code> and insert the
following:</p>
<pre><location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mysite.settings
PythonPath "['/srv'] + sys.path"
PythonDebug On
</location></pre>
<h5>restart the apache server</h5>
<pre># /etc/init.d/apache2 restart</pre>
<p>You should see Django's "It Worked!" page.</p>
<h5>Set up a MySql database and user</h5>
<p>Note, use the password you entered when installing MySql</p>
<pre># mysql -u root -p
Enter password:
mysql> CREATE DATABASE django_db;
Query OK, 1 row affected (0.01 sec)
mysql> GRANT ALL ON django_db.* TO 'djangouser'@'localhost' IDENTIFIED BY 'yourpassword';
Query OK, 0 rows affected (0.03 sec)
mysql> quit
Bye</pre>
<h5>Edit the Django database settings</h5>
Edit <code>mysite/settings.py</code>:
<pre>DATABASE_ENGINE = 'mysql' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'ado_mssql'.
DATABASE_NAME = 'django_db' # Or path to database file if using sqlite3.
DATABASE_USER = 'djangouser' # Not used with sqlite3.
DATABASE_PASSWORD = 'yourpassword' # Not used with sqlite3.
DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3.</pre>
<h5>Do a 'syncdb' to create the database tables</h5>
<pre># cd mysite
# python manage.py syncdb
Creating table auth_message
Creating table auth_group
Creating table auth_user
Creating table auth_permission
Creating table django_content_type
Creating table django_session
Creating table django_site
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use 'sofeng'):
E-mail address: sofeng@email.com
Password:
Password (again):
Superuser created successfully.
Installing index for auth.Message model
Installing index for auth.Permission model
Loading 'initial_data' fixtures...
No fixtures found.</pre>
<h5>upload a mercurial django project</h5>
<p>on the remote instance, install mercurial:</p>
<pre># apt-get install mercurial</pre>
<p>on your local machine with the mercurial repo, run:</p>
<pre>$ hg clone -e 'ssh -i /home/sofeng/.ec2-elasticfox/id_django-keypair.pem' yourproj ssh://root@yourdns.compute-1.amazonaws.com//srv/yourproj</pre>
where <code>/home/sofeng/.ec2-elasticfox/id_django-keypair.pem</code> is
the private key associated with your instance and
<code>yourdns.compute-1.amazonaws.com</code> is the
public domain name associated with your instance.
<p>back on the remote instance:</p>
<pre># cd /srv/mozblog
# hg update</pre>
<pre># python manage.py syncdb</pre>
<h5>set up apache to serve static files</h5>
<ul>
<li>Create a link to the media files:
<pre># cd /var/www
# ln -s /srv/mozblog/media site_media
# ln -s /usr/share/python-support/python-django/django/contrib/admin/media/ admin_media</pre>
</li>
<li>Edit <code>/etc/apache2/httpd.conf</code>:
<pre><location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mozblog.settings
PythonPath "['/srv'] + sys.path"
PythonDebug On
</location>
<location "/site_media">
SetHandler None
</location>
<location "/admin_media">
SetHandler None
</location></pre>
</li>
</ul>
<h5>Restart the apache server</h5>
<pre># /etc/init.d/apache2 restart</pre>
<br>
Notes on users and groups on Linux
2008-08-19T20:24:39-07:00https://www.saltycrane.com/blog/2008/08/notes-users-and-groups-linux/<p>Here are some notes on some basic tasks dealing with users, groups,
and permissions on Ubuntu Linux. All these commands (except passwd)
are done as root.
If you are not root, prepend <code>sudo</code> to all the commands.</p>
<ul>
<li>Add a user, sofeng
<pre># adduser sofeng</pre>
</li>
<li>Change your password:
<pre>$ passwd</pre>
</li>
<li>Create a developer group
<pre># addgroup developer</pre>
</li>
<li>Add user sofeng to the developer group
<pre># adduser sofeng developer</pre>
</li>
<li>Give developer group <code>sudo</code> power:<br>
Add the following line to <code>/etc/sudoers</code>:
<pre>%developer ALL=(ALL) ALL</pre>
</li>
<li>Change owner to sofeng, and group to developer, of directory, mydir,
and all its subdirectories:
<pre># chown -R sofeng:developer mydir</pre>
</li>
<li>Change permisions of directory, mydir, and all its subdirectories,
to be writable by the group, developer:
<pre># chmod -R g+w mydir</pre>
</li>
</ul>
Somewhere on your Python path
2008-08-05T15:11:00-07:00https://www.saltycrane.com/blog/2008/08/somewhere-your-python-path/<p>As I install new python packages, I sometimes see instructions
which say something like "check out the code, and place it somewhere
on your Python path". These are very simple instructions, but
since it is not automatic like a Windows installer, or Ubuntu's
package management system, it causes me to pause. Where on my
Python path should I put it? I could put all my packages in
random places and update my PYTHONPATH environment variable every time. I also
thought about putting new packages in Python's <code>site-packages</code>
directory. This is probably a good option. However, I tend to like to
have all my important stuff in my home directory so I can easier
maintain it across multiple machines. (Also, I forget
where the <code>site-packages</code> lives (it is
<code>/usr/lib/python2.5/site-packages</code> on Ubuntu Hardy).)
So my solution was to create my own <code>python-packages</code>
directory in <code>~/lib</code>. I set the PYTHONPATH in my
<code>~/.bashrc</code> as follows:
<pre>export PYTHONPATH=$HOME/lib/python-packages</pre>
and then put all my Python packages here.
</p>
<br><br>
<p>Update: Some Python packages are distributed with other stuff besides the actual
Python package. To handle these cases, I created a <code>dist</code> directory
inside my <code>python-packages</code> directory, and created symbolic links
from the actual package directory in <code>dist</code> to
<code>python-packages</code>. Finally, I made the entire
<code>python-packages</code> directory a Mercurial repository so I can
finely control my Python environment and easily maintain it across multiple machines.
Here's what my <code>~/lib/python-packages</code> currently looks like:
</p>
<pre>drwxr-xr-x 9 sofeng sofeng 4096 2008 08/05 21:40 dist
drwxr-xr-x 4 sofeng sofeng 4096 2008 08/05 21:31 django_openidconsumer
lrwxrwxrwx 1 sofeng sofeng 43 2008 08/05 21:31 elementtree -> dist/elementtree-1.2.6-20050316/elementtree
lrwxrwxrwx 1 sofeng sofeng 31 2008 08/05 21:31 openid -> dist/python-openid-1.2.0/openid
lrwxrwxrwx 1 sofeng sofeng 31 2008 08/05 21:31 openid2.2 -> dist/python-openid-2.2.1/openid
lrwxrwxrwx 1 sofeng sofeng 27 2008 08/05 21:41 pygments -> dist/Pygments-0.10/pygments
lrwxrwxrwx 1 sofeng sofeng 29 2008 08/05 21:31 urljr -> dist/python-urljr-1.0.1/urljr
lrwxrwxrwx 1 sofeng sofeng 29 2008 08/05 21:31 yadis -> dist/python-yadis-1.1.0/yadis</pre>
<p><em>Update 2008-09-14</em>:
<a href="http://groups.google.com/group/django-developers/msg/5407cdb400157259">Here
is a post on the django-developers mailing list</a> by Kevin Teague which
explains the large number of technologies related to Python package management
and deployment including PyPi, Distutils, Eggs, Easy Install,
<a href="http://pypi.python.org/pypi/virtualenv">VirtualEnv</a>, and
<a href="http://pypi.python.org/pypi/zc.buildout">Buildout</a>. Kevin admits
that package management and deployment is an area in Python where there is room
for a great deal of improvemnt. He notes that the symlinking method
that I use can work for simple needs, but it fails for more complicated use cases,
such as tracking package dependencies. The new Virtualenv and Buildout
technologies seem to be interesting-- I will have to check them out when I have
time. I found this link <a href="http://simonwillison.net/2008/Sep/15/packaging/">via
Simon Willison</a>
</p>
<p><em>Update 2008-09-24:</em> <a href="http://blog.ianbicking.org/">Ian Bicking</a>,
author of Virtualenv, has <a href="http://www.openplans.org/projects/topp-engineering/blog/2008/09/24/pyinstall-a-new-hope/">just released pyinstall</a> which seems to be an improved easy_install. I have not tried
it yet, but I believe Ian Bicking writes good code.</p>
<p><em>Update 2008-10-24:</em><a href="http://glyph.twistedmatrix.com/">
Glyph Lefkowitz</a>, lead architect of
<a href="http://twistedmatrix.com/trac/">Twisted</a>,
suggests using
<code>twisted.python.modules</code> for solving
<a href="http://glyf.livejournal.com/71805.html">Python Path
Programming Problems</a>. This looks like something I could use in my
current project-- I just wish I understood it.</p>
<p><em>Update 2008-12-16:</em>
<ul>
<li>James Bennett (Django release manager):
<a href="http://www.b-list.org/weblog/2008/dec/14/packaging/">
Please, for the love of Guido, stop using setuptools and easy_install,
and use distutils and pip instead.</a> (pip is the new name for pyinstall,
by Ian Bicking.)</li>
<li>Ian Bicking: <a href="http://blog.ianbicking.org/2008/12/14/a-few-corrections-to-on-packaging/">
A Few Corrections To “On Packaging”</a></li>
</ul>
</p>
Notes on moving Ubuntu Wubi to a standard ext3 partition using LVPM
2008-08-04T12:34:59-07:00https://www.saltycrane.com/blog/2008/08/notes-moving-ubuntu-wubi-standard-ext3-partition-using-lvpm/<p>Here are my notes for moving my
<a href="http://wubi-installer.org/">wubi</a> Ubuntu install
to a dedicated ext3 partition. I used the
<a href="http://lubi.sourceforge.net/lvpm.html">Loopmounted
Virtual Partition Manager (LVPM)</a> to do the transfer.
From the webpage:
<blockquote>
The Loopmounted Virtual Partition Manager allows users to upgrade their existing Wubi or Lubi installation to a standard Ubuntu system by transferring all data, settings, and applications from the original install to a dedicated partition. The advantages of upgrading using LVPM are better disk performance and reliability, and the ability to replace the original operating system with Ubuntu.
</blockquote>
</p>
<h5>Add 2 new partitions</h5>
<p>My hard disk had only 1 partition containing Windows XP.
</p>
<ul>
<li>Boot to Xubuntu 8.04 Hardy Heron Live CD</li>
<li>Go to System, Partition Editor</li>
<li>Click the existing partition, and "Resize".</li>
<li>Click "Apply".<br>
<ul>
<li>Note 1: the first time I tried this, I got <em>ERROR: Extended
record needed (1712>1024), not yet supported. Please try to
free less space.</em> So I slid my partition divider a little
more to the right and tried again and it worked.</li>
<li>Note 2: This took a few minutes when it worked.</li>
</ul>
</li>
<li>In the unallocated section, create a new partition.
I made this my swap partition. It was 1065 MB, Primary Partition,
Filesystem: linux-swap. It became sda2. Click "Add".
</li>
<li>Then I created my main Linux partition. Used the remaining
size, filesystem: ext3, Primary Partition. This is sda3. Click
"Add".
</li>
<li>Click "Apply".</li>
</ul>
<h5>Install and run LVPM</h5>
<ul>
<li>Boot into the Wubi Ubuntu installation.</li>
<li>Go to
<a href="http://sourceforge.net/project/showfiles.php?group_id=198821">
http://sourceforge.net/project/showfiles.php?group_id=198821</a>,
download, save, install, and run it.
</li>
<li>Select "Transfer" and then the new main linux partition (sda3)</li>
<li>Reboot</li>
<li>When I selected Ubuntu in the Grub boot loader, I got the
following error mesage: <em>Error 17: Cannot mount selected partition.</em>
</li>
<li>To fix this, I hit "e" to edit the command, and changed
<code>root ()/ubuntu/disks</code> to <code>root (hd0,2)</code>.
Then I hit "b" to boot. <code>hd0</code> means the first hard disk,
and <code>2</code> means the 3rd partition.
</li>
<li>After booting into the new ext3 Ubuntu install, I edited the
<code>/boot/grub/menu.lst</code> file to change
<code>root ()/ubuntu/disks</code> to <code>root (hd0,2)</code>
</li>
<li>All pau.</li>
</ul>
Install wmii snapshot
2008-07-29T14:24:21-07:00https://www.saltycrane.com/blog/2008/07/install-wmii-snapshot/<p>These are my notes for installing wmii snapshot in my
home directory on Ubuntu 8.04 Hardy.</p>
<ul>
<li>Download the latest snapshot (wmii+ixp-20080520) from the
<a href="http://www.suckless.org/wmii/">wmii homepage</a>
and save it to <code>~/incoming</code>.
</li>
<li>Untar the snapshot.
<pre>$ cd ~/incoming
$ tar -zxvf wmii+ixp-20080520.tgz</pre>
</li>
<li>Install prerequisites:
<pre>$ sudo apt-get install build-essential xorg-dev</pre>
<pre>$ sudo apt-get install dwm-tools</pre>
</li>
<li>Build: (I left the defaults for everything during make config except
for the path. I changed this to a temporary directory, <code>~/tmp/wmii</code>.)
<pre>$ cd ~/incoming/wmii+ixp-20080520
$ make config
$ make
$ make install</pre>
</li>
<li>Move bin, etc, lib, and share to a new wmii directory.
<pre>$ mkdir ~/lib/wmii
$ mv ~/tmp/wmii/* ~/lib/wmii</pre>
</li>
<li>Create links in my <code>~/bin</code> directory:
<pre>$ cd ~/bin
$ ln -s ../lib/wmii/bin/wihack wihack
$ ln -s ../lib/wmii/bin/wmii9rc wmii9rc
$ ln -s ../lib/wmii/bin/wmii9menu wmii9menu
$ ln -s ../lib/wmii/bin/wmii.rc wmii.rc
$ ln -s ../lib/wmii/bin/wmii.sh wmii.sh</pre>
</li>
<li>Create a .xinitrc script:
<pre>xmodmap ~/.Xmodmap
gnome-screensaver&
urxvt&
until wmii; do
true
done</pre>
</li>
<li>And link <code>~/.xsession</code> to it:
<pre>$ ln -s ~/.xinitrc ~/.xsession</pre>
</li>
<li>Log out of the current window manager and then select X client script
as the session and log in.</li>
</ul>
New PC setup notes
2008-07-28T23:43:47-07:00https://www.saltycrane.com/blog/2008/07/new-pc-setup-notes/<p>Here is how I set up my new work PC with Xubuntu, wmii, and conkeror using the
wubi windows installer and my home mercurial repository of dot files.
Notes: <code>ti</code> is a python script I use to interface with the
remote machine that holds my mercurial repositories. I also have another
python script which concatenates my configuration files in <code>~/etc</code>
with my local machine-specific configuration files in <code>~/local/localmachine/etc</code>
and puts them in my home directory. <code>crayola</code> is
my home machine. <code>cotton</code> is my new work machine. (Actually I
haven't changed the name yet.)
</p>
<ul>
<li>I used the <a href="http://wubi-installer.org/">wubi installer</a>
to install Xubuntu as part of the Windows filesystem.</li>
<li>I restarted the computer and selected to boot into Xubuntu. (I
actually had to restart twice because the first time it was configuring
stuff.)</li>
<li>I ran the Update Manager.</li>
<li>I installed Mercurial:
<pre>$ sudo apt-get install mercurial</pre>
</li>
<li>I cloned my home directory repository and set it up:
<pre>$ cd /home
$ sudo mv sofeng sofeng-old
$ sudo hg clone ssh://sofeng@myrepos.machine.com//home/sofeng
$ sudo mkdir -p ~/local/cotton/etc
$ sudo mkdir -p ~/local/cotton/bin
$ sudo chown -R sofeng:sofeng /home/sofeng
$ cp ~/local/crayola/etc/.* ~/local/cotton/etc
$ . ~/.bashrc</pre>
</li>
<li>I installed some stuff:
<pre>$ sudo apt-get install wmii
$ sudo apt-get install rxvt-unicode
$ sudo apt-get install emacs-snapshot
$ sudo apt-get install lastfm
$ sudo apt-get install conky</pre>
</li>
<li>I logged out of Xfce and logged into wmii. Mod4+Enter to get a
<code>urxvt</code> terminal. </li>
<li>I cloned my conkeror repository:
<pre>$ cd ~/lib
$ ti clone conk</pre>
</li>
</ul>
Mount drives manually on Ubuntu Linux
2008-06-06T22:47:00-07:00https://www.saltycrane.com/blog/2008/06/mount-drives-manually-on-ubuntu-linux/
<p>Since I'm running the <a href="http://www.suckless.org/wiki/wmii/">
wmii</a> window manager instead of Gnome or KDE, devices such as
USB drives, external hard drives, and SD card readers don't automatically
mount. This can be good or bad depending on your preference. Here are
my notes on mounting devices manually. I am no expert at this stuff,
so use this at your own risk. There is definitely the possibility of
losing data when dealing with this stuff. Read the man pages and use
this just as a refresher. As an example, I will mount a SD card.</p>
<br /><b>To see what devices are available:</b>
<pre>$ sudo fdisk -l</pre>
This gives me the following output:
<pre>Disk /dev/sda: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x38000000
Device Boot Start End Blocks Id System
/dev/sda1 1 7 56196 de Dell Utility
/dev/sda2 8 660 5245222+ b W95 FAT32
/dev/sda3 * 661 30017 235810102+ 83 Linux
/dev/sda4 30018 30394 3028252+ 5 Extended
/dev/sda5 30018 30394 3028221 82 Linux swap / Solaris
Disk /dev/sdf: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0008bc3c
Device Boot Start End Blocks Id System
/dev/sdf1 1 38913 312568641 83 Linux
Disk /dev/sdd: 512 MB, 512229376 bytes
9 heads, 8 sectors/track, 13895 cylinders
Units = cylinders of 72 * 512 = 36864 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdd1 4 13896 500107+ 6 FAT16</pre>
<code>/dev/sda3</code> is my main hard disk partition and <code>/dev/sdf1</code>
is an external hard drive formatted for Linux. The last device,
<code>/dev/sdd1</code> is my SD card.<br />
<br /><b>Create a mount point</b>
<pre>$ sudo mkdir /media/sdcard</pre>
<br /><b>Mount the device</b>
<pre>$ sudo mount /dev/sdd1 /media/sdcard</pre>
Now you can access the files at <code>/media/sdcard</code>.<br />
<br /><b>List mounted devices</b>
<pre>$ mount -l</pre>
You should see things such as:
<pre>/dev/sda3 on / type ext3 (rw,errors=remount-ro) []
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
/dev/sdf1 on /media/disk type ext3 (rw,nosuid,nodev,uhelper=hal) []
tmpfs on /lib/modules/2.6.24-18-generic/volatile type tmpfs (rw,mode=0755)
/dev/sdd1 on /media/sdcard type vfat (rw) []</pre>
The last line shows device <code>/dev/sdd1</code> mounted at <code>
/media/sdcard</code>.<br />
<br /><b>Unmount the device</b>
<pre>$ sudo umount /media/sdcard</pre>
or
<pre>$ sudo umount /dev/sdd1</pre>
/usr/bin/python: bad interpreter: Permission denied error
2008-05-12T09:59:00-07:00https://www.saltycrane.com/blog/2008/05/usrbinpython-bad-interpreter-permission/<p>
I have a Python script,
myscript.py with a <code>#!/usr/bin/python</code>
<a href="http://en.wikipedia.org/wiki/Shebang_(Unix)">shebang</a>* at the top
and tried to execute it on Ubuntu Linux using
<code>./myscript.py</code>. I got the following error message:
</p>
<pre>bash: ./myscript.py: /usr/bin/python: bad interpreter: Permission denied </pre>
<p>
Here are things to check:
</p>
<ul>
<li>The file should be executable (use <code>chmod +x myscript.py</code>)</li>
<li>The file shoud have Unix line endings</li>
<li>The file shouldn't be on a fat32 or ntfs filesystem. Apparently, bash can't handle scripts that are
stored on fat32 or ntfs</li>
<li>The shebang line must be less than 128 characters (on Linux)</li>
</ul>
<p>
<em>UPDATE 2015-09-23:</em> 7 years later, I ran into this error again on our Jenkins server...
A shebang line that is <strong>too long</strong> will also cause this error.
On Linux, the maximum length is 128 characters. See:
<ul>
<li><a href="https://github.com/pypa/pip/issues/1773">https://github.com/pypa/pip/issues/1773</a></li>
<li><a href="https://github.com/pypa/virtualenv/issues/596">https://github.com/pypa/virtualenv/issues/596</a></li>
<li><a href="http://superuser.com/questions/634555/shebang-line-length-limitation">http://superuser.com/questions/634555/shebang-line-length-limitation</a></li>
<li><a href="http://www.in-ulm.de/~mascheck/various/shebang/#length">http://www.in-ulm.de/~mascheck/various/shebang/#length</a></li>
</ul>
</p>
<hr>
<p>* <code>#!/usr/bin/env python</code> would be the more portable shebang.</p>
Recommended books
2008-04-17T19:13:00-07:00https://www.saltycrane.com/blog/2008/04/recommended-books/<p>I love having a subscription to <a href="http://search.safaribooksonline.com/home">
Safari Books Online</a>. Currently my company provides a free subscription,
but if I get a new job, I might consider subscribing myself. Since I get to
browse a number of books at no cost, I thought I'd note which books are my
favorites. (Note, I am not being paid by Safari Books Online.)</p>
<h4 id="general-software">General Software</h4>
<ul>
<li><em>Structure and Interpretation of Computer Programs, Second Edition</em>,
Harold Abelson and Gerald Jay Sussman, MIT Press, ?year?<br />
I learned about this book through a job posting. It might bring you to
tears if you get it. I'm only in the second chapter. It is used in an introductory
Computer Science course at MIT. It uses Scheme (Lisp) to demonstrate concepts.<br />
Available free online at:
<a href="http://mitpress.mit.edu/sicp/full-text/book/book.html">
http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html</a><br />
A <a href="http://swiss.csail.mit.edu/classes/6.001/abelson-sussman-lectures/">
free video lecture series</a> is also available.
</li>
</ul>
<h4 id="c">C</h4>
<ul>
<li><em>The C Programming Language, Second Edition</em>, Brian W. Kernighan
and Dennis M. Ritchie, Prentice Hall, 1988<br />
The definitive C book.
</li>
</ul>
<h4 id="python">Python</h4>
<ul>
<li><em>Core Python Programming, Second Edition</em>, Wesley J. Chun,
Prentice Hall, September 18, 2006<br />
Usually I like O'Reilly books best, but I slightly prefer Chun's text
to <em>Learning Python</em>.<br />
Available at
<a href="http://search.safaribooksonline.com/0132269937?tocview=true">
Safari Books Online</a>
</li>
<li><em>The Django Book</em>, Apress, December 2007<br />
I think this is the first official Django book.<br />
</li>
</ul>
<h4 id="sqlite">SQLite</h4>
<ul>
<li><em>The Definitive Guide to SQLite</em>, Mike Owens, Apress, May 2006<br />
I browsed a few SQL books but liked this one better than most.
It has a good theory section.<br />
Available at
<a href="http://apress.com/book/view/9781590596739">
Apress.com</a>
</li>
</ul>
<h4 id="linux">Linux or related</h4>
<ul>
<li><em>X Power Tools</em>, Chris Tyler, O'Reilly, December 15, 2007<br />
Lots of good information on the X Window System and more; easy to understand.
I wish the basic Ubuntu or
Linux books had some of this information.<br />
Available at
<a href="http://search.safaribooksonline.com/9780596101954?tocview=true">
Safari Books Online</a>
</li>
<li>
<em>SSH, The Secure Shell: The Definitive Guide, Second Edition</em>,
Daniel J. Barrett; Richard E. Silverman; Robert G. Byrnes,
O'Reilly, May 10, 2005<br>
Available at
<a href="http://search.safaribooksonline.com/book/networking/ssh/0596008953">
Safari Books Online</a>
</li>
</ul>
<h4 id="ruby">Ruby</h4>
<ul>
<li><em>why's poignant guide to Ruby</em> by <a href="http://whytheluckystiff.net/">why
the lucky stiff</a><br />
Only in chapter 3, but very funny.<br />
Available free online at: <a href="http://poignantguide.net/ruby/">http://poignantguide.net/ruby/</a>
</li>
</ul>
<h4 id="javascript">Javascript</h4>
<ul>
<li>
<em>Javascript: The Good Parts</em>, Douglas Crockford, O'Reilly, May 2008<br>
This is the first Javascript book I read as I switched to Javascript and Frontend development. I was debating whether to read it since it was so old but I found it to be good and recommend it.
</li>
<li>
<em>You Don't Know JS: ES6 & Beyond</em>, Kyle Simpson, O'Reilly, December 2015<br>
This is the primary way I learned ES6. It is very detail oriented.<br>
Available free online
<a href="https://github.com/getify/You-Dont-Know-JS/tree/master/es6%20%26%20beyond">
on github
</a>
</li>
<li>
<em>JavaScript Allongé, the "Six" Edition</em>, Reg "raganwald" Braithwaite, Leanpub, 2016<br>
This was the third Javascript book I read and it was the most fun. It teaches functional programming concepts such as closures, shadowing, higher order functions, combinators, decorators, etc using ES6.<br>
Available free online at:
<a href="https://leanpub.com/javascriptallongesix/read">https://leanpub.com/javascriptallongesix/read</a>
</li>
</ul>
<h4 id="non-technical">Non technical</h4>
<ul>
<li>The Hitchhiker's Guide to the Galaxy by Douglas Adams</li>
<li>Calvin and Hobbes by Bill Watterson</li>
<li>Crime and Punishment by Fyodor Dostoyevsky</li>
<li>Mere Christianity by C. S. Lewis</li>
<li>Screwtape Letters by C. S. Lewis</li>
</ul>
<h4 id="see-also">See also</h4>
<ul>
<li><a href="http://www.reddit.com/r/books/comments/ch0wt/a_reading_list_for_the_selftaught_computer/">
http://www.reddit.com/r/books/comments/ch0wt/a_reading_list_for_the_selftaught_computer/</a></li>
<li><a href="http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read">
http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read</a></li>
<li><a href="http://www.reddit.com/r/books/comments/cq4qe/reddits_bookshelf/">
http://www.reddit.com/r/books/comments/cq4qe/reddits_bookshelf/</a></li>
</ul>
How to install Mercurial 1.0 on Ubuntu Hardy
2008-04-17T12:52:00-07:00https://www.saltycrane.com/blog/2008/04/how-to-install-mercurial-10-on-ubuntu/<p>Mercurial 1.0 is out!
<a href="http://www.selenic.com/mercurial">Mercurial</a> is a next generation
"fast, lightweight
source control management system" boasting conversions by notable projects
NetBeans, OpenJDK, Globulation2, Xine, Mozilla, grml, and OpenSolaris.</p>
<p>From the <a href="http://www.selenic.com/pipermail/mercurial/2008-March/018014.html">
release notes</a>, version 1.0 has improvements for copying, renaming,
removing, and merging. I want to use the most robust version possible,
so I decided to upgrade. Ubuntu Gutsy's Mercurial version is
<a href="http://packages.ubuntu.com/gutsy/devel/mercurial">0.9.4</a>, and
even Hardy will only have <a href="http://packages.ubuntu.com/hardy/mercurial">
0.9.5</a>. Luckily, Mercurial 1.0 is
<a href="http://peak.telecommunity.com/DevCenter/EasyInstall">easy-installable</a>.
That makes it easy to install:</p>
<br /><b>Install Mercurial 1.0</b>
<ol>
<li>If you don't already have Mercurial 0.9.4 installed, I'd suggest
installing it because it will setup /etc/mercurial for using Mercurial
extenstions.<br />
<pre>sudo apt-get install mercurial</pre>
</li>
<li>Install Python Easy Install, build tools, and Python header files.<br />
<pre>sudo apt-get install python-setuptools python-dev build-essential</pre>
</li>
<li>Install Mercurial 1.0<br />
<pre>sudo easy_install -U mercurial</pre>
</li>
</ol>
That was easy! Test it out by typing <code>hg version</code>.
<pre>Mercurial Distributed SCM (version 1.0)
Copyright (C) 2005-2008 Matt Mackall <mpm> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.</mpm></pre>
<br /><b>Upgrade to 1.0.1</b><br />
Update 6/2/2008: Mercurial 1.0.1 has been released as a bugfix version.
You can upgrade from 1.0 to 1.0.1 using Easy Install:
<pre>sudo easy_install -U mercurial</pre>
<br /><b>Install hgk extension</b><br />
If you try the <code>hg view</code> command, you will get a
<code>sh: hgk: not found</code> error. To fix this, I copied the hgk executable
from the contrib directory in the source package to a directory in my path:
<ol>
<li>Download and unpack the Mercurial source package.<br />
<pre>cd /tmp
wget http://www.selenic.com/mercurial/release/mercurial-1.0.tar.gz
tar zxvf mercurial-1.0.tar.gz</pre>
</li>
<li>Copy hgk somewhere on your path. E.g.:<br />
<pre>cp mercurial-1.0/contrib/hgk ~/bin</pre>
</li>
</ol>
Note, if you get a <code>/usr/bin/env: wish: No such file or directory</code>
error, it means you need to install the Tk package:
<pre>sudo apt-get install tk8.5</pre>
<br /><b>Other Notes</b><br />
<ul>
<li>hbisect warning<br />
Per the <a href="http://www.selenic.com/mercurial/wiki/index.cgi/UpgradeNotes">
Upgrade Notes</a>, bisect is now a built-in command and the hbisect extension
should not be used. If you get a
<code>extension 'hgext/hbisect' overrides commands: bisect</code>
warning, remove the <code>hbisect=</code> line from your configuration file
(located at /etc/mercurial/hgrc.d/hgext.rc or ~/.hgrc).
<br /><br />
</li>
<li>Where are the files located?<br />
The hgext and mercurial files should be located at
<code>/usr/lib/python2.5/site-packages/mercurial-1.0-py2.5-linux-*.egg</code>
<br /><br />
</li>
<li>stdio.h error during easy_install<br />
If you got the following error while trying to run easy_install,
it probably means you don't have the "build-essential"
package installed. <code>$ sudo apt-get install build-essential</code>.
<pre style="height: 50px; overflow:scroll">In file included from /usr/lib/gcc/i486-linux-gnu/4.2.3/include/syslimits.h:7,
from /usr/lib/gcc/i486-linux-gnu/4.2.3/include/limits.h:11,
from /usr/include/python2.5/Python.h:18,
from mercurial/mpatch.c:23:
/usr/lib/gcc/i486-linux-gnu/4.2.3/include/limits.h:122:61: error: limits.h: No such file or directory
In file included from mercurial/mpatch.c:23:
/usr/include/python2.5/Python.h:32:19: error: stdio.h: No such file or directory
/usr/include/python2.5/Python.h:34:5: error: #error "Python.h requires that stdio.h define NULL."
/usr/include/python2.5/Python.h:37:20: error: string.h: No such file or directory
/usr/include/python2.5/Python.h:39:19: error: errno.h: No such file or directory
/usr/include/python2.5/Python.h:41:20: error: stdlib.h: No such file or directory
/usr/include/python2.5/Python.h:43:20: error: unistd.h: No such file or directory
/usr/include/python2.5/Python.h:55:20: error: assert.h: No such file or directory
In file included from /usr/include/python2.5/Python.h:57,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyport.h:7:20: error: stdint.h: No such file or directory
In file included from /usr/include/python2.5/Python.h:57,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyport.h:89: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_uintptr_t’
/usr/include/python2.5/pyport.h:90: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_intptr_t’
/usr/include/python2.5/pyport.h:113: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_ssize_t’
/usr/include/python2.5/pyport.h:231:76: error: math.h: No such file or directory
/usr/include/python2.5/pyport.h:238:22: error: sys/time.h: No such file or directory
/usr/include/python2.5/pyport.h:239:18: error: time.h: No such file or directory
/usr/include/python2.5/pyport.h:257:24: error: sys/select.h: No such file or directory
/usr/include/python2.5/pyport.h:296:22: error: sys/stat.h: No such file or directory
In file included from /usr/include/python2.5/Python.h:76,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pymem.h:50: warning: parameter names (without types) in function declaration
/usr/include/python2.5/pymem.h:51: error: expected declaration specifiers or ‘...’ before ‘size_t’
In file included from /usr/include/python2.5/Python.h:78,
from mercurial/mpatch.c:23:
/usr/include/python2.5/object.h:104: error: expected specifier-qualifier-list before ‘Py_ssize_t’
/usr/include/python2.5/object.h:108: error: expected specifier-qualifier-list before ‘Py_ssize_t’
/usr/include/python2.5/object.h:131: error: expected declaration specifiers or ‘...’ before ‘*’ token
/usr/include/python2.5/object.h:131: warning: type defaults to ‘int’ in declaration of ‘Py_ssize_t’
/usr/include/python2.5/object.h:131: error: ‘Py_ssize_t’ declared as function returning a function
/usr/include/python2.5/object.h:131: warning: function declaration isn’t a prototype
/usr/include/python2.5/object.h:149: error: ‘readbufferproc’ declared as function returning a function
/usr/include/python2.5/object.h:150: error: ‘writebufferproc’ declared as function returning a function
/usr/include/python2.5/object.h:151: error: ‘segcountproc’ declared as function returning a function
/usr/include/python2.5/object.h:152: error: ‘charbufferproc’ declared as function returning a function
/usr/include/python2.5/object.h:215: error: expected specifier-qualifier-list before ‘lenfunc’
/usr/include/python2.5/object.h:229: error: expected specifier-qualifier-list before ‘lenfunc’
/usr/include/python2.5/object.h:244: error: expected declaration specifiers or ‘...’ before ‘FILE’
/usr/include/python2.5/object.h:258: warning: ‘struct _typeobject’ declared inside parameter list
/usr/include/python2.5/object.h:258: warning: its scope is only this definition or declaration, which is probably not what you want
/usr/include/python2.5/object.h:259: warning: ‘struct _typeobject’ declared inside parameter list
/usr/include/python2.5/object.h:262: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/object.h:262: error: field ‘ob_size’ declared as a function
/usr/include/python2.5/object.h:264: error: field ‘tp_basicsize’ declared as a function
/usr/include/python2.5/object.h:264: error: field ‘tp_itemsize’ declared as a function
/usr/include/python2.5/object.h:309: error: field ‘tp_weaklistoffset’ declared as a function
/usr/include/python2.5/object.h:324: error: field ‘tp_dictoffset’ declared as a function
/usr/include/python2.5/object.h:389: error: expected declaration specifiers or ‘...’ before ‘FILE’
In file included from /usr/include/python2.5/Python.h:79,
from mercurial/mpatch.c:23:
/usr/include/python2.5/objimpl.h:97: warning: parameter names (without types) in function declaration
/usr/include/python2.5/objimpl.h:98: error: expected declaration specifiers or ‘...’ before ‘size_t’
/usr/include/python2.5/objimpl.h:228: error: ‘PyGC_Collect’ declared as function returning a function
/usr/include/python2.5/objimpl.h:249: error: field ‘gc_refs’ declared as a function
/usr/include/python2.5/objimpl.h:288: warning: parameter names (without types) in function declaration
In file included from /usr/include/python2.5/Python.h:83,
from mercurial/mpatch.c:23:
/usr/include/python2.5/unicodeobject.h:55:19: error: ctype.h: No such file or directory
/usr/include/python2.5/unicodeobject.h:118:21: error: wchar.h: No such file or directory
In file included from /usr/include/python2.5/Python.h:83,
from mercurial/mpatch.c:23:
/usr/include/python2.5/unicodeobject.h:384: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/unicodeobject.h:385: error: field ‘length’ declared as a function
/usr/include/python2.5/unicodeobject.h:447: error: ‘PyUnicodeUCS4_GetSize’ declared as function returning a function
/usr/include/python2.5/unicodeobject.h:521: warning: type defaults to ‘int’ in declaration of ‘wchar_t’
/usr/include/python2.5/unicodeobject.h:521: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/python2.5/unicodeobject.h:539: warning: type defaults to ‘int’ in declaration of ‘wchar_t’
/usr/include/python2.5/unicodeobject.h:539: error: expected ‘;’, ‘,’ or ‘)’ before ‘*’ token
/usr/include/python2.5/unicodeobject.h:1102: error: ‘PyUnicodeUCS4_Tailmatch’ declared as function returning a function
/usr/include/python2.5/unicodeobject.h:1114: error: ‘PyUnicodeUCS4_Find’ declared as function returning a function
/usr/include/python2.5/unicodeobject.h:1123: error: ‘PyUnicodeUCS4_Count’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:84,
from mercurial/mpatch.c:23:
/usr/include/python2.5/intobject.h:24: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/intobject.h:38: warning: parameter names (without types) in function declaration
/usr/include/python2.5/intobject.h:41: error: ‘PyInt_AsSsize_t’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:86,
from mercurial/mpatch.c:23:
/usr/include/python2.5/longobject.h:25: error: ‘_PyLong_AsSsize_t’ declared as function returning a function
/usr/include/python2.5/longobject.h:26: warning: parameter names (without types) in function declaration
/usr/include/python2.5/longobject.h:69: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘_PyLong_NumBits’
/usr/include/python2.5/longobject.h:85: error: expected declaration specifiers or ‘...’ before ‘size_t’
/usr/include/python2.5/longobject.h:108: error: expected declaration specifiers or ‘...’ before ‘size_t’
In file included from /usr/include/python2.5/Python.h:87,
from mercurial/mpatch.c:23:
/usr/include/python2.5/floatobject.h:15: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:89,
from mercurial/mpatch.c:23:
/usr/include/python2.5/complexobject.h:39: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:92,
from mercurial/mpatch.c:23:
/usr/include/python2.5/stringobject.h:36: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/stringobject.h:36: error: field ‘ob_size’ declared as a function
/usr/include/python2.5/stringobject.h:67: error: ‘PyString_Size’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:94,
from mercurial/mpatch.c:23:
/usr/include/python2.5/tupleobject.h:25: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/tupleobject.h:25: error: field ‘ob_size’ declared as a function
/usr/include/python2.5/tupleobject.h:40: error: ‘PyTuple_Size’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:95,
from mercurial/mpatch.c:23:
/usr/include/python2.5/listobject.h:23: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/listobject.h:23: error: field ‘ob_size’ declared as a function
/usr/include/python2.5/listobject.h:38: error: field ‘allocated’ declared as a function
/usr/include/python2.5/listobject.h:47: error: ‘PyList_Size’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:96,
from mercurial/mpatch.c:23:
/usr/include/python2.5/dictobject.h:55: error: field ‘me_hash’ declared as a function
/usr/include/python2.5/dictobject.h:71: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/dictobject.h:72: error: field ‘ma_fill’ declared as a function
/usr/include/python2.5/dictobject.h:73: error: field ‘ma_used’ declared as a function
/usr/include/python2.5/dictobject.h:79: error: field ‘ma_mask’ declared as a function
/usr/include/python2.5/dictobject.h:108: error: ‘PyDict_Size’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:98,
from mercurial/mpatch.c:23:
/usr/include/python2.5/setobject.h:36: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/setobject.h:38: error: field ‘fill’ declared as a function
/usr/include/python2.5/setobject.h:39: error: field ‘used’ declared as a function
/usr/include/python2.5/setobject.h:45: error: field ‘mask’ declared as a function
/usr/include/python2.5/setobject.h:79: error: ‘PySet_Size’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:99,
from mercurial/mpatch.c:23:
/usr/include/python2.5/methodobject.h:82: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:101,
from mercurial/mpatch.c:23:
/usr/include/python2.5/funcobject.h:22: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:102,
from mercurial/mpatch.c:23:
/usr/include/python2.5/classobject.h:13: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/classobject.h:24: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/classobject.h:31: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:103,
from mercurial/mpatch.c:23:
/usr/include/python2.5/fileobject.h:11: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/fileobject.h:12: error: expected specifier-qualifier-list before ‘FILE’
/usr/include/python2.5/fileobject.h:38: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/fileobject.h:40: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
/usr/include/python2.5/fileobject.h:57: error: expected declaration specifiers or ‘...’ before ‘FILE’
/usr/include/python2.5/fileobject.h:58: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘Py_UniversalNewlineFread’
In file included from /usr/include/python2.5/Python.h:105,
from mercurial/mpatch.c:23:
/usr/include/python2.5/traceback.h:13: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:106,
from mercurial/mpatch.c:23:
/usr/include/python2.5/sliceobject.h:23: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:107,
from mercurial/mpatch.c:23:
/usr/include/python2.5/cellobject.h:10: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:109,
from mercurial/mpatch.c:23:
/usr/include/python2.5/genobject.h:13: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:110,
from mercurial/mpatch.c:23:
/usr/include/python2.5/descrobject.h:46: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/descrobject.h:50: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/descrobject.h:55: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/descrobject.h:60: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/descrobject.h:65: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:111,
from mercurial/mpatch.c:23:
/usr/include/python2.5/weakrefobject.h:16: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/weakrefobject.h:65: error: ‘_PyWeakref_GetWeakrefCount’ declared as function returning a function
In file included from /usr/include/python2.5/Python.h:114,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyerrors.h:10: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/pyerrors.h:17: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/pyerrors.h:31: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/pyerrors.h:44: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/pyerrors.h:52: error: field ‘ob_refcnt’ declared as a function
/usr/include/python2.5/pyerrors.h:329: error: expected declaration specifiers or ‘...’ before ‘size_t’
/usr/include/python2.5/pyerrors.h:330: error: format string argument not a string type
/usr/include/python2.5/pyerrors.h:331: error: expected declaration specifiers or ‘...’ before ‘size_t’
In file included from /usr/include/python2.5/Python.h:118,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyarena.h:50: error: expected declaration specifiers or ‘...’ before ‘size_t’
In file included from /usr/include/python2.5/Python.h:120,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pythonrun.h:34: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:35: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:37: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:38: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:39: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:44: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:54: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:60: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:77: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:142: error: expected ‘)’ before ‘*’ token
/usr/include/python2.5/pythonrun.h:144: error: expected ‘)’ before ‘*’ token
In file included from /usr/include/python2.5/Python.h:122,
from mercurial/mpatch.c:23:
/usr/include/python2.5/sysmodule.h:12: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
In file included from /usr/include/python2.5/Python.h:124,
from mercurial/mpatch.c:23:
/usr/include/python2.5/import.h:33: error: expected declaration specifiers or ‘...’ before ‘size_t’
/usr/include/python2.5/import.h:33: error: expected declaration specifiers or ‘...’ before ‘FILE’
In file included from /usr/include/python2.5/Python.h:126,
from mercurial/mpatch.c:23:
/usr/include/python2.5/abstract.h:421: error: ‘PyObject_Size’ declared as function returning a function
/usr/include/python2.5/abstract.h:433: error: ‘PyObject_Length’ declared as function returning a function
/usr/include/python2.5/abstract.h:436: error: ‘_PyObject_LengthHint’ declared as function returning a function
/usr/include/python2.5/abstract.h:774: error: ‘PyNumber_AsSsize_t’ declared as function returning a function
/usr/include/python2.5/abstract.h:947: error: ‘PySequence_Size’ declared as function returning a function
/usr/include/python2.5/abstract.h:956: error: ‘PySequence_Length’ declared as function returning a function
/usr/include/python2.5/abstract.h:1078: error: ‘PySequence_Count’ declared as function returning a function
/usr/include/python2.5/abstract.h:1097: error: ‘_PySequence_IterSearch’ declared as function returning a function
/usr/include/python2.5/abstract.h:1122: error: ‘PySequence_Index’ declared as function returning a function
/usr/include/python2.5/abstract.h:1161: error: ‘PyMapping_Size’ declared as function returning a function
/usr/include/python2.5/abstract.h:1171: error: ‘PyMapping_Length’ declared as function returning a function
In file included from /usr/include/python2.5/compile.h:5,
from /usr/include/python2.5/Python.h:128,
from mercurial/mpatch.c:23:
/usr/include/python2.5/code.h:11: error: field ‘ob_refcnt’ declared as a function
In file included from /usr/include/python2.5/Python.h:131,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pystrtod.h:11: error: expected declaration specifiers or ‘...’ before ‘size_t’
In file included from /usr/include/python2.5/Python.h:151,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyfpe.h:129:20: error: signal.h: No such file or directory
/usr/include/python2.5/pyfpe.h:130:20: error: setjmp.h: No such file or directory
In file included from /usr/include/python2.5/Python.h:151,
from mercurial/mpatch.c:23:
/usr/include/python2.5/pyfpe.h:132: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PyFPE_jbuf’
mercurial/mpatch.c:54:24: error: sys/types.h: No such file or directory
mercurial/mpatch.c:58:25: error: arpa/inet.h: No such file or directory
mercurial/mpatch.c:60:23: error: inttypes.h: No such file or directory
mercurial/mpatch.c: In function ‘lalloc’:
mercurial/mpatch.c:77: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:77: error: (Each undeclared identifier is reported only once
mercurial/mpatch.c:77: error: for each function it appears in.)
mercurial/mpatch.c:82: warning: implicit declaration of function ‘malloc’
mercurial/mpatch.c:82: warning: incompatible implicit declaration of built-in function ‘malloc’
mercurial/mpatch.c:89: warning: implicit declaration of function ‘free’
mercurial/mpatch.c: In function ‘combine’:
mercurial/mpatch.c:197: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:224: warning: implicit declaration of function ‘memcpy’
mercurial/mpatch.c:224: warning: incompatible implicit declaration of built-in function ‘memcpy’
mercurial/mpatch.c: In function ‘decode’:
mercurial/mpatch.c:244: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:249: warning: incompatible implicit declaration of built-in function ‘memcpy’
mercurial/mpatch.c:250: warning: implicit declaration of function ‘ntohl’
mercurial/mpatch.c:250: error: ‘uint32_t’ undeclared (first use in this function)
mercurial/mpatch.c:250: error: expected expression before ‘)’ token
mercurial/mpatch.c:251: error: expected expression before ‘)’ token
mercurial/mpatch.c:252: error: expected expression before ‘)’ token
mercurial/mpatch.c: In function ‘apply’:
mercurial/mpatch.c:310: warning: incompatible implicit declaration of built-in function ‘memcpy’
mercurial/mpatch.c:317: warning: incompatible implicit declaration of built-in function ‘memcpy’
mercurial/mpatch.c: In function ‘fold’:
mercurial/mpatch.c:330: warning: passing argument 2 of ‘PyList_GetItem’ makes pointer from integer without a cast
mercurial/mpatch.c:332: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:335: warning: passing argument 2 of ‘decode’ makes integer from pointer without a cast
mercurial/mpatch.c: In function ‘patches’:
mercurial/mpatch.c:355: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:360: error: ‘PyObject’ has no member named ‘ob_refcnt’
mercurial/mpatch.c:371: warning: passing argument 1 of ‘calcsize’ makes integer from pointer without a cast
mercurial/mpatch.c:376: warning: passing argument 2 of ‘PyString_FromStringAndSize’ makes pointer from integer without a cast
mercurial/mpatch.c:382: warning: passing argument 3 of ‘apply’ makes integer from pointer without a cast
mercurial/mpatch.c:383: error: ‘PyObject’ has no member named ‘ob_refcnt’
mercurial/mpatch.c:383: error: ‘PyObject’ has no member named ‘ob_type’
mercurial/mpatch.c: In function ‘patchedsize’:
mercurial/mpatch.c:401: error: ‘NULL’ undeclared (first use in this function)
mercurial/mpatch.c:407: warning: incompatible implicit declaration of built-in function ‘memcpy’
mercurial/mpatch.c:408: error: ‘uint32_t’ undeclared (first use in this function)
mercurial/mpatch.c:408: error: expected expression before ‘)’ token
mercurial/mpatch.c:409: error: expected expression before ‘)’ token
mercurial/mpatch.c:410: error: expected expression before ‘)’ token
mercurial/mpatch.c: At top level:
mercurial/mpatch.c:435: error: ‘NULL’ undeclared here (not in a function)
mercurial/mpatch.c: In function ‘initmpatch’:
mercurial/mpatch.c:442: warning: passing argument 2 of ‘PyErr_NewException’ from incompatible pointer type
mercurial/mpatch.c:442: warning: passing argument 3 of ‘PyErr_NewException’ from incompatible pointer type
error: Setup script exited with error: command 'gcc' failed with exit status 1
</pre>
</li>
<li>Python.h error during easy_install<br />
If you got the following error while trying to run easy_install,
it probably means you don't have the "python-dev"
package installed. <code>$ sudo apt-get install python-dev</code>.
<pre style="height: 50px; overflow:scroll">Searching for mercurial
Reading http://cheeseshop.python.org/pypi/mercurial/
Couldn't find index page for 'mercurial' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://cheeseshop.python.org/pypi/
Reading http://cheeseshop.python.org/pypi/Mercurial/1.0
Reading http://www.selenic.com/mercurial
Best match: mercurial 1.0
Downloading http://www.selenic.com/mercurial/release/mercurial-1.0.tar.gz
Processing mercurial-1.0.tar.gz
Running mercurial-1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-MfmZhU/mercurial-1.0/egg-dist-tmp-JDYH6V
mercurial/mpatch.c:23:20: error: Python.h: No such file or directory
mercurial/mpatch.c:64: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
mercurial/mpatch.c: In function 'lalloc':
mercurial/mpatch.c:92: warning: implicit declaration of function 'PyErr_Occurred'
mercurial/mpatch.c:93: warning: implicit declaration of function 'PyErr_NoMemory'
mercurial/mpatch.c: In function 'decode':
mercurial/mpatch.c:265: warning: implicit declaration of function 'PyErr_SetString'
mercurial/mpatch.c:265: error: 'mpatch_Error' undeclared (first use in this function)
mercurial/mpatch.c:265: error: (Each undeclared identifier is reported only once
mercurial/mpatch.c:265: error: for each function it appears in.)
mercurial/mpatch.c: In function 'calcsize':
mercurial/mpatch.c:283: error: 'mpatch_Error' undeclared (first use in this function)
mercurial/mpatch.c: In function 'apply':
mercurial/mpatch.c:306: error: 'mpatch_Error' undeclared (first use in this function)
mercurial/mpatch.c: At top level:
mercurial/mpatch.c:322: error: expected ')' before '*' token
mercurial/mpatch.c:344: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
mercurial/mpatch.c:392: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token
mercurial/mpatch.c:432: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'methods'
mercurial/mpatch.c:439: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'initmpatch'
error: Setup script exited with error: command 'gcc' failed with exit status 1</pre>
</li>
<li><em>Couldn't find a setup script</em> Error:<br />
If you get the following error when trying to run <code>sudo easy_install -U mercurial</code>,
it could mean that there is a "mercurial" directory in your current working directory. Change
to a different directory and run the command again.
<pre>Processing mercurial
error: Couldn't find a setup script in mercurial</pre>
</li>
<li><em>error: can't create or remove files in install directory</em><br>
<pre style="height:50px; overflow: auto">error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.5/site-packages/test-easy-install-28328.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/lib/python2.5/site-packages/
Perhaps your account does not have write access to this directory? If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account. If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.
For information on other options, you may wish to consult the
documentation at:
http://peak.telecommunity.com/EasyInstall.html
Please make the appropriate changes for your system and try again.</pre>
You will get this error if you don't use the <code>sudo</code> command. Use
<pre>$ sudo easy_install -U mercurial</pre>
</ul>
Install coLinux (and Ubuntu Hardy) on Win XP using Slirp to internet and TAP to host behind a corporate firewall/proxy server
2008-04-14T14:43:00-07:00https://www.saltycrane.com/blog/2008/04/install-colinux-and-ubuntu-gutsy-on-win/<p><a href="http://www.colinux.org">coLinux</a> allows you to run a full Linux
distribution, such as Ubuntu Hardy, on top of Windows at near native speeds.
It runs as a single Windows process with its own specially allocated
address space. The guest linux system can run from either a separate
disk partition or from a single file on your Windows filesystem.
</p>
<p>My initial impression of coLinux was so positive, I had planned to name this
post, <em>Goodbye Cygwin, hello coLinux</em>. After 2 months of using
coLinux, I still am using Cygwin a little bit. Despite being slow, (almost
unbearably slow with my company's mandatory Whole Disk Encryption), Cygwin
is good at integrating the GNU tools with Windows. Currently, I am still using
Cygwin to script my Windows-only project version control system and use the
Cygwin version of Mercurial for my personal version control. I'm not sure
if using the Linux version of Mercurial would work on Samba shares. The more
I learn about Linux and the less I depend on Windows applications, the less I
will need Cygwin.
(<em>Update: Mercurial 1.0 works great on Samba shares. The important thing
is to setup the owner of the share to be your user id and not root.
See <a href="#samba">samba config</a> below.</em>)</p>
<span id="fullpost">
</span><p>Besides that, coLinux is great. The best part for me was being able
to run the dynamic, tiling, scriptable, keyboard-driven
window manager, <a href="http://www.suckless.org/wiki/wmii">wmii</a>.
I access Windows files (and network drives) using <a href="http://www.samba.org">
Samba</a>. Shell commands, emacs, ssh are all there and are so fast.
GUI applications such as Firefox, OpenOffice and Gimp are available as well. However,
these are slow with the slirp networking setup. With a TAP connection between
coLinux and the host OS, these apps run well, however, I couldn't get the
TAP connection to work in conjuction with internet access so I settled for slirp
only. If I figure this out, I will update the post.
Here are my install notes.</p>
<p>Update 4/28/2008: I got my private TAP connection to the host working
along with a slirp connection for internet. This makes my X server connections
fast enough to run all my GUI apps like OpenOffice, <strike>evince</strike>
kpdf for pdf
files, and my web browser. See <a href="#networking2">this section</a>
for details.</p>
<br /><strong><a name="toc">Contents</a></strong><br />
<ul>
<li><a href="#install">Install coLinux</a></li>
<li><a href="#config">Configure CoLinux</a></li>
<li><a href="#run">Run CoLinux</a></li>
<li><a href="#networking">Configure networking with slirp only</a></li>
<li><a href="#networking2">Configure networking with slirp and TAP</a></li>
<li><a href="#update">Update your Ubuntu Dapper installation</a></li>
<li><a href="#xterm">Run an Xterm and other apps</a></li>
<li><a href="#expandroot">Expand the root file system image</a></li>
<li><a href="#hardy">Upgrade to Ubuntu Hardy</a></li>
<li><a href="#startup">Disable some startup services</a></li>
<li><a href="#setup">Configure Ubuntu</a></li>
<li><a href="#homefs">Create a home filesystem image</a></li>
<li><a href="#swap">Setup the swap file</a></li>
<li><a href="#samba">Mount Windows filesystem as Samba share</a></li>
<li><a href="#wmii">Set up wmii window manager</a></li>
</ul>
<br /><strong><a href="#toc" name="install">Install coLinux</a></strong><br />
<ol>
<li>Download the coLinux installer and the Ubuntu root file system image
<ul>
<li>Go to <a href="http://sourceforge.net/projects/colinux/files">http://sourceforge.net/projects/colinux/files</a></li>
<li>Under coLinux-stable, 0.7.2-linux-2.6.22, download coLinux-0.7.2.exe
</li>
<li>Under Root FS Images - 2.6.x-based, Ubuntu,
download Ubuntu-6.06.1.ext3.1gb.bz2</li>
</ul>
</li>
<li>Run the coLinux Windows installer
<ul>
<li>Accept defaults except for:</li>
<li>Destination Folder: change to c:\coLinux</li>
<li>You do not need to install WinPCap library.</li>
<li>You will get a warning dialog saying that "TAP-Win32 Adapter V8 (coLinux)"
has not passed Windows Logo testing. Click "Continue Anyway"</li>
</ul>
</li>
</ol>
<br /><strong><a href="#toc" name="config">Configure CoLinux</a></strong><br />
I already had Cygwin running so I did this stuff in a bash shell.
You can use Windows commands as well to do the same thing.
<ol>
<li>Copy <code>Ubuntu-6.06.1.ext3.1bg.bz2</code> to <code>c:\coLinux</code>.
(Keep a copy of this file as a backup to revert your Ubuntu filesystem
if you mess things up.)
</li>
<li>Decompress the Ubuntu file system image. (Note, this could take a minute or more depending on your system.)<br />
<code>$ cd /cygdrive/c/coLinux</code><br />
<code>$ bunzip2 Ubuntu-6.06.1.ext3.1gb.bz2</code>
</li>
<li>Create a swap file<br />
<code>$ dd if=/dev/zero of=swap_device bs=1M count=512</code><br />
<code>$ dd if=/dev/zero of=swap_device.2gb bs=1M count=2K</code><br />
(In Windows, you can use <code>fsutil file createnew swap_device 536870912
</code>)
</li>
<li>Create a configuration file<br />
<code>$ cp example.conf colinux.conf</code><br />
Edit colinux.conf and set the following:<br />
<pre>cobd0="c:\coLinux\Ubuntu-6.06.1.ext3.1gb"
cobd1="c:\coLinux\swap_device"
mem=512
eth0=slirp
eth1=tuntap # only needed for TAP connection</pre>
The following items should already be set by default:<br />
<pre>kernel=vmlinux
root=/dev/cobd0
ro
initrd=initrd.gz</pre>
</li>
</ol>
<br /><strong><a href="#toc" name="run">Run CoLinux</a></strong><br />
<ol>
<li>Open a Windows <code>cmd.exe</code> shell</li>
<li><code>cd c:\colinux</code></li>
<li><code>colinux-daemon.exe -t nt @colinux.conf</code><br />
(the -t nt is optional but makes it easier to copy/paste)</li>
<li>login as "root" with password "root"</li>
<li>Try commands such as <code>ls</code>, <code>pwd</code>, etc.</li>
<li>To exit, use the <code>halt</code> command.</li>
</ol>
<br /><strong><a href="#toc" name="networking">Configure networking with
slirp only</a></strong><br />
<ol>
<li>If you use a http proxy server to get to the internet, do the following
inside the running CoLinux session:<br />
<code>root@ubuntu:~# echo "export http_proxy='http://your-proxy-server.com:yourportnumber'" >> .bash_profile</code><br />
<code>root@ubuntu:~# source .bash_profile</code>
</li>
<li>Otherwise, there is nothing to do. The Ubuntu image already has the
correct /etc/network/interfaces and /etc/resolv.conf files setup.<br /><br />
/etc/network/interfaces should look like this:<br />
<pre># This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface (slirp)
auto eth0
iface eth0 inet static
address 10.0.2.15
broadcast 10.0.2.255
netmask 255.255.255.0
gateway 10.0.2.2
</pre>
</li>
<br />/etc/resolv.conf should look like this:<br />
<pre>nameserver 10.0.2.3</pre>
<li>To test internet access, use something like <code>wget</code> since <code>
ping</code> doesn't work with slirp.<br />
<code>root@ubuntu:~# wget www.google.com</code>
</li>
</ol>
<br /><strong><a href="#toc" name="networking2">Configure networking with
slirp to internet and TAP to host</a></strong><br />
In this configuration, I set up TAP to form a private connection to the
host and slirp to access the internet. If you want to use X applications,
this will make the response much better than with slirp only. I had to
play around with the IP addresses to find one that didn't conflict with
the static IP addresses on our network. I ended up choosing "10.0.3.16"
for the TAP colinux address and "10.0.3.17" for my Windows side.
<ol>
<li>TAP should be installed when you install coLinux</li>
<li>In Windows, go to "Start", "Control Panel", "Network Connections"</li>
<li>Find the "TAP-Win32 Adapter V8 (coLinux)" adapter. For me, it was named
"Local Area Connection 2". Right click on it and select "Properties".</li>
<li>On the "General" tab, scroll down and select "Internet Protocol (TCP/IP)".
Click on "Properties".</li>
<li>Select "Use the following IP address:". Enter in "10.0.3.17" for the
"IP address:" and "255.255.254.0" for the "Subnet mask:". Leave the rest
blank. Click "OK" twice to finish.</li>
<li>in coLinux, edit /etc/network/interfaces to look like the following:
<pre>auto lo
# Loopback interface
iface lo inet loopback
# Network interface for internet access (slirp)
auto eth0
iface eth0 inet static
address 10.0.2.15
broadcast 10.0.2.255
netmask 255.255.255.0
gateway 10.0.2.2
# Network interface for private connection to Host OS (tap-win32)
auto eth1
iface eth1 inet static
address 10.0.3.16
network 10.0.3.0
netmask 255.255.254.0
broadcast 10.0.3.255</pre>
</li>
<li>Make sure your "colinux.conf" file has the following lines:
<pre>eth0=slirp
eth1=tuntap</pre>
</li>
<li>You will need to set your DISPLAY variable to "10.0.3.17:0"<br />
<code>root@ubuntu:~# export DISPLAY=10.0.3.17:0</code>
</li>
</ol>
<br /><strong><a href="#toc" name="update">Update your Ubuntu Dapper installation</a></strong><br />
<ol>
<li><code>root@ubuntu:~# apt-get update</code><br />
You can ignore the following warning:
<pre>W: GPG error: http://security.ubuntu.com dapper-security Release: Could not exec
ute /usr/bin/gpgv to verify signature (is gnupg installed?)
W: GPG error: http://fr.archive.ubuntu.com dapper-updates Release: Could not exe
cute /usr/bin/gpgv to verify signature (is gnupg installed?)
W: GPG error: http://fr.archive.ubuntu.com dapper-backports Release: Could not e
xecute /usr/bin/gpgv to verify signature (is gnupg installed?)
W: GPG error: http://fr.archive.ubuntu.com dapper Release: Could not execute /us
r/bin/gpgv to verify signature (is gnupg installed?)
W: You may want to run apt-get update to correct these problems</pre>
</li>
<li><code>root@ubuntu:~# apt-get upgrade</code><br />
Answer yes to prompts.</li>
<li>That was easy.</li>
</ol>
<br /><strong><a href="#toc" name="xterm">Run an Xterm and other apps</a></strong><br />
<p><em>Warning, getting the X Window System working properly was one of the most
frustrating parts of the setup for me. The behavior seemed to be even
non-deterministic. This is what worked for me. You may need to consult other
sources to get it working for you. I am starting to read </em><em>X Power Tools</em>
to learn more about the X Window System and will update this section if I learn
anything new. I used Cygwin's X server because I already had Cygwin installed.
Another option is <a href="http://www.freedesktop.org/wiki/Xming">Xming</a>.</p>
<p>Update 5/6/2008: After switching to the Slirp+TAP networking configuration,
I've been getting errors when trying to start my X clients:
<pre style="height: 40px; overflow:auto">$ xeyes
No protocol specified
Error: Can't open display: 10.0.3.17:0
$ xterm
No protocol specified
xterm Xt error: Can't open display: 10.0.3.17:0
$ urxvt
No protocol specified
urxvt: can't open display 10.0.3.17:0, aborting.</pre>
To resolve this, I have been starting up a Cygwin bash shell, and running the
<code>xhost +</code> command to disable access control. This worked, but it
was annoying to do this all the time. I discovered that there is a <code>-ac</code>
command line option to <code>Xwin</code> which will do the same thing.
I've updated my batch file below to reflect this. Here is a list of all the
<code>Xwin</code> command line options:
<pre style="height: 50px; overflow:auto">use: X [:<display>] [option]
-a # mouse acceleration (pixels)
-ac disable access control restrictions
-audit int set audit trail level
-auth file select authorization file
bc enable bug compatibility
-br create root window with black background
+bs enable any backing store support
-bs disable any backing store support
-c turns off key-click
c # key-click volume (0-100)
-cc int default color visual class
-co file color database file
-core generate core dump on fatal error
-dpi int screen resolution in dots per inch
-deferglyphs [none|all|16] defer loading of [no|all|16-bit] glyphs
-f # bell base (0-100)
-fc string cursor font
-fn string default font name
-fp string default font path
-help prints message with these options
-I ignore all remaining arguments
-ld int limit data space to N Kb
-lf int limit number of open files to N
-ls int limit stack space to N Kb
-logo enable logo in screen saver
nologo disable logo in screen saver
-nolisten string don't listen on protocol
-noreset don't reset after last client exists
-reset reset after last client exists
-p # screen-saver pattern duration (minutes)
-pn accept failure to listen on all ports
-nopn reject failure to listen on all ports
-r turns off auto-repeat
r turns on auto-repeat
-render [default|mono|gray|color] set render color alloc policy
-s # screen-saver timeout (minutes)
-sp file security policy file
-su disable any save under support
-t # mouse threshold (pixels)
-terminate terminate at server reset
-to # connection time out
-tst disable testing extensions
ttyxx server started from init on /dev/ttyxx
v video blanking for screen-saver
-v screen-saver without video blanking
-wm WhenMapped default backing-store
-x string loads named extension at init time
-maxbigreqsize set maximal bigrequest size
+extension name Enable extension
-extension name Disable extension
-query host-name contact named host for XDMCP
-broadcast broadcast for XDMCP
-indirect host-name contact named host for indirect XDMCP
-port port-num UDP port number to send messages to
-from local-address specify the local address to connect from
-once Terminate server after one session
-class display-class specify display class to send in manage
-cookie xdm-auth-bits specify the magic cookie for XDMCP
-displayID display-id manufacturer display ID for request
The X Keyboard Extension adds the following arguments:
-kb disable the X Keyboard Extension
+kb enable the X Keyboard Extension
[+-]accessx [ timeout [ timeout_mask [ feedback [ options_mask] ] ] ]
enable/disable accessx key sequences
-ar1 set XKB autorepeat delay
-ar2 set XKB autorepeat interval
-noloadxkb don't load XKB keymap description
-xkbdb file that contains default XKB keymaps
-xkbmap XKB keyboard description to load on startup
-depth bits_per_pixel
Specify an optional bitdepth to use in fullscreen mode
with a DirectDraw engine.
-emulate3buttons [timeout]
Emulate 3 button mouse with an optional timeout in
milliseconds.
-engine engine_type_id
Override the server's automatically selected engine type:
1 - Shadow GDI
2 - Shadow DirectDraw
4 - Shadow DirectDraw4 Non-Locking
-fullscreen
Run the server in fullscreen mode.
-refresh rate_in_Hz
Specify an optional refresh rate to use in fullscreen mode
with a DirectDraw engine.
-screen scr_num [width height [x y] | [[WxH[+X+Y]][@m]] ]
Enable screen scr_num and optionally specify a width and
height and initial position for that screen. Additionally
a monitor number can be specified to start the server on,
at which point, all coordinates become relative to that
monitor (Not for Windows NT4 and 95). Examples:
-screen 0 800x600+100+100@2 ; 2nd monitor offset 100,100 size 800x600
-screen 0 1024x768@3 ; 3rd monitor size 1024x768
-screen 0 @1 ; on 1st monitor using its full resolution (the default)
-lesspointer
Hide the windows mouse pointer when it is over an inactive
Cygwin/X window. This prevents ghost cursors appearing where
the Windows cursor is drawn overtop of the X cursor
-nodecoration
Do not draw a window border, title bar, etc. Windowed
mode only.
-mwextwm
Run the server in multi-window external window manager mode.
-internalwm
Run the internal window manager.
-rootless
Run the server in rootless mode.
-multiwindow
Run the server in multi-window mode.
-multiplemonitors
EXPERIMENTAL: Use the entire virtual screen if multiple
monitors are present.
-clipboard
Run the clipboard integration module.
Do not use at the same time as 'xwinclip'.
-nounicodeclipboard
Do not use Unicode clipboard even if NT-based platform.
-scrollbars
In windowed mode, allow screens bigger than the Windows desktop.
Moreover, if the window has decorations, one can now resize
it.
-[no]trayicon
Do not create a tray icon. Default is to create one
icon per screen. You can globally disable tray icons with
-notrayicon, then enable it for specific screens with
-trayicon for those screens.
-clipupdates num_boxes
Use a clipping region to constrain shadow update blits to
the updated region when num_boxes, or more, are in the
updated region. Currently supported only by `-engine 1'.
-[no]unixkill
Ctrl+Alt+Backspace exits the X Server.
-[no]winkill
Alt+F4 exits the X Server.
-xkbrules XKBRules
Equivalent to XKBRules in XF86Config files.
-xkbmodel XKBModel
Equivalent to XKBModel in XF86Config files.
-xkblayout XKBLayout
Equivalent to XKBLayout in XF86Config files.
For example: -xkblayout de
-xkbvariant XKBVariant
Equivalent to XKBVariant in XF86Config files.
For example: -xkbvariant nodeadkeys
-xkboptions XKBOptions
Equivalent to XKBOptions in XF86Config files.
-logfile filename
Write logmessages to <filename> instead of /tmp/Xwin.log.
-logverbose verbosity
Set the verbosity of logmessages. [NOTE: Only a few messages
respect the settings yet]
0 - only print fatal error.
1 - print additional configuration information.
2 - print additional runtime information [default].
3 - print debugging and tracing information.
-[no]keyhook
Grab special windows key combinations like Alt-Tab or the Menu key.
These keys are discarded by default.
-swcursor
Disable the usage of the windows cursor and use the X11 software cursor instead</filename></display></pre>
You can view this list by typing <code>Xwin --help</code> in a Cygwin bash
shell.
Also, in the process, I tried the Xming X server. However, I couldn't get it
to work with wmii, so I decided to stick with Cygwin's X server.</p>
<ol>
<li>Edit your startxwin.bat file (located in "C:\cygwin\usr\X11R6\bin")
so that it starts X in rootless mode, with clipboard support, disables
access control, and doesn't run anything else.
Run the new bat file.<br />
<pre>@echo off
SET DISPLAY=127.0.0.1:0.0
SET CYGWIN_ROOT=\cygwin
SET RUN=%CYGWIN_ROOT%\bin\run -p /usr/X11R6/bin
SET PATH=.;%CYGWIN_ROOT%\bin;%CYGWIN_ROOT%\usr\X11R6\bin;%PATH%
SET XAPPLRESDIR=/usr/X11R6/lib/X11/app-defaults
SET XCMSDB=/usr/X11R6/lib/X11/Xcms.txt
SET XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB
SET XNLSPATH=/usr/X11R6/lib/X11/locale
if not exist %CYGWIN_ROOT%\tmp\.X11-unix\X0 goto CLEANUP-FINISH
attrib -s %CYGWIN_ROOT%\tmp\.X11-unix\X0
del %CYGWIN_ROOT%\tmp\.X11-unix\X0
:CLEANUP-FINISH
if exist %CYGWIN_ROOT%\tmp\.X11-unix rmdir %CYGWIN_ROOT%\tmp\.X11-unix
%RUN% XWin -clipboard -silent-dup-error -rootless -ac</pre>
</li>
<li>If you are using Slirp-only networking:
<ul>
<li>Obtain your Windows host IP address by opening a cmd.exe shell and
running <code>ipconfig</code>
</li>
<li>In coLinux, set your DISPLAY variable to this IP address:<br />
<code>root@ubuntu:~# export DISPLAY=xxx.xxx.xxx.xxx:0.0</code><br />
where <code>xxx.xxx.xxx.xxx</code> is your IP address. Don't forget
the <code>:0.0</code> at the end.<br />
</li>
</ul>
</li>
<li>If you are using TAP plus Slirp configuration described above,
set your DISPLAY variable to "10.0.3.17:0"<br />
<code>root@ubuntu:~# export DISPLAY=10.0.3.17:0</code>
</li>
<li><code>root@ubuntu:~# apt-get install xterm</code><br />
Answer yes to prompts.
</li>
<li><code>root@ubuntu:~# xterm&</code><br />
Xterm comes up!
</li>
<li>You can now install and run other high level apps like firefox or emacs</li>
</ol>
<br /><strong><a href="#toc" name="expandroot">Expand the root file system image</a></strong><br />
You will need to expand the size your root filesystem image to
hold the larger Ubuntu installation. I expanded it to 4GB for Xubuntu Hardy
and am currently using about 87% of that. You might want to expand it to 5GB
or more especially if you will be installing a lot of apps. See
<a href="http://colinux.wikia.com/wiki/ExpandingRoot">
http://colinux.wikia.com/wiki/ExpandingRoot</a> for more information.
Note, Google's calculator function
can be useful for calculating block sizes. (e.g. entering in <code>4GB / 16KB</code>
into the Google search box returns: <em>(4 gigabytes) / (16 kilobytes) = 262 144</em>)
<ol>
<li>If you have coLinux running, shut it down, with <code>halt</code></li>
<li>In a Cygwin bash shell, make a copy of your current image.<br />
<code>$ cp Ubuntu-6.06.1.ext3.1gb Ubuntu-6.06.1.ext3.4gb</code><br />
This may take a while.
</li>
<li>Expand the new file:<br />
<code>$ dd bs=16384 if=/dev/zero of=Ubuntu-6.06.1.ext3.4gb seek=65536 count=262144</code><br /><br />
bs=16384 sets the blocksize to 16KB<br />
seek=65536 skips the first 65536 blocks (1GB/16KB = 65536)<br />
count=262144 adds 262144 blocks to the image (4GB/16KB = 262144)<br /><br />
This may take a while also. It should display the following when complete:
<pre>262144+0 records in
262144+0 records out
4294967296 bytes (4.3 GB) copied, 154.362 s, 27.8 MB/s</pre>
</li>
<li>Change your colinux.conf file to use the new file system image:<br />
<pre>cobd0="c:\coLinux\Ubuntu-6.06.1.ext3.4gb"</pre>
</li>
<li>Switch back to the cmd.exe console, and start coLinux again<br />
</li>
<li><code>root@ubuntu:~# apt-get install ext2resize</code></li>
<li><code>root@ubuntu:~# ext2online /dev/cobd0</code></li>
<li>Look at your available space:<br />
<code>root@ubuntu:~# df</code>
<pre>Filesystem 1K-blocks Used Available Use% Mounted on
/dev/cobd0 4128448 411148 3507624 11% /
varrun 128096 36 128060 1% /var/run
varlock 128096 0 128096 0% /var/lock
devshm 128096 0 128096 0% /dev/shm</pre>
</li>
<li>Note, you may want to make a backup copy of your new expanded root file
system image at this point in case things get hosed up later on. Be sure
to <code>halt</code> coLinux first. Note, this will take even longer than
the 1gb filesystem copy.
</li>
</ol>
<br /><strong><a href="#toc" name="hardy">Upgrade to Ubuntu Hardy</a></strong><br />
Update 4/28/2008: When I first installed coLinux, Ubuntu Gutsy was the latest
release. Because there was no direct upgrade path from Ubuntu 6.06 Dapper to
Ubuntu 7.10 Gutsy, I had to incrementally upgrade from Dapper to Edgy, Edgy to
Feisty, and Feisty to Gutsy. Later, when Hardy came out, I upgraded from
Gutsy to Hardy. If you are upgrading to Dapper to Hardy for the first time,
you can use the incremental method, or the direct method. Ubuntu supports
a direct upgrade from 6.06 Dapper Drake to 8.04 Hardy Heron because both are
Long Term Support (LTS) releases. See the
<a href="https://help.ubuntu.com/community/HardyUpgrades">Ubuntu upgrade
documentation</a> for more information. Note, I did not test the direct
upgrade method.
<ol>
<li>In coLinux:<br />
<code>root@ubuntu:~# apt-get update</code></li>
<li><code>root@ubuntu:~# apt-get install ubuntu-minimal</code></li>
<li><code>root@ubuntu:~# apt-get install ubuntu-standard</code></li>
<li><code>root@ubuntu:~# apt-get install xubuntu-desktop</code></li>
<li>Edit your sources.list file and change "dapper" to "edgy"<br />
<code>root@ubuntu:~# nano /etc/apt/sources.list</code><br />
</li>
<li><code>root@ubuntu:~# apt-get update</code></li>
<li><code>root@ubuntu:~# apt-get upgrade</code></li>
<li><code>root@ubuntu:~# apt-get dist-upgrade</code></li>
<li>Change "edgy" to "feisty" in /etc/apt/sources.list and repeat steps 6-8.</li>
<li>Change "feisty" to "gutsy" in /etc/apt/sources.list and repeat steps 6-8.</li>
<li>Change "gutsy" to "hardy" in /etc/apt/sources.list and repeat steps 6-8.</li>
<li>Remove unneeded packages:<br />
<code>root@ubuntu:~# apt-get autoremove</code>
</li>
</ol>
<br /><strong><a href="#toc" name="startup">Disable some startup services</a></strong><br />
This is optional but may give you fewer warnings. In coLinux, go to the
/etc/rc2.d directory and follow the instructions in the README file for disabling
services. Here are some suggested services to disable:
<ul>
<li>acpid, acpi-support, apmd: used for power management</li>
<li>bluetooth: used for bluetooth</li>
<li>hotkey-setup: used to configure laptop hotkeys</li>
<li>pcmciautils: tools for PCMCIA cards (PC cards)</li>
<li>powernowd: controls CPU speed and voltage using the sysfs interface</li>
<li>vbesave: used to save the video card state</li>
</ul>
<br /><strong><a href="#toc" name="setup">Configure Ubuntu</a></strong><br />
<ol>
<li>Don't start gdm (the display manager which starts the X server)<br />
<code>root@ubuntu:~# mv /etc/rc2.d/S13gdm /etc/rc2.d/K87gdm</code></li>
<li>Set the timezone<br />
Start your Cygwin X server if it is not already running.<br />
<code>root@ubuntu:~# time-admin &</code><br />
Set your timezone.
</li>
<li>Set the root password<br />
<code>root@ubuntu:~# passwd</code>
</li>
<li>Set local machine name:<br />
<code>root@ubuntu:~# nano /etc/hostname</code><br />
<code>root@ubuntu:~# nano /etc/hosts</code><br />
</li>
<li>Create a user account<br />
<code>root@ubuntu:~# adduser sofeng</code><br />
Note, adduser is a higher level command than useradd. adduser will set up
a group and home directory.
</li>
<li>Set permissions on /dev/tty* (Note: not sure if this is needed)<br />
<code>root@ubuntu:~# chgrp tty /dev/tty*</code><br />
Edit /etc/group to have the following line:<br />
<pre>tty:x:5:sofeng</pre>
</li>
<li>Make a pty device<br />
<code>root@ubuntu:~# cd /dev</code><br />
<code>root@ubuntu:~# MAKEDEV pty</code><br />
</li>
<li>Add environment variables to .bash_profile<br />
Use the <code>printenv</code> command to list your environment variables.
</li>
</ol>
<br /><strong><a href="#toc" name="homefs">Create a home filesystem image</a></strong><br />
<ol>
<li>Create a 10GB blank file. In Cygwin,<br />
<pre>$ dd if=/dev/zero of=home-fs-image.10gb bs=1M count=10K</pre>
</li>
<li>Add the following to your c:\coLinux\colinux.conf file:<br />
<pre>cobd2="c:\coLinux\home-fs-image.10gb"</pre>
</li>
<li>Restart coLinux, then do the following:<br />
Create an ext2 file system on the new disk image<br />
<pre>root@ubuntu:~# mke2fs -F /dev/cobd2</pre>
Turn this into an ext3 filesystem<br />
<pre>root@ubuntu:~# tune2fs -i 0 -j /dev/cobd4</pre>
</li>
<li>Mount the new filesystem to /home<br />
<pre>root@ubuntu:~# mv /home /home_backup
root@ubuntu:~# mkdir /home
root@ubuntu:~# mount /dev/cobd2 /home</pre>
Check it worked:<br />
<pre>root@ubuntu:~# df</pre>
</li>
<li>To mount the filesystem automatically at startup, add the following line
to your /etc/fstab:<br />
<pre>/dev/cobd2 /home ext3 defaults 0 0</pre>
</li>
</ol>
<br /><strong><a href="#toc" name="swap">Setup the swap file</a></strong><br />
<ol>
<li>Add the following to /etc/fstab:<br />
<pre>/dev/cobd1 swap swap defaults 0 0</pre>
</li>
<li>Initialize the swap partition<br />
<pre>root@ubuntu:~# sudo mkswap /dev/cobd1</pre>
</li>
<li>make use of swap now, without rebooting system:<br />
<pre>root@ubuntu:~# swapon -a</pre>
</li>
<li>Check that swap total is non-zero:<br />
<pre>root@ubuntu:~# free</pre>
</li>
</ol>
<br /><strong><a href="#toc" name="samba">Mount Windows filesystem as Samba share</a></strong>
<ol>
<li>Setup Windows networking:<br />
<ul>
<li>In Windows Explorer, right click on your C: drive and select "Properties"</li>
<li>Click on the "Sharing" tab</li>
<li>Select "Share this folder". Set the "Share name:" to "E$". Setup "Permissions" as
necessary.</li>
</ul>
</li>
<li>create the location for your samba share:<br />
<code>$ sudo mkdir -p /mnt/cdrive</code>
</li>
<li>add the following to /etc/fstab:<br />
<pre>//yourcomputer.example.com/E$ /mnt/cdrive smbfs auto,user,rw,dir_mode=0777,file_mode=0777,uid=sofeng,gid=sofeng,credentials=/etc/smb-credentials 0 0</pre>
<code>/mnt/cdrive</code> is the mount point<br />
<code>smbfs</code> means it is a samba filesystem type<br />
<code>auto</code> means it will be mounted at startup<br />
<code>user</code> means any user can mount it<br />
<code>rw</code> means it will be readable and writable<br />
<code>file_mode=0777</code> specifies that files will be readable, writable, and executable by everyone (this replaces the deprecated fmask)<br />
<code>dir_mode=0777</code> specifies that directories will be readable, writable, and executable by everyone (this replaces the deprecated dmask)<br />
<code>uid=sofeng</code> means the owner is sofeng<br />
<code>gid=sofeng</code> means the group is sofeng<br />
<code>credentials=/etc/smb-credentials</code> specifies the file that contains the user name and password<br />
</li>
<li>create /etc/smb-credentials<br />
<pre>username=yourdomain\yourusername
password=yourpassword</pre>
<code>$ sudo chmod 600 /etc/smb-credentails</code>
</li>
<li>Mount the drive:<br />
<code>$ sudo mount /mnt/cdrive</code>
</li>
</ol>
<br /><strong><a href="#toc" name="wmii">Set up wmii window manager</a></strong><br />
<ol>
<li>Install wmii window manager<br />
<code>apt-get install wmii</code>
</li>
<li>I use emacs so the following steps are used to change the mod key.</li>
<li>Download <a href="http://webpages.charter.net/krumsick/">KeyTweak</a> for
Windows and remap your "Left Windows" key to "Context Menu".</li>
<li>Use xmodmap to setup the "Menu" key to be "mod4". For my keyboard, my
<code>~/.Xmodmap</code> file looks like this:<br />
<pre>clear mod4
keycode 115 =
keycode 127 =
keycode 117 = Super_L
add mod4 = Super_L</pre>
Then run xmodmap in one of your startup scripts (.xinitrc, .bash_profile, etc.)<br />
<pre>xmodmap ~/.Xmodmap</pre>
</li>
</ol>
Backup on Linux: rsnapshot vs. rdiff-backup (vs. Time Machine)
2008-02-07T13:22:00-08:00https://www.saltycrane.com/blog/2008/02/backup-on-linux-rsnapshot-vs-rdiff/<p>Apple's release of Leopard and the included backup utility,
<a href="http://www.apple.com/macosx/features/timemachine.html">
Time Machine</a>, has generated a lot of talk about backups recently. I will
admit Time Machine is pretty cool and believe that it is a bit more than
a glorified GUI on top of an existing *nix tool as some have claimed.
However, the core functionality is very similar to the command-line tool,
<a href="http://www.rsnapshot.org/">rsnapshot</a>, which is itself
based on a <a href="http://samba.org/rsync/">rsync</a> script by Mike
Rubel. Time Machine added a couple of features and a GUI to make it
easy to use. Since I prefer the command line over GUIs most of the time
anyways, rsnapshot seemed perfect for me.</p>
<p>To be complete, I researched a number of other backup utilities for Linux.
<a href="http://www.dirvish.org/">
Dirvish</a> and <a href="http://code.google.com/p/flyback/">flyback</a>
were out because I prefer the command line and they didn't seem to offer
anything more than rsnapshot. Scripting rsync wouldn't get me anything
more than rsnapshot either, plus it would be more work. In the end,
I eliminated all but rsnapshot and another command line tool called
<a href="http://www.nongnu.org/rdiff-backup/">
rdiff-backup</a>. Rdiff-backup has some advantages over rsnapshot (and Time
Machine) because it stores compressed deltas of each version of a file
instead of a complete copy of the file each time it changes. This is not
a big deal for small files, but for large files that change often, it
makes a significant
difference. However, the big disadvantage of rdiff-backup, for me, was
the inablility to perform different levels of backup, such as hourly,
daily, weekly, monthly, etc. Depending on the needs of the user, this
could negate the space saving advantage by needing to keep a large number
of snapshots.</p>
<p>I ended up choosing rsnapshot over rdiff-backup because of this last
reason. It seems rdiff-backup is closer to a version control tool and
rsnapshot closer to a traditional backup solution. It would be great to
create a hybrid of the two tools to gain
the advantages of each. I started to dig into the source of rdiff-backup
(python source makes me happy), but I didn't want to get too sidetracked
with another project. For now, I am using <a href="http://www.selenic.com/mercurial/">
Mercurial</a> to version control
my <code>/etc</code> and <code>/home/.*</code> config files and rsnapshot
as a broader, general purpose backup tool.</p>
<br /><p>Here is my comparison of rsnapshot and rdiff-backup:</p>
<table>
<tr>
<td valign="top">Similar Features</td>
<td valign="top">
<ul>
<li>both use an rsync-like algorithm to transfer data (rsnapshot
actually uses rsync; rdiff-backup uses the python librsync library)
</li>
<li>both can be used over ssh (though rsnapshot cannot push over ssh without some extra scripting)</li>
<li>both use a simple copy of the source for the current backup</li>
</ul>
</td>
</tr>
<tr>
<td valign="top">Written in</td>
<td valign="top">rsnapshot is written in Perl; rdiff-backup is written in Python and C</td>
</tr>
<tr>
<td valign="top">Size</td>
<td valign="top">rdiff-backup stores previous versions
as compressed deltas to the current version similar to a version
control system. rsnapshot uses actual files and hardlinks to save
space. For small files, storage size is similar. For large files
that change often, such as logfiles, databases, etc., rdiff-backup
requires significantly less space for a given number of versions.</td>
</tr>
<tr>
<td valign="top">Speed</td>
<td valign="top">rdiff-backup is slower than rsnapshot</td>
</tr>
<tr>
<td valign="top">Metadata</td>
<td valign="top">rdiff-backup stores file metadata, such as ownership, permissions, and
dates, separately.</td>
</tr>
<tr>
<td valign="top">Transparency</td>
<td valign="top">For rsnapshot, all versions of the backup are accessible as plain files.
For rdiff-backup, only the current backup is accessible as plain files.
Previous versions are stored as rdiff deltas.</td>
</tr>
<tr>
<td valign="top">Backup levels</td>
<td valign="top">rsnapshot supports multiple levels of backup such as monthly, weekly,
and daily. rdiff-backup can only delete snapshots earlier than a given
date; it cannot delete snapshots in between two dates.</td>
</tr>
<tr>
<td valign="top">Community</td>
<td valign="top">
Based on the number of responses to my post on the mailing lists
(<a href="https://sourceforge.net/p/rsnapshot/mailman/rsnapshot-discuss/thread/592a1f150802041321y61b3a2a3j567f1e62e4cca594%40mail.gmail.com/#msg18526014">rsnapshot: 6</a>,
<a href="http://lists.nongnu.org/archive/html/rdiff-backup-users/2008-02/msg00008.html">
rdiff-backup: 0</a>), rsnapshot has a more active
community.
</td>
</tr>
</table>
<p>Reference: <a href="http://search.safaribooksonline.com/0596102461/I11707__ChapterStart__Chapter_7">
Chapter 7 in <em>Backup & Recovery</em> O'Reilly 2007</a></p>
How to get a summary of disk usage with du
2008-02-02T11:48:00-08:00https://www.saltycrane.com/blog/2008/02/how-to-get-summary-of-disk-usage-with/<p>The following command gives me a listing of the sizes of all the directories in my root directory.</p>
<pre>$ sudo du -chs /*</pre>
<br /><p>Here is the results of that command on my new (1 day old) Dell 530N Ubuntu desktop: (Note, the 5.0GB for the <code>/home</code> directory is due to the 5.0GB <code>ubuntu-dell-reinstall.iso</code> file. The rest of my <code>/home</code> directory is only 18MB.)</p>
<pre>4.8M /bin
18M /boot
0 /cdrom
128K /dev
9.1M /etc
5.0G /home
4.0K /initrd
0 /initrd.img
154M /lib
16K /lost+found
12K /media
4.0K /mnt
13M /opt
0 /proc
740K /root
6.3M /sbin
4.0K /srv
0 /sys
80K /tmp
2.1G /usr
317M /var
0 /vmlinuz
7.6G total</pre>
<br /><p>To determine disk usage in a different directory, <code>cd</code> to that directory and run <code>sudo du -chs *</code></p>
Undoing changes by pppoeconf
2008-02-02T07:56:00-08:00https://www.saltycrane.com/blog/2008/02/undoing-changes-by-pppoeconf/
<p>I just got my new Dell with Ubuntu and thought I needed to configure something to get my new DSL service up and running so I ran <code>pppoeconf</code>. (It turns out I needed to register with ATT first. See my <a href="http://www.saltycrane.com/blog/2008/02/setting-up-att-yahoo-dsl-with-ubuntu/">previous post</a>.) So I needed to undo the changes I made with <code>pppoeconf</code>. Googling around a little, it appears there is no undo command for pppoeconf. However, I found the key configuration file was: <code>/etc/network/interfaces</code>. I looked at it, but wasn't sure which stuff to delete. I popped in my Ubuntu live installation disk (included with my Dell). Using the live CD, the internet worked great. So I looked at the <code>/etc/network/interfaces</code> while using the live CD and then edited my installed version to match.</p>
<br /><p>Here is what my <code>/etc/network/interfaces</code> file looked like after running <code>pppoeconf</code>:</p>
<pre>auto lo
iface lo inet loopback
auto dsl-provider
iface dsl-provider inet ppp
pre-up /sbin/ifconfig eth0 up # line maintained by pppoeconf
provider dsl-provider
auto eth0
iface eth0 inet manual</pre>
<br /><p>Here is the default <code>/etc/network/interfaces</code> on Ubuntu 7.10 Gutsy (only two lines):</p>
<pre>auto lo
iface lo inet loopback</pre>
Setting up ATT Yahoo DSL with Ubuntu linux
2008-02-02T07:01:00-08:00https://www.saltycrane.com/blog/2008/02/setting-up-att-yahoo-dsl-with-ubuntu/
<p>I ordered new ATT/Yahoo DSL service the day before ordering my Dell 530N desktop with Ubuntu 7.10 Gutsy installed. ATT told me only Windows and Mac were supported for their DSL service. I guess they have an install CD that only runs on Windows or Mac. After some internet searching, I found that it is possible to set up the DSL service without the install CD. <a href="http://www.dslreports.com/faq/8346">Here is the link</a> which helped me with the process. The following is the summary of my steps on Ubuntu linux. I am using a SpeedStream DSL modem; this may or may not work with other modems.</p>
<ol>
<li>Connect the modem to everything (phone line to phone jack, ethernet cable to your PC, and, of course, the power adapter).</li>
<li>Turn on the modem.</li>
<li>On the underside of my modem was a yellow sticker that said "For Advanced Modem Configuration Go to: http://192.169.0.1"</li>
<li>Go to <code>http://192.169.0.1</code> in your web browser</li>
<li>Enter the following login information:
<ul><li>Member ID: sbcyahooreg@sbcglobal.net</li>
<li>Password: sbcyahooreg </li>
</ul></li>
<li>Enter in the Modem Access Code found on the bottom of your DSL modem.</li>
<li>It will look like you are connected to the internet at this point, but you won't be able to get to Yahoo or Google. This login provides limited connectivity so you can get to the ATT registration site.</li>
<li>Go to <code>https://sbcreg.sbcglobal.net</code> to get to the registration site. (If you are part of another domain, you will need to go to a different registration site. e.g. att.net customers go to http://helpme.att.net/register)</li>
<li>Complete the registration process. You will need your DSL phone number. When you get to the Member ID and password confirmation page, you are done. Print/write down/memorize this information. You do not need to complete the rest of the registration process.</li>
<li>Go back to <code>http://192.169.0.1</code> in your web browser and enter in your new Member ID and password. (You will probably have to logout and then login again.)</li>
<li>You should now be connected to the internet.</li>
</ol>
<p>Note: this assumes the default Ubuntu 7.10 network configuration which automatically detects your active DSL connection. If you ran <code>pppoeconf</code> by mistake, see <a href="http://www.saltycrane.com/blog/2008/02/undoing-changes-by-pppoeconf/">this post</a>.</p>
Dell 530N Ubuntu vs. build it yourself
2008-02-01T22:56:00-08:00https://www.saltycrane.com/blog/2008/02/dell-530n-ubuntu-vs-build-it-yourself/<p>I just received my new Dell 530N with Ubuntu 7.10 pre-loaded. I seriously considered building a PC myself but after weighing the pros and cons decided to go with the Dell. The main reason I chose the Dell was because it was the faster/easier option. The cost comparison was pretty similar. Hal's Notes has <a href="http://digimojo.blogspot.com/2007/09/dell-ubuntu-vs-build-it-yourself.html">an article</a> which compares the cost of the 530N vs. the cost of parts from newegg.com. Here is my very similar anaylsis.</p>
<p>My 530N was $349 (unfortunately i had to pay $30 for shipping) and included the following:</p>
<ul>
<li>Intel Pentium Dual Core processor E2160 (1.80GHz,800FSB)</li>
<li>1GB DDR2 SDRAM at 667MHz</li>
<li>128MB NVIDIA GeForce 8300GS</li>
<li>250GB SATA II Hard Drive (7200RPM)</li>
<li>48X CDRW/DVD Combination Drive</li>
</ul>
<p>The only thing I upgraded was the processor-- from the E2140 to the E2160 because I had read it was better for overclocking (if it turns out to be possible to overclock with Dell's motherboard). </p>
<p>For the build it myself option, I started with a budget configuration from <a href="http://techreport.com/articles.x/13660/2">The Tech Report</a>. The total for that machine in November 2007 was $519. Adjusting the items to closer match the Dell 530N got me down to $409. See below.</p>
<ul>
<li> LITE-ON Combo Black SATA Model DH-52C2S-04 - OEM <br />
$24.99</li>
<li> Antec NSK4480 Black/ Silver 0.8mm cold-rolled steel construction ATX Mid Tower Computer Case 380W Power Supply - Retail <br />
$79.95</li>
<li> Western Digital Caviar SE16 WD2500KS 250GB 7200 RPM SATA 3.0Gb/s Hard Drive - OEM <br />
$69.99</li>
<li> EVGA 128-P2-N428-LR GeForce 7200GS 512MB (128MB on Board) 64-bit GDDR2 PCI Express x16 Video Card - Retail <br />
$34.99</li>
<li> Kingston 1GB 240-Pin DDR2 SDRAM DDR2 667 (PC2 5300) Desktop Memory Model KVR667D2N5/1G - Retail <br />
$19.99</li>
<li> GIGABYTE GA-P35-DS3L LGA 775 Intel P35 ATX Intel Motherboard - Retail <br />
$99.99</li>
<li> Intel Pentium E2160 Allendale 1.8GHz LGA 775 65W Dual-Core Processor Model BX80557E2160 - Retail <br />
$78.99</li>
<li>Subtotal: $408.89</li>
</ul>
<p>I didn't change the motherboard and case specified in article. I am guessing they are better than the Dell. Besides that, the Dell comes out cheaper. Note, that if I had upgraded Dell's components, say to 2GB RAM, a 320GB hard disk, a DVD burner, and a 256MB video card, the advantage goes out the window. I am guessing this is how Dell makes money-- lure the customer in with a low base price and charge extra for upgrades.</p>
<p>I went back and forth on the decision multiple times. One thing that almost made me build it myself was the much better chance of being able to overclock my processor with a standard motherboard. According to <a href="http://www.hardwarezone.com/articles/view.php?cid=2&id=2332&pg=2">this article</a> on hardwarezone.com, the E2160 can be overclocked to almost double and will still remain very stable. With standard motherboards, this is not too difficult to achieve. However, Dell's motherboards make it much more difficult. Hopefully for me, someone will figure it out and post an article about it.</p>
Unaswered Questions
2008-01-25T15:32:00-08:00https://www.saltycrane.com/blog/2008/01/unaswered-questions/<div class="Section1"> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy">If you know the answer, please leave a comment.</span></font></p> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy"> </span></font></p> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy">1. How do I unpack a tarball to a specific directory?</span></font></p> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy">I know I can uncompress a tar file in the current working directory with tar –xvf mytarfile.tar. Can I upack it to another directory instead?</span></font></p> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy"> </span></font></p> <p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size: 10.0pt;font-family:Arial;color:navy"> </span></font></p> </div>
How to scroll in GNU Screen
2008-01-24T17:30:00-08:00https://www.saltycrane.com/blog/2008/01/how-to-scroll-in-gnu-screen/<p><a href="http://www.gnu.org/software/screen/">GNU Screen</a> is great.
But, after half a year of using it, I've only now figured out how to
scroll into screen's buffer (courtesy of <a href="http://www.kuro5hin.org/story/2004/3/9/16838/14935">
this tutorial</a>). Normally, I just use SHIFT+PGUP to scroll
up in <a href="http://software.schmorp.de/pkg/rxvt-unicode.html">urxvt</a>'s buffer. However, if I have two regions in screen, this
doesn't work. To scroll in screen's buffer, I need to enter "copy mode".
To do this, type <strong><code>C-a [</code></strong> or
<strong><code>C-a ESC</code></strong>. (The latter option works great for me
because I bound the prefix key to the backtick key instead of <strong>
<code>C-a</code></strong>.) Once in "copy mode", I can scroll using the
arrow keys or PGUP/PGDOWN keys. To exit "copy mode", I press ESC. Scrolling
within screen's buffer instead of urxvt's also prevents me from losing my
place in my scrollback buffer when I switch screen "windows" (terminal
sessions).</p>
<h5 id="useful-commands">Other useful commands</h5>
<ul>
<li><strong><code>C-a S</code></strong> splits into two regions</li>
<li><strong><code>C-a tab</code></strong> switches input focus to the next region</li>
<li><strong><code>C-a X</code></strong> kills the current region</li>
<li><strong><code>C-a :resize <em style="font-weight: normal">numlines</em></code></strong> resizes
the current region to <em style="font-weight: normal">numlines</em> lines.</li>
</ul>
<h5 id="custom-key-bindings">Custom key bindings for scrolling <em style="font-weight: normal">(updated 2009-04-01)</em></h5>
<p>If you don't want to hit <code>C-a ESC PGUP</code> everytime to page up,
you can create keyboard shortcuts. I use CTRL+K to scroll up one line and META+K to
scroll up one page (I know these are kind of weird key combinations). To create these
shortcuts, I put the following in my <code>.screenrc</code>.</p>
<pre>bindkey "^[k" eval "copy" "stuff ^b" # enter copy mode and move up one page
bindkey "^k" eval "copy" "stuff k" # enter copy mode and move up one line
bindkey -m "^[k" stuff ^b # move up one page
bindkey -m "^k" stuff k # move up one line</pre>
<p>Now, when I'm in screen and press <code>M-k</code>, screen enters copy mode and
scrolls up one page. Likewise, <code>C-k</code> enters copy mode and scrolls up one line.
As before, to exit copy mode, hit <code>ESC</code>.</p>
<h5 id="copy-paste">How to copy and paste in screen <em style="font-weight: normal">(added 2010-02-03)</em></h5>
<p>Thanks to this <a href="http://arundelo.livejournal.com/390.html">
GNU Screen cheat-sheet</a>.</p>
<ul>
<li>Enter scrollback mode using <code>C-a [</code> or <code>C-a ESC</code></li>
<li>Press the space bar to begin selecting text.</li>
<li>Move the cursor using h,j,k,l,C-b,C-f or arrow keys/PGUP/PGDOWN</li>
<li>Hit the space bar again to copy the selected text.</li>
<li>Hit <code>ESC</code> to exit scrollback mode.</li>
<li>Press <code>C-a ]</code> to paste.</li>
</ul>
<h5 id="multi-display">How to open 2 terminals connected to the same session (Multi display mode) <em style="font-weight: normal">(updated 2012-04-05)</em></h5>
<p>The -x option can be used to attach multiple terminals to the same screen session. Pretty cool!
Open a terminal and create a screen session named "local"</p>
<pre class="console">$ screen -S local </pre>
<p>Start a second terminal, then attach it to the existing session:</p>
<pre class="console">$ screen -x local </pre>
<h5 id="eliminate-copy-mode-delay">How to eliminate apparent delay when entering copy mode <em style="font-weight: normal">(updated 2012-04-05)</em></h5>
<p>To get rid of the apparent delay when entering copy mode, I set msgminwait to 0 in my ~/.screenrc</p>
<pre>msgminwait 0</pre>
<h5 id="screenrc">My ~/.screenrc on github</h5>
<p>Here is <a href="https://github.com/saltycrane/homedir/blob/master/etc/.screenrc">my ~/.screenrc on github</a>.</p>
<h5 id="see-also">See also</h5>
<p>See also my post, <a href="/blog/2012/10/how-start-long-running-process-screen-and-detach-it/">
How to start a long-running process in screen and detach from it</a>
and some "screen vs. tmux" links in
<a href="http://www.saltycrane.com/blog/2012/10/how-start-long-running-process-screen-and-detach-it/#comments">
the comments</a>.
</p>
How to install pyqt4 on ubuntu linux
2008-01-15T12:41:00-08:00https://www.saltycrane.com/blog/2008/01/how-to-install-pyqt4-on-ubuntu-linux/<h5>Install and run "hello world" example</h5>
<ul>
<li><pre>apt-cache search pyqt</pre></li>
<li><pre>sudo apt-get install python-qt4</pre></li>
<li>Create a file <code>~/tmp/helloworld.py</code>
<pre class="python">import sys
from PyQt4.QtGui import *
app = QApplication(sys.argv)
button = QPushButton("Hello World", None)
button.show()
app.exec_()</pre>
</li>
<li>Run it:
<pre>python ~/tmp/helloworld.py</pre>
You should see a window with a single button pop up.
</li>
</ul>
<br>
<h5>Install additional examples</h5>
<ul>
<li>For more examples, install the <code>python-qt4-doc</code> package:
<pre>sudo apt-get install python-qt4-doc</pre>
</li>
<li>After installation, the examples are located at:
<pre>/usr/share/doc/python-qt4-doc/examples</pre>
</li>
</ul>
Ubuntu Gutsy Emacs Copy Paste
2007-11-07T23:40:00-08:00https://www.saltycrane.com/blog/2007/11/ubuntu-gutsy-emacs-copy-paste/
<p>I have Ubuntu 7.10 Gutsy Gibson (Gnome) and installed the <code>emacs22</code> package. Copy and paste did not work from Emacs to the Firefox search box and from the Firefox location box to Emacs. I did get it working after a little Google searching. But it made me think back to Tim Bray's comments in <a href="http://www.tbray.org/ongoing/When/200x/2006/08/16/Back-to-the-Mac">Back to the Mac</a> regarding the importance of a simple thing like copy and paste.</p>
<p>Here is the fix, which worked like a charm. I got this from the Ubuntu forums. Put the following two lines in your .emacs file.
<pre>(setq x-select-enable-clipboard t)
(setq interprogram-paste-function 'x-cut-buffer-or-selection-value)</pre>
</p>
First steps with Ubuntu Gutsy
2007-10-27T17:13:00-07:00https://www.saltycrane.com/blog/2007/10/first-steps-with-ubuntu-gutsy/
I recently installed Ubuntu 7.10 Gutsy Gibson on my Dell E1405 laptop dual booting with Windows Vista. Here are my first steps at configuration. Note, I tried Kubuntu (KDE) for a while, but ended up going with the Gnome version because it seemed stabler, a little faster, is more popular (i.e. more support articles aimed at regular Ubuntu), and also the Compbiz Fusion / Beryl stuff looked cool. I'll probably add some KDE apps as I discover them in the future.
<ol>
<li>Get internet working with Verizon wireless EVDO express card (Novatel V640)<br />
<a href="http://www.savvyadmin.com/2007/06/03/ubuntu-dell-5700-evdo/">http://www.savvyadmin.com/2007/06/03/ubuntu-dell-5700-evdo/</a><br />
This actually works with no problems. And it seems to be faster under Ubuntu than under Vista.
</li>
<br />
<li>Select package manager repositories
<ul>
<li>From the Gnome desktop's System menu, Administration -> Synaptic Package Manager</li>
<li>Settings -> Repositories</li>
<li>in the Ubuntu Software tab, check:<br />
main, universe, and restricted
</li>
<li>in the Updates tab, check:<br />
gutsy-security<br />
gutsy-updates
</li>
<li>After closing the dialog, be sure to click the "Reload" button.</li>
</ul>
</li>
<br />
<li>Run Update Manager<br />
System -> Administration -> Update Manager
<ul>
<li>press the Check button</li>
<li>press the Install Updates button</li>
</ul>
</li>
<br />
<li>Configure Firefox
<ul>
<li>set home page to del.icio.us</li>
<li>remove bookmarks toolbar</li>
<li>save downloads to /home/sofeng/incoming</li>
<li>install del.icio.us Bookmarks Firefox Add-on<br />
<a href="https://addons.mozilla.org/en-US/firefox/addon/3615">https://addons.mozilla.org/en-US/firefox/addon/3615</a>
</li>
<li>install IE Tab Firefox Add-on (oh, not available for linux)</li>
<li>install flash plugin (free version)<br />
<kbd>sudo apt-get install mozilla-plugin-gnash</kbd>
</li>
<li>youtube doesn't work</li>
<li>remove free flash plugin<br />
<kbd>sudo apt-get autoremove mozilla-plugin-gnash</kbd>
</li>
<li>install flash plugin (non-free version)<br />
<kbd>sudo apt-get install flashplugin-nonfree</kbd><br />
got an error message:<br />
<pre>sofeng@tortoise:~$ sudo apt-get install flashplugin-nonfree
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package flashplugin-nonfree is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package flashplugin-nonfree has no installation candidate</pre>
It appears as if I didn't add the multiverse repository. But I did. It turns out I didn't press the "Reload" button after adding the new repositories. hmmph. went back and did that and everything worked good.
</li>
</ul>
</li>
<li>Install and configure emacs<br />
<kbd>sofeng@tortoise:~$ sudo apt-get install emacs22</kbd><br />
copy in my .emacs file
</li>
<br />
<li>Install and configure my shell and terminal
<ul>
<li>copy in my .bashrc</li>
<li>copy in my .Xdefaults</li>
<li>copy in my .screenrc</li>
<li>install urxvt:<br />
<kbd>sofeng@tortoise:~$ sudo apt-get install rxvt-unicode</kbd>
</li>
<li>To my suprise, I actually like gnome-terminal better than urxvt. The gnome-terminal font was better than urxvt and I couldn't figure out how to use that one with urxvt. While I was evaluating gnome-terminal, I found out it actually has a <big><em>real</em> transparent background!</big> This is sweet. Google "ubuntu transparent background" to find out more. I will still use screen inside of gnome-terminal in lieu of gnome-terminal's tabs. There's no replacement for screen yet.</li>
</ul>
</li>
<br />
<li>Turn off PC speaker to get rid of annoying beeps<br />
<kbd>sofeng@tortoise:~/$ sudo sh -c 'echo "blacklist pcspkr" >> /etc/modprobe.d/blacklist'</kbd><br />
from <a href="http://ubuntu-tutorials.com/2007/07/26/turning-off-the-system-hardware-beep-linux-tutorial/">http://ubuntu-tutorials.com/2007/07/26/turning-off-the-system-hardware-beep-linux-tutorial/</a>
</li>
<br />
<li>Do not dim display when on battery power<br />
From the Gnome System menu, Preferences -> Power Management<br />
On Battery Power tab:<br />
Dim display brightness by: 0%
</li>
</ol>
Installing Ubuntu with Vista on Dell 1405
2007-10-12T00:19:00-07:00https://www.saltycrane.com/blog/2007/10/installing-ubuntu-with-vista-on-dell/I just installed <a href="http://www.ubuntu.com/">Ubuntu</a> linux on my Dell E1405 personal laptop. A friend had talked about setting up an Ubuntu machine so I thought it would be good for talking tech and also good experience. The setup so far has been relatively easy. The most difficulty has been that I had to delete two of Dell's partitions on the hard disk before I could install Ubuntu. I am dual booting Windows Vista, which was already installed, and Ubuntu 7.04 Feisty Fawn. I know Gutsy Gibson is only a few days away but I couldn't wait.<br /><br /> My Dell laptop came with 4 primary partitions already loaded:
<ol>
<li>the Dell Utility FAT16 partition: 47 MB</li>
<li>the Dell Recovery partition: 10 GB</li>
<li>the main Windows Vista NTFS partition: 62.48 GB</li>
<li>the MediaDirect FAT32 partion: 2 GB</li>
</ol>
This is the reason I only had about 25 GB free on my 80 GB SCSI hard disk. So it seems that the hard disk is only allowed to have 4 primary partitions. I deleted the Recovery partition. That was an easy decision because I didn't want it in the first place. Then I had to decide between the Utility or the MediaDirect partition. MediaDirect allows you to bypass Windows and directly view multimedia files such as pictures, music or DVDs. The Utility partion is used for diagnosing your computer in case something goes wrong. It appears to run a bunch of hardware tests. Both seemed nice to have but I decided to be practical and save the Utility partition. It will be nice to have something to test my hardware if things go wrong in the future. The rest of the installation was easy. I used the guide from <a href="http://apcmag.com/5046/how_to_dual_boot_vista_with_linux_vista_installed_first">APC </a>. It was the top of my Google search for "dual boot ubuntu vista" and it worked fine. The next step is getting my EVDO broadband card working so I can access the internet.
My software tools list
2007-08-10T13:32:00-07:00https://www.saltycrane.com/blog/2007/08/current-configuration/<style type="text/css">
td {
vertical-align: top;
}
</style>
<p>Inspired by Mark Pilgrim's
<a href="http://web.archive.org/web/20101214224709/http://diveintomark.org/archives/2006/06/26/essentials-2006">
2006 Essentials list</a>,
below is a list of my current software tools. If you notice a lot of
"I switched from ..." statements, keep in mind that I am a
<a href="http://undefined.com/ia/2006/10/10/the-fourteen-types-of-programmers-type-2-those-that-like-shiny-things/">
programmer who likes shiny things</a>.
</p>
<h4 id="other-lists">Other lists</h4>
<ul>
<li><a href="http://web.archive.org/web/20101214234142/http://diveintomark.org/archives/2008/10/28/essentials-2008">
Mark Pilgrim's Essentials, 2008 edition</a>
</li>
<li><a href="http://web.archive.org/web/20081226033542/http://adam.gomaa.us/blog/essentials-cop-out/">
Adam Gomaa's Essentials (2008)</a>
</li>
<li>Here is a <a href="http://mark.pilgrim.usesthis.com/">2010 update
for Mark Pilgrim</a>. After 2 and a half years, I still have several items
in common: <a href="#operating-system">Ubuntu</a> running <a href="#terminal">rxvt-unicode</a>
and <a href="#editor">Emacs 23</a>, the <a href="#keyboard">Unicomp keyboard</a>,
an <a href="#mobile-phone">Android phone</a>, <a href="#email">Gmail</a>,
Google Reader, Google Docs, and Pandora.
</li>
<li><a href="http://salvatore.sanfilippo.usesthis.com/">Salvatore Sanfilippo on usesthis.com (2011)</a></li>
<li><a href="http://aaron.boodman.usesthis.com/">Aaron Boodman on usesthis.com (2011)</a></li>
</ul>
<h4 id="contents">Contents</h4>
<table>
<tr>
<td>
<ul>
<li><a href="#operating-system">Operating System</a>: Ubuntu</li>
<li><a href="#window-manager">Window Manager</a>: Qtile</li>
<li><a href="#editor">Editor</a>: Emacs</li>
<li><a href="#terminal">Terminal</a>: urxvt + screen</li>
<li><a href="#vcs">Version Control System</a>: Git</li>
<li><a href="#high-high-level-language">"High-high-level" Language</a>: Python</li>
<li><a href="#web-framework">Web Framework</a>: Django/Flask</li>
</ul>
</td>
<td>
<ul>
<li><a href="#web-browser">Web Browser</a>: Firefox</li>
<li><a href="#email">Email</a>: Gmail</li>
<li><a href="#graphical-diff">Graphical Diff</a>: KDiff3</li>
<li><a href="#keyboard">Keyboard</a>: Leopold Tenkeyless (MX Browns)</li>
<li><a href="#office-chair">Office Chair</a>: Undecided</li>
<li><a href="#mobile-phone">Mobile Phone</a>: Motorola Droid 4</li>
</ul>
</td>
</tr>
</table>
<h4 id="my-list">My software tools list</h4>
<table style="font-size:100%">
<tr>
<td ><b>Category</b></td>
<td ><b>Currently using</b></td>
<td ><b>Comments</b></td>
</tr>
<tr>
<td id="operating-system" ><p>Operating System</p>
<a href="#operating-system" title="Section permalink" class="sectionlink">¶</a></td>
<td ><p><a href="http://www.ubuntu.com">Ubuntu</a> 14.04 Trusty Tahr</p>
</td>
<td >
<p>My first
Ubuntu install was in 2007 (dual-boot) and I got my first Windows-free machine
in 2008. I have used Cygwin
and coLinux when on Windows. Cygwin integrates better with Windows applications,
but coLinux is super fast and allows you to run a full Linux distro on top of
Windows. Windows does have some advantages, but overall I prefer Linux.</p>
<p><em>Update 2008-09</em>: Switching jobs means I can now use Linux at work.
I'm now using Linux about 90% of the time. My wife still uses Windows Vista on our laptop.</p>
<p>I don't have enough experience
with OSX to draw any authoritative conclusions, though I think
<a href="http://en.wikipedia.org/wiki/Mark_Pilgrim">Mark Pilgrim</a>
has <a href="http://diveintomark.org/archives/2006/06/02/when-the-bough-breaks">
biased me against Apple</a>. Also, I think Linux's
<a href="http://c2.com/cgi/wiki?FreeAsInBeer">free as in beer</a> (and somewhat
related free as in speech) characteristics
vs. Mac's expensive (and somewhat related proprietary) characteristics
resonate with the cheap engineer in me.</p>
<p>Additional Linux vs. Mac commentary:</p>
<ul>
<li>
<a href="http://en.wikipedia.org/wiki/Jamie_Zawinski">JWZ</a> 2000:
<a href="http://www.jwz.org/doc/linux.html">Unix/linux sucks less, but it still sucks.</a>
</li>
<li>
JWZ <a href="http://jwz.livejournal.com/494040.html">2005</a>,
<a href="http://jwz.livejournal.com/780264.html">2007</a>: Linux sucks more.
</li>
<li>
<a href="http://en.wikipedia.org/wiki/Paul_Graham">Paul Graham</a> 2005:
<a href="http://www.paulgraham.com/mac.html">Return of the Mac.</a>
</li>
<li>
Mark Pilgrim 2006: <a href="http://diveintomark.org/archives/2006/05/30/bye-apple">
Bye, Apple</a>
</li>
<li>
<a href="http://en.wikipedia.org/wiki/Cory_Doctorow">Cory Doctorow</a> 2006:
<a href="http://www.boingboing.net/2006/06/29/mark-pilgrims-list-o.html">
Me too</a>.
</li>
<li>
<a href="http://en.wikipedia.org/wiki/Tim_Bray">Tim Bray</a> 2006:
<a href="http://www.tbray.org/ongoing/When/200x/2006/06/15/Switch-From-Mac">Me too</a>,
<a href="http://www.tbray.org/ongoing/When/200x/2006/08/16/Back-to-the-Mac">Back to the Mac</a>
</li>
<li>
Steve Yegge 2008:
<a href="http://steve-yegge.blogspot.com/2008/04/settling-osx-focus-follows-mouse-debate.html">
Switching to OSX for the fonts</a>
</li>
<li>
Of course, <a href="http://en.wikipedia.org/wiki/Linus_Torvalds">Linus Torvalds</a>
<a href="http://www.smh.com.au/news/technology/q-and-a-with-linus-torvalds/2008/02/05/1202090403120.html?page=2">
"prefers" Linux</a> (2008)
</li>
<li><a href="http://en.wikipedia.org/wiki/Donald_Knuth">Donald Knuth</a>,
author of <em>The Art of Computer Programming</em>,
<a href="http://www.informit.com/articles/printerfriendly.aspx?p=1193856">
uses Ubuntu Linux for work, and Macs for play</a> (2008).
</li>
<li>Salvatore Sanfilippo (author of <a href="http://code.google.com/p/redis/">Redis</a>)
chooses to use
a Mac so he can <em>"focus on what <he's> doing without spending hours trying to
fix unrelated desktop stuff"</em> but says Linux is
<a href="http://antirez.com/post/linux-better-for-coding.html">
<em>"still better for coding"</em></a> (2009).
</li>
<li>Ted Dziuba: <a href="http://harmful.cat-v.org/software/operating-systems/osx/osx-unsuitable-web-development">
<em>MacOS X is an Unsuitable Platform for Web Development</em>
</a> (2011)</li>
<li>Bozhidar Batsov: <a href="http://batsov.com/Linux/Windows/Rant/2011/06/11/linux-desktop-experience-killing-linux-on-the-desktop.html">
<em>The Linux desktop experience is killing Linux on the desktop</em>
</a> (2011)</li>
</ul>
<p>History: 1987: <a href="http://en.wikipedia.org/wiki/Apple_IIGS">Apple GS/OS</a>,
1994: <a href="http://en.wikipedia.org/wiki/Windows_3.1x">Windows 3.1</a>,
2000: <a href="http://en.wikipedia.org/wiki/Windows_98">Windows 98</a>,
2001: <a href="http://en.wikipedia.org/wiki/Windows_XP">Windows XP</a>,
2007: <a href="http://en.wikipedia.org/wiki/Windows_Vista">Windows Vista</a>,
2007: <a href="http://en.wikipedia.org/wiki/Ubuntu_%28operating_system%29">Ubuntu</a>
</p>
</td>
</tr>
<tr>
<td id="window-manager"><p>Window Manager</p>
<a href="#window-manager" title="Section permalink" class="sectionlink">¶</a></td>
<td ><p><a href="http://www.qtile.org/">Qtile</a></p></td>
<td >
<p>The WM written in Python,
<a href="http://panela.blog-city.com/fun_of_tiling_window_managers.htm">
recommended by Matt Harrison</a>.
</p>
<p>Previously, wmii: dynamic, tiling, scriptable window manager that doesn't
require a mouse. It sucks less.</p>
<p>I switched from <a href="http://www.nongnu.org/ratpoison/">ratpoison</a>
at the same time I started using coLinux because running native Linux
allowed me to use any Linux window manager as well.</p>
<p>Recently, some have switched from wmii to
<a href="http://xmonad.org/">xmonad</a>, the new
<a href="http://www.haskell.org/">Haskell</a> tiling window manager.
It has some nice features over wmii, including dual head support, but after
a brief excursion, I slightly prefer wmii's way of doing things.</p>
<p>If you're a hard core Lisper,
<a href="http://www.nongnu.org/stumpwm/">stumpwm</a> is the window manager
for you. It has a <a href="http://en.wikipedia.org/wiki/REPL">REPL</a>. This one
seems a little too hard core for me, especially since I don't know Lisp.</p>
<p>History: 2007: ratpoison, 2007: wmii, 2012: Qtile</p>
</td>
</tr>
<tr>
<td id="editor"><p>Editor/IDE</p>
<a href="#editor" title="Section permalink" class="sectionlink">¶</a></td>
<td><p><a href="http://www.gnu.org/software/emacs">GNU Emacs</a> 24.3</p></td>
<td >
<p>Switched from Eclipse in 2007. It was a slow transition, but
I think Emacs is worth the investment.</p>
<p><b>On Emacs vs. Vim:</b> I really like that Vim can be used easily on remote
machines. Currently, I use TRAMP for remote file editing. It is very cool,
but it is slow, and I sometimes fall back on nano (*gasp*) for a quick
config file edit when I am logged into a remote terminal. I also suspect
that the dual modes of Vim are more efficient than Emacs-- I'm just not
smart enough to get used to it. Maybe someday I will switch to
<a href="http://www.gnu.org/software/emacs/manual/html_node/viper/index.html#Top">Viper
mode</a>. Why Emacs over Vim? I choose Emacs because it is more powerful
and closer to an IDE. Emacs Lisp allows you to do whatever you want.
</p>
<p><b>On Emacs vs. Eclipse:</b> I found Eclipse was too slow and heavy, used too
much screen real estate, and was too difficult to customize. Scripting
Eclipse required writing extensions in Java whereas Emacs can be extended
in elegant Lisp. I think Eclipse (and other IDE's) are better for code
exploration and they probably have features that I'm not aware of since
I don't use them. I figure, though, that with enough Emacs Lisp, Emacs
can do anything those IDEs can do. If you really want those features,
it's a question of if you want to spend the time achieving perfection
in Emacs or getting real work done in another IDE. I choose perfection. ;)
</p>
<p>Additional commentary:</p>
<ul>
<li><a href="http://blog.ianbicking.org/the-vs-emacs.html">
THE vs. Emacs</a>, Ian Bicking (2005)</li>
<li><a href="http://steve.yegge.googlepages.com/effective-emacs">
Effective Emacs</a>, Steve Yegge (2005)</li>
<li><a href="http://blog.ianbicking.org/other-editors.html">
Other Editors?</a>, Ian Bicking (2005)</li>
<li><a href="http://glyph.twistedmatrix.com/2008/04/structured-python-editor.html">
Structured Python Editor</a>, Glyph Lefkowitz (2008)</li>
<li><a href="http://chalain.livejournal.com/74234.html">
Editor Wars: Revenge of the... oh, whatever.</a>, Chalain (2008)</li>
<li><a href="http://metajack.im/2008/09/05/what-you-can-learn-from-emacs/">
What You Can Learn From Emacs</a>, Jack Moffitt (2008)</li>
<li><a href="http://glyph.twistedmatrix.com/2008/12/emacs-test.html">
The Emacs Test</a>, Glyph Lefkowitz (2008)</li>
<li><a href="http://www.redreddesign.com/blog/emacs-extensions-i-cant-live-without/">
emacs extensions i can’t live without</a> (2009)</li>
</ul>
<p>Screencasts:</p>
<ul>
<li><a href="http://platypope.org/yada/emacs-demo/">I need a cool European accent</a></li>
<li><a href="http://www.vimeo.com/1013263">What You Can Learn From ido.el</a></li>
<li><a href="http://www.youtube.com/watch?v=76Ygeg9miao#t=01m47s">YASnippet demo</a></li>
</ul>
<p>Install method:
<del><a href="http://www.saltycrane.com/blog/2008/10/installing-emacs-23-cvs-ubuntu-hardy/">
from source</a></del>
<code>sudo apt-get install emacs</code>
</p>
<p>Useful packages:</p>
<ul>
<li><a href="http://www.gnu.org/software/tramp/">
Tramp</a>: transparent remote file access</li>
<li><a href="http://www.emacswiki.org/emacs/InteractivelyDoThings">
ido</a>: Buffer switching and more</li>
<li><a href="http://orgmode.org/">Org-Mode</a>: note taking, task lists</li>
<li><a href="http://github.com/yoshiki/yaml-mode">yaml-mode</a>: for yaml</li>
<li><a href="http://jblevins.org/projects/markdown-mode/">markdown-mode</a>: for markdown</li>
<li><a href="http://code.google.com/p/yasnippet/">YASnippet</a>:
Textmate-inspired templating. Great for making HTML less tedious.</li>
<li><a href="https://github.com/nonsequitur/smex">smex: IDO for M-x</a></li>
<li><a href="https://github.com/magnars/multiple-cursors.el">multiple-cursors</a>: this is really fun *and* useful.</li>
<li><a href="http://web-mode.org/">web-mode</a>: for HTML/CSS/Javascript</li>
</ul>
<p>My emacs config on github: <a href="http://github.com/saltycrane/emacs">http://github.com/saltycrane/emacs</a></p>
<p>History: 2001: Emacs/NEdit/UltraEdit, 2006: Eclipse, 2007: Emacs</p>
</td>
</tr>
<tr>
<td id="terminal"><p>Terminal</p>
<a href="#terminal" title="Section permalink" class="sectionlink">¶</a></td>
<td><p><a href="http://software.schmorp.de/pkg/rxvt-unicode.html">urxvt</a></p>
9.19
+ <br><a href="http://www.gnu.org/software/screen/">screen</a></td>
<td>
<p>urxvt supports xft (anti-aliased) fonts, real transparency (not that
I actually use transparency with Qtile), and fading (which I do use with Qtile)
and it is much faster and lighter than gnome-terminal or konsole. screen allows me to switch
terminal sessions without ugly tabs, attach to remote sessions, search through
the scrollback buffer, and more. <em>Update:</em> urxvt also has embedded perl.</p>
<p>Install method:
<del><a href="/blog/2009/11/how-make-urxvt-look-gnome-terminal/">from source</a></del>
<code>sudo apt-get install rxvt-unicode</code>
</p>
</td>
</tr>
<tr>
<td id="vcs"><p>Version Control System</p>
<a href="#vcs" title="Section permalink" class="sectionlink">¶</a></td>
<td>
<p><a href="http://git-scm.com/">Git</a></p>
</td>
<td>
<p>Switched from Subversion to Mercurial in June 2007. The merging in Mercurial
is very nice and can be done without thinking. I do miss Subversion/Subclipse's
revision history viewer, file compare, and ability to isolate files apart from
changesets. <em>Update 2010-04-05:</em> See <a href="#c8633">my comment
below</a>.
</p>
<p><em>Update 2011-05:</em> Switched my personal repos to Git. Use Git almost
exclusively now.
</p>
<p>Additional commentary:</p>
<ul>
<li><a href="http://jonchu.posterous.com/16445171">
DVCS: Why I chose Mercurial over Git</a> (2010)</li>
<li><a href="http://blog.extracheese.org/2010/05/why-i-switched-to-git-from-mercurial.html">
Why I Switched to Git From Mercurial</a> (2010)</li>
</ul>
<p>Install method: <code>sudo apt-get install mercurial</code></p>
<p>History: 2001: SCCS, 2003: Other, 2007: Subversion, 2007: Mercurial,
2010: Mercurial/Git, 2011: Git</p>
</td>
</tr>
<tr>
<td id="high-level-language"><p>"High-level" Language</p>
<a href="#high-level-language" title="Section permalink" class="sectionlink">¶</a></td>
<td><p>None</p></td>
<td>
<p>Previously, I used C. Now I use don't use any statically-typed languague. Wouldn't mind learning <del>C++</del> Go. Although,
Linus <a href="http://thread.gmane.org/gmane.comp.version-control.git/57643/focus=57918">
doesn't like it.</a></p>
</td>
</tr>
<tr>
<td id="high-high-level-language" ><p>"High-high-level" Language</p>
<a href="#high-high-level-language" title="Section permalink" class="sectionlink">¶</a></td>
<td ><p><a href="http://www.python.org/">Python</a> 2.7</p></td>
<td >
<p>My love for Python is strong. I switched from Perl in 2005
and have no regrets. Object-oriented, easy to read (no more
<a href="http://en.wikipedia.org/wiki/There_is_more_than_one_way_to_do_it">
TIMTOWTDI</a>), and smart people use it. I also want to learn Javascript 2
becuase it is the
<a href="http://steve-yegge.blogspot.com/2007/02/next-big-language.html">
"Next Big Language"</a> and Lisp because it is the
<a href="http://www.paulgraham.com/avg.html">"most powerful language"</a>.</p>
<p><b>On Python vs. Ruby:</b> from what I've read, I characterize Ruby as the
more expressive language more similar to Perl (than Python is) and Python
as the more regimented language. Since I like regimented, I like Python.</p>
<p><b>On Python vs. Lisp:</b> I've concluded that I lack the intelligence to
harness enough of Lisp's power to counteract its non-practicality (e.g.
lack of libraries).</p>
<p>Additional commentary:</p>
<ul>
<li><a href="http://en.wikipedia.org/wiki/Paul_Graham">Paul Graham</a>:
<a href="http://www.paulgraham.com/icad.html">Python is
getting closer to Lisp</a> (2002)</li>
<li>Paul Prescod: <a href="http://www.prescod.net/python/IsPythonLisp.html">
no it isn't</a></li>
<li>Paraphrase of <a href="http://www.swiss.ai.mit.edu/~gjs/">
Gerald Jay Sussman</a> (one of the creators of Scheme):
<a href="http://blog.snowtide.com/2009/03/24/why-mit-now-uses-python-instead-of-scheme-for-its-undergraduate-cs-program">
Why MIT now uses python instead of scheme for its undergraduate CS program</a>
(2009)</li>
</ul>
<p>Install method: preinstalled on Ubuntu</p>
<p>History: 2001: <a href="http://www.perl.org/">Perl</a>,
2005: <a href="http://www.python.org/">Python</a>
</p>
</td>
</tr>
<tr>
<td id="web-framework" ><p>Web Framework</p>
<a href="#web-framework" title="Section permalink" class="sectionlink">¶</a></td>
<td >
<p>
<a href="http://www.djangoproject.com/">Django</a> /
<a href="http://flask.pocoo.org/">Flask</a>
</p>
</td>
<td >
<p><em>2014-10:</em>I now use Flask at work. Previously, I used Django. I think I like SQLAlchemy and Jinja2 better than Django. Django has a lot more built in and is put together better. Flask's thread locals are convenient and messy. Probably <a href="http://www.pylonsproject.org/">Pyramid</a> is the best but no one uses it.</p>
<p>Here are some links:</p>
<ul>
<li><a href="http://www.ianbicking.org/">Ian Bicking</a>:
<a href="http://blog.ianbicking.org/theres-so-much-more-than-rails.html">
There's so much more than Rails</a> (2005)</li>
<li><a href="http://blog.ianbicking.org/2008/01/12/what-php-deployment-gets-right/">
What PHP deployment gets right</a> (2008)</li>
<li><a href="http://www.feedmagnet.com/blog/django-vs-rails/">
Django vs. Rails</a> (2009)</li>
</ul>
<p>Install method: <code>pip install Django</code> / <code>pip install Flask</code></p>
</td>
</tr>
<tr>
<td id="web-browser"><p>Web Browser</p>
<a href="#web-browser" title="Section permalink" class="sectionlink">¶</a></td>
<td><p>Firefox</p></td>
<td >
<p>Not to be confused with <a href="http://www.konqueror.org">
Konqueror</a>, Conkeror is an emacs-like, keyboard driven, scriptable,
Mozilla-based web browser. I've used it almost full
time since January 2008. It is still considered alpha stage software
so there are a number of bugs. However, it is still pretty sweet. I use
Firefox as a backup (and IE Tab for Launchcast and Netflix on Windows).</p>
<p>Unfortunately, one of the annoying things in Firefox 2 is present in
Conkeror as well-- memory leaks. Based on
<a href="http://kb.mozillazine.org/Memory_Leak">this Mozilla article</a>
and some brief personal experience, Firefox 3 has made fixes in this area.
It would be nice if Conkeror could benefit from the Firefox 3 fixes.</p>
<p><em>Update 2010-05-04:</em> Conkeror is really awesome, but I had some memory/cpu issues with it
a while ago and haven't put in the work to merge my custom keybindings
with the latest Conkeror code. Also waiting to see if there will be
an Emacs version of <a href="http://vimium.github.com/">Vimium</a>
(since, after all <a href="http://vimperator.org/">vimperator</a>
followed conkeror).
</p>
<p>I'm currently using a combination of Google Chrome and Firefox.
Chrome is faster, especially for Javascript-intensive sites, but I really
<a href="http://superuser.com/questions/91334/alternative-to-arrow-keys-in-google-chrome-location-bar">
<em>hate</em> the Google Chrome Omnibar</a>. If you have a solution
or workaround, please post an answer. Firefox also has some Add-ons
that Chrome doesn't.
</td>
</tr>
<tr>
<td id="email" ><p>Email</p>
<a href="#email" title="Section permalink" class="sectionlink">¶</a></td>
<td ><p>Gmail</p></td>
<td >
<p><em>Update 2010-05-04:</em>
Gmail. It's slow but it has a lot of features and it's in
my web browser and it's easy and it works and I don't really like using
email that much anyways. Oh, and, big plus, syncing works almost perfect
with my Android phone. (Actual full switch to Gmail was around December 2008.)
</p>
</td>
</tr>
<tr>
<td id="graphical-diff" ><p>Graphical diff/merge</p>
<a href="#graphical-diff" title="Section permalink" class="sectionlink">¶</a></td>
<td ><p><a href="http://kdiff3.sourceforge.net/">KDiff3</a></p></td>
<td >
<p>I started using KDiff a while ago on Windows and have
always liked it. I'm thinking, though, since I'm an Emacs person, I ought to
use <a href="http://www.gnu.org/software/emacs/manual/html_mono/ediff.html">
Ediff</a>.</p>
<p>Install method: <code>sudo apt-get install kdiff3</code></p>
</td>
</tr>
<tr>
<td id="keyboard"><p>Keyboard</p>
<a href="#keyboard" title="Section permalink" class="sectionlink">¶</a></td>
<td>
<p><a href="http://pckeyboards.stores.yahoo.net//en104wh.html">
Unicomp Endurapro</a> /
<a href="http://elitekeyboards.com/products.php?sub=leopold,tenkeyless&pid=fc200rtab">
Leopold Tenkeyless w/ Cherry MX Browns
</a></p>
</td>
<td>
<p>I realize a keyboard is hardware and not software, but it is
hardware that can evoke strong opinions, so I made an exception.
</p>
<p>One problem with hardware is I can't try out all the different types
and choose the one I like the best as I can with software. So
my selection is based on reviews not on actual experience. And,
after buying the Endurapro, I found I actually did not like the
integrated mouse stick, so I should have bought a Customizer 104/105
and saved $30.</p>
<p><em>Update 2011-05:</em> Got the Leopold tenkeyless for work.
Tenkeyless is a must for quicker mouse access.
And I actually like the lighter touch of the
Cherry MX Browns versus the Unicomp's buckling springs. I like it
so much I want to replace my Unicomp at home with another Leopold.
</p>
<p><em>Update:</em> Got a second Leopold for home to replace my Unicomp.</p>
<p>Additional commentary:</p>
<ul>
<li><a href="http://www.jwz.org/gruntle/wrists.html">
my wrists and welcome to them.</a>, JWZ (1999)</li>
<li><a href="http://www.dansdata.com/deck.htm">
Deck keyboard</a>, Dan (2005)</li>
<li><a href="http://bc.tech.coop/blog/060131.html">
Surviving Emacs - Part 4</a>, Bill Clementson (2006)</li>
<li><a href="http://glyf.livejournal.com/55780.html">
Keyboard Fight</a>, Glyph Lefkowitz (2006)</li>
<li><a href="http://diveintomark.org/archives/2006/05/30/bye-apple">
Bye, Apple</a>, Mark Pilgrim (2006)</li>
<li><a href="http://glyph.twistedmatrix.com/2009/01/meandering-review-of-logitech.html">
A Meandering Review of the Logitech Illuminated Keyboard</a>,
Glyph Lefkowitz (2009)</li>
</ul>
<p>History: 2009: Unicomp Endurapro, 2011: Leopold Tenkeyless (MX Browns)</p>
</td>
</tr>
<tr>
<td id="office-chair"><p>Office Chair</p>
<a href="#office-chair" title="Section permalink" class="sectionlink">¶</a></td>
<td><p>Undecided</p></td>
<td><p>Herman Miller Aeron or Steelcase Leap</p>
</td>
</tr>
<tr>
<td id="mobile-phone"><p>Mobile Phone</p>
<a href="#mobile-phone" title="Section permalink" class="sectionlink">¶</a></td>
<td><p><a href="http://www.motorola.com/Consumers/US-EN/Consumer-Product-and-Services/Mobile-Phones/Motorola-DROID-US-EN">Motorola Droid 4</a></p></td>
<td>
<p>After almost 10 years (and 3 generations) of the Palm Treo, I got the
Motorola Droid.</p>
<p>The Droid is pretty awesome, but I don't like switching
to landscape mode to use the keyboard-- especially when toolbars and headers
take up half the screen. I think Palm and Blackberry got this form factor Right.
Just as I prefer the keyboard over the mouse on my PC, I want to use the keyboard
and D-pad over the touchscreen on my phone.
<em>Update 2010-05-04:</em> The keyboard is mediocre and use of the D-pad
center button sucks. My Palm Treo 650 D-pad worked much better.
</p>
<p>My reasons for choosing an
<a href="http://en.wikipedia.org/wiki/Android_%28operating_system%29">Android</a>
phone are: I <em>need</em> a physical keyboard.
I <em>want</em> to hack on my phone. I <em>think</em> Android has more potential than Palm's
<a href="http://en.wikipedia.org/wiki/WebOS">webOS</a>
or <a href="http://en.wikipedia.org/wiki/Maemo">Maemo</a>.</p>
<p>Useful apps:</p>
<ul>
<li><a href="http://code.google.com/p/connectbot/">Connectbot</a>
Secure shell (SSH) client for the Android platform. Use this all the time. Awesome.</li>
<li><a href="http://www.android.com/market/#app=listen">Listen</a>
Listen from Google Labs brings podcast search, subscribe, download and stream to your Android-powered device.</li>
<li><a href="http://mytracks.appspot.com/">My Tracks</a>
Record GPS tracks. Monitor your performance. Share your outdoor activities with friends.
Really enjoy this one. Love tracking my romping.
</li>
</ul>
<p><em>2010-05-04 Updates:</em></p>
<ul>
<li><a href="http://www.google.com/mobile/voice/">Google Voice</a>: Free text messages
and better voicemail. Another useful app. Makes voicemail suck a lot less.</li>
<li><a href="http://levelupstudio.com/foxyring">FoxyRing</a>: automatically
control the ringer volume based on ambient noise level. I was doubtful about this one,
but it actually works well for me.</li>
<li><a href="http://www.flixster.com/mobile/apps/android">Flixter (Movies)</a>:
Useful for checking movie showtimes.
</li>
<li><a href="http://www.youversion.com/mobile/android">Bible (YouVersion)</a>:
This version is pretty good. I miss my <a href="http://www.plkr.org/">Plucker</a>
created Bible on my Palm though. I wish there was something like Plucker for
Android.
</li>
<li>DroidLight: use the LED camera flash as a flashlight. Simple and useful.
Works much better than using the screen as a flashlight.
</li>
<li><a href="http://www.google.com/mobile/navigation/">Google Navigation</a>: I
don't need my Garmin anymore. (Each has advantages and disadvantages, but I
have found it to be a capable replacement.)</li>
<li>Just want to mention that the Calendar sucks. It is great that I can sync
with my Google calendar, but I really hate the Android Calendar UI.</li>
</ul>
<p>Additional Commentary/Links:</p>
<ul>
<li><a href="http://randomfoo.net/2009/06/15/thoughts-on-the-palm-pre-g2-and-iphone-3g">
Thoughts on the Palm Pre, G2, and iPhone 3G</a> Leonard Lin (2009 June)</li>
<li><a href="http://jwz.livejournal.com/1055120.html">
JWZ gets a Palm Pre</a> (2009 June)</li>
<li><a href="http://jwz.livejournal.com/1055369.html">
Emacs on Android</a> (2009 June)</li>
<li><a href="http://daringfireball.net/2009/08/the_android_opportunity">
The Android Opportunity</a>, John Gruber (2009 August)</li>
<li><a href="http://jwz.livejournal.com/1108212.html">
Dear Palm, it's just not working out</a>, JWZ (2009 October)</li>
</ul>
<p>History: 2002: <a href="http://en.wikipedia.org/wiki/Treo_180">Handspring Treo 180</a>,
2003: <a href="http://en.wikipedia.org/wiki/Treo_600">Handspring Treo 600</a>,
2005: <a href="http://en.wikipedia.org/wiki/Treo_650">Palm Treo 650</a>,
2009: <a href="http://en.wikipedia.org/wiki/Motorola_Droid">Motorola Droid</a>
2011: <a href="http://en.wikipedia.org/wiki/Droid_4">Motorola Droid 4</a>
</p>
</td>
</tr>
</table>