SaltyCrane Blog — Notes on JavaScript and web development

Install coLinux (and Ubuntu Hardy) on Win XP using Slirp to internet and TAP to host behind a corporate firewall/proxy server

coLinux allows you to run a full Linux distribution, such as Ubuntu Hardy, on top of Windows at near native speeds. It runs as a single Windows process with its own specially allocated address space. The guest linux system can run from either a separate disk partition or from a single file on your Windows filesystem.

My initial impression of coLinux was so positive, I had planned to name this post, Goodbye Cygwin, hello coLinux. After 2 months of using coLinux, I still am using Cygwin a little bit. Despite being slow, (almost unbearably slow with my company's mandatory Whole Disk Encryption), Cygwin is good at integrating the GNU tools with Windows. Currently, I am still using Cygwin to script my Windows-only project version control system and use the Cygwin version of Mercurial for my personal version control. I'm not sure if using the Linux version of Mercurial would work on Samba shares. The more I learn about Linux and the less I depend on Windows applications, the less I will need Cygwin. (Update: Mercurial 1.0 works great on Samba shares. The important thing is to setup the owner of the share to be your user id and not root. See samba config below.)

Besides that, coLinux is great. The best part for me was being able to run the dynamic, tiling, scriptable, keyboard-driven window manager, wmii. I access Windows files (and network drives) using Samba. Shell commands, emacs, ssh are all there and are so fast. GUI applications such as Firefox, OpenOffice and Gimp are available as well. However, these are slow with the slirp networking setup. With a TAP connection between coLinux and the host OS, these apps run well, however, I couldn't get the TAP connection to work in conjuction with internet access so I settled for slirp only. If I figure this out, I will update the post. Here are my install notes.

Update 4/28/2008: I got my private TAP connection to the host working along with a slirp connection for internet. This makes my X server connections fast enough to run all my GUI apps like OpenOffice,evincekpdf for pdf files, and my web browser. See this section for details.


Contents

Install coLinux
  1. Download the coLinux installer and the Ubuntu root file system image
  2. Run the coLinux Windows installer
    • Accept defaults except for:
    • Destination Folder: change to c:\coLinux
    • You do not need to install WinPCap library.
    • You will get a warning dialog saying that "TAP-Win32 Adapter V8 (coLinux)" has not passed Windows Logo testing. Click "Continue Anyway"

Configure CoLinux
I already had Cygwin running so I did this stuff in a bash shell. You can use Windows commands as well to do the same thing.
  1. Copy Ubuntu-6.06.1.ext3.1bg.bz2 to c:\coLinux. (Keep a copy of this file as a backup to revert your Ubuntu filesystem if you mess things up.)
  2. Decompress the Ubuntu file system image. (Note, this could take a minute or more depending on your system.)
    $ cd /cygdrive/c/coLinux
    $ bunzip2 Ubuntu-6.06.1.ext3.1gb.bz2
  3. Create a swap file
    $ dd if=/dev/zero of=swap_device bs=1M count=512
    $ dd if=/dev/zero of=swap_device.2gb bs=1M count=2K
    (In Windows, you can use fsutil file createnew swap_device 536870912)
  4. Create a configuration file
    $ cp example.conf colinux.conf
    Edit colinux.conf and set the following:
    cobd0="c:\coLinux\Ubuntu-6.06.1.ext3.1gb"
    cobd1="c:\coLinux\swap_device"
    mem=512
    eth0=slirp
    eth1=tuntap # only needed for TAP connection
    The following items should already be set by default:
    kernel=vmlinux
    root=/dev/cobd0
    ro
    initrd=initrd.gz

Run CoLinux
  1. Open a Windows cmd.exe shell
  2. cd c:\colinux
  3. colinux-daemon.exe -t nt @colinux.conf
    (the -t nt is optional but makes it easier to copy/paste)
  4. login as "root" with password "root"
  5. Try commands such as ls, pwd, etc.
  6. To exit, use the halt command.

Configure networking with slirp only
  1. If you use a http proxy server to get to the internet, do the following inside the running CoLinux session:
    root@ubuntu:~# echo "export http_proxy='http://your-proxy-server.com:yourportnumber'" >> .bash_profile
    root@ubuntu:~# source .bash_profile
  2. Otherwise, there is nothing to do. The Ubuntu image already has the correct /etc/network/interfaces and /etc/resolv.conf files setup.

    /etc/network/interfaces should look like this:
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).
    
    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # The primary network interface (slirp)
    auto eth0
    iface eth0 inet static
       address 10.0.2.15
       broadcast 10.0.2.255
       netmask 255.255.255.0
       gateway 10.0.2.2
    

  3. /etc/resolv.conf should look like this:
    nameserver 10.0.2.3
  4. To test internet access, use something like wget since ping doesn't work with slirp.
    root@ubuntu:~# wget www.google.com

Configure networking with slirp to internet and TAP to host
In this configuration, I set up TAP to form a private connection to the host and slirp to access the internet. If you want to use X applications, this will make the response much better than with slirp only. I had to play around with the IP addresses to find one that didn't conflict with the static IP addresses on our network. I ended up choosing "10.0.3.16" for the TAP colinux address and "10.0.3.17" for my Windows side.
  1. TAP should be installed when you install coLinux
  2. In Windows, go to "Start", "Control Panel", "Network Connections"
  3. Find the "TAP-Win32 Adapter V8 (coLinux)" adapter. For me, it was named "Local Area Connection 2". Right click on it and select "Properties".
  4. On the "General" tab, scroll down and select "Internet Protocol (TCP/IP)". Click on "Properties".
  5. Select "Use the following IP address:". Enter in "10.0.3.17" for the "IP address:" and "255.255.254.0" for the "Subnet mask:". Leave the rest blank. Click "OK" twice to finish.
  6. in coLinux, edit /etc/network/interfaces to look like the following:
    auto lo
    # Loopback interface
    iface lo inet loopback
    
    # Network interface for internet access (slirp)
    auto eth0
    iface eth0 inet static
       address 10.0.2.15
       broadcast 10.0.2.255
       netmask 255.255.255.0
       gateway 10.0.2.2
    
    # Network interface for private connection to Host OS (tap-win32)
    auto eth1
    iface eth1 inet static
       address 10.0.3.16
       network 10.0.3.0
       netmask 255.255.254.0
       broadcast 10.0.3.255
  7. Make sure your "colinux.conf" file has the following lines:
    eth0=slirp
    eth1=tuntap
  8. You will need to set your DISPLAY variable to "10.0.3.17:0"
    root@ubuntu:~# export DISPLAY=10.0.3.17:0

Update your Ubuntu Dapper installation
  1. root@ubuntu:~# apt-get update
    You can ignore the following warning:
    W: GPG error: http://security.ubuntu.com dapper-security Release: Could not exec
    ute /usr/bin/gpgv to verify signature (is gnupg installed?)
    W: GPG error: http://fr.archive.ubuntu.com dapper-updates Release: Could not exe
    cute /usr/bin/gpgv to verify signature (is gnupg installed?)
    W: GPG error: http://fr.archive.ubuntu.com dapper-backports Release: Could not e
    xecute /usr/bin/gpgv to verify signature (is gnupg installed?)
    W: GPG error: http://fr.archive.ubuntu.com dapper Release: Could not execute /us
    r/bin/gpgv to verify signature (is gnupg installed?)
    W: You may want to run apt-get update to correct these problems
  2. root@ubuntu:~# apt-get upgrade
    Answer yes to prompts.
  3. That was easy.

Run an Xterm and other apps

Warning, getting the X Window System working properly was one of the most frustrating parts of the setup for me. The behavior seemed to be even non-deterministic. This is what worked for me. You may need to consult other sources to get it working for you. I am starting to readX Power Tools to learn more about the X Window System and will update this section if I learn anything new. I used Cygwin's X server because I already had Cygwin installed. Another option is Xming.

Update 5/6/2008: After switching to the Slirp+TAP networking configuration, I've been getting errors when trying to start my X clients:

$ xeyes
No protocol specified
Error: Can't open display: 10.0.3.17:0
$ xterm
No protocol specified
xterm Xt error: Can't open display: 10.0.3.17:0
$ urxvt
No protocol specified
urxvt: can't open display 10.0.3.17:0, aborting.
To resolve this, I have been starting up a Cygwin bash shell, and running the xhost + command to disable access control. This worked, but it was annoying to do this all the time. I discovered that there is a -ac command line option to Xwin which will do the same thing. I've updated my batch file below to reflect this. Here is a list of all the Xwin command line options:
use: X [:] [option]
-a #                   mouse acceleration (pixels)
-ac                    disable access control restrictions
-audit int             set audit trail level
-auth file             select authorization file
bc                     enable bug compatibility
-br                    create root window with black background
+bs                    enable any backing store support
-bs                    disable any backing store support
-c                     turns off key-click
c #                    key-click volume (0-100)
-cc int                default color visual class
-co file               color database file
-core                  generate core dump on fatal error
-dpi int               screen resolution in dots per inch
-deferglyphs [none|all|16] defer loading of [no|all|16-bit] glyphs
-f #                   bell base (0-100)
-fc string             cursor font
-fn string             default font name
-fp string             default font path
-help                  prints message with these options
-I                     ignore all remaining arguments
-ld int                limit data space to N Kb
-lf int                limit number of open files to N
-ls int                limit stack space to N Kb
-logo                  enable logo in screen saver
nologo                 disable logo in screen saver
-nolisten string       don't listen on protocol
-noreset               don't reset after last client exists
-reset                 reset after last client exists
-p #                   screen-saver pattern duration (minutes)
-pn                    accept failure to listen on all ports
-nopn                  reject failure to listen on all ports
-r                     turns off auto-repeat
r                      turns on auto-repeat 
-render [default|mono|gray|color] set render color alloc policy
-s #                   screen-saver timeout (minutes)
-sp file               security policy file
-su                    disable any save under support
-t #                   mouse threshold (pixels)
-terminate             terminate at server reset
-to #                  connection time out
-tst                   disable testing extensions
ttyxx                  server started from init on /dev/ttyxx
v                      video blanking for screen-saver
-v                     screen-saver without video blanking
-wm                    WhenMapped default backing-store
-x string              loads named extension at init time 
-maxbigreqsize         set maximal bigrequest size 
+extension name        Enable extension
-extension name        Disable extension
-query host-name       contact named host for XDMCP
-broadcast             broadcast for XDMCP
-indirect host-name    contact named host for indirect XDMCP
-port port-num         UDP port number to send messages to
-from local-address    specify the local address to connect from
-once                  Terminate server after one session
-class display-class   specify display class to send in manage
-cookie xdm-auth-bits  specify the magic cookie for XDMCP
-displayID display-id  manufacturer display ID for request
The X Keyboard Extension adds the following arguments:
-kb                    disable the X Keyboard Extension
+kb                    enable the X Keyboard Extension
[+-]accessx [ timeout [ timeout_mask [ feedback [ options_mask] ] ] ]
                       enable/disable accessx key sequences
-ar1                   set XKB autorepeat delay
-ar2                   set XKB autorepeat interval
-noloadxkb             don't load XKB keymap description
-xkbdb                 file that contains default XKB keymaps
-xkbmap                XKB keyboard description to load on startup
-depth bits_per_pixel
 Specify an optional bitdepth to use in fullscreen mode
 with a DirectDraw engine.
-emulate3buttons [timeout]
 Emulate 3 button mouse with an optional timeout in
 milliseconds.
-engine engine_type_id
 Override the server's automatically selected engine type:
  1 - Shadow GDI
  2 - Shadow DirectDraw
  4 - Shadow DirectDraw4 Non-Locking
-fullscreen
 Run the server in fullscreen mode.
-refresh rate_in_Hz
 Specify an optional refresh rate to use in fullscreen mode
 with a DirectDraw engine.
-screen scr_num [width height [x y] | [[WxH[+X+Y]][@m]] ]
 Enable screen scr_num and optionally specify a width and
 height and initial position for that screen. Additionally
 a monitor number can be specified to start the server on,
 at which point, all coordinates become relative to that
 monitor (Not for Windows NT4 and 95). Examples:
  -screen 0 800x600+100+100@2 ; 2nd monitor offset 100,100 size 800x600
  -screen 0 1024x768@3        ; 3rd monitor size 1024x768
  -screen 0 @1 ; on 1st monitor using its full resolution (the default)
-lesspointer
 Hide the windows mouse pointer when it is over an inactive
 Cygwin/X window.  This prevents ghost cursors appearing where
 the Windows cursor is drawn overtop of the X cursor
-nodecoration
 Do not draw a window border, title bar, etc.  Windowed
 mode only.
-mwextwm
 Run the server in multi-window external window manager mode.
-internalwm
 Run the internal window manager.
-rootless
 Run the server in rootless mode.
-multiwindow
 Run the server in multi-window mode.
-multiplemonitors
 EXPERIMENTAL: Use the entire virtual screen if multiple
 monitors are present.
-clipboard
 Run the clipboard integration module.
 Do not use at the same time as 'xwinclip'.
-nounicodeclipboard
 Do not use Unicode clipboard even if NT-based platform.
-scrollbars
 In windowed mode, allow screens bigger than the Windows desktop.
 Moreover, if the window has decorations, one can now resize
 it.
-[no]trayicon
 Do not create a tray icon.  Default is to create one
 icon per screen.  You can globally disable tray icons with
 -notrayicon, then enable it for specific screens with
 -trayicon for those screens.
-clipupdates num_boxes
 Use a clipping region to constrain shadow update blits to
 the updated region when num_boxes, or more, are in the
 updated region.  Currently supported only by `-engine 1'.
-[no]unixkill
 Ctrl+Alt+Backspace exits the X Server.
-[no]winkill
 Alt+F4 exits the X Server.
-xkbrules XKBRules
 Equivalent to XKBRules in XF86Config files.
-xkbmodel XKBModel
 Equivalent to XKBModel in XF86Config files.
-xkblayout XKBLayout
 Equivalent to XKBLayout in XF86Config files.
 For example: -xkblayout de
-xkbvariant XKBVariant
 Equivalent to XKBVariant in XF86Config files.
 For example: -xkbvariant nodeadkeys
-xkboptions XKBOptions
 Equivalent to XKBOptions in XF86Config files.
-logfile filename
 Write logmessages to  instead of /tmp/Xwin.log.
-logverbose verbosity
 Set the verbosity of logmessages. [NOTE: Only a few messages
 respect the settings yet]
  0 - only print fatal error.
  1 - print additional configuration information.
  2 - print additional runtime information [default].
  3 - print debugging and tracing information.
-[no]keyhook
 Grab special windows key combinations like Alt-Tab or the Menu key.
 These keys are discarded by default.
-swcursor
 Disable the usage of the windows cursor and use the X11 software cursor instead
You can view this list by typing Xwin --help in a Cygwin bash shell. Also, in the process, I tried the Xming X server. However, I couldn't get it to work with wmii, so I decided to stick with Cygwin's X server.

  1. Edit your startxwin.bat file (located in "C:\cygwin\usr\X11R6\bin") so that it starts X in rootless mode, with clipboard support, disables access control, and doesn't run anything else. Run the new bat file.
    @echo off
    SET DISPLAY=127.0.0.1:0.0
    SET CYGWIN_ROOT=\cygwin
    SET RUN=%CYGWIN_ROOT%\bin\run -p /usr/X11R6/bin
    SET PATH=.;%CYGWIN_ROOT%\bin;%CYGWIN_ROOT%\usr\X11R6\bin;%PATH%
    SET XAPPLRESDIR=/usr/X11R6/lib/X11/app-defaults
    SET XCMSDB=/usr/X11R6/lib/X11/Xcms.txt
    SET XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB
    SET XNLSPATH=/usr/X11R6/lib/X11/locale
    if not exist %CYGWIN_ROOT%\tmp\.X11-unix\X0 goto CLEANUP-FINISH
    attrib -s %CYGWIN_ROOT%\tmp\.X11-unix\X0
    del %CYGWIN_ROOT%\tmp\.X11-unix\X0
    :CLEANUP-FINISH
    if exist %CYGWIN_ROOT%\tmp\.X11-unix rmdir %CYGWIN_ROOT%\tmp\.X11-unix
    
    %RUN% XWin -clipboard -silent-dup-error -rootless -ac
  2. If you are using Slirp-only networking:
    • Obtain your Windows host IP address by opening a cmd.exe shell and running ipconfig
    • In coLinux, set your DISPLAY variable to this IP address:
      root@ubuntu:~# export DISPLAY=xxx.xxx.xxx.xxx:0.0
      where xxx.xxx.xxx.xxx is your IP address. Don't forget the :0.0 at the end.
  3. If you are using TAP plus Slirp configuration described above, set your DISPLAY variable to "10.0.3.17:0"
    root@ubuntu:~# export DISPLAY=10.0.3.17:0
  4. root@ubuntu:~# apt-get install xterm
    Answer yes to prompts.
  5. root@ubuntu:~# xterm&
    Xterm comes up!
  6. You can now install and run other high level apps like firefox or emacs

Expand the root file system image
You will need to expand the size your root filesystem image to hold the larger Ubuntu installation. I expanded it to 4GB for Xubuntu Hardy and am currently using about 87% of that. You might want to expand it to 5GB or more especially if you will be installing a lot of apps. See http://colinux.wikia.com/wiki/ExpandingRoot for more information. Note, Google's calculator function can be useful for calculating block sizes. (e.g. entering in 4GB / 16KB into the Google search box returns: (4 gigabytes) / (16 kilobytes) = 262 144)
  1. If you have coLinux running, shut it down, with halt
  2. In a Cygwin bash shell, make a copy of your current image.
    $ cp Ubuntu-6.06.1.ext3.1gb Ubuntu-6.06.1.ext3.4gb
    This may take a while.
  3. Expand the new file:
    $ dd bs=16384 if=/dev/zero of=Ubuntu-6.06.1.ext3.4gb seek=65536 count=262144

    bs=16384 sets the blocksize to 16KB
    seek=65536 skips the first 65536 blocks (1GB/16KB = 65536)
    count=262144 adds 262144 blocks to the image (4GB/16KB = 262144)

    This may take a while also. It should display the following when complete:
    262144+0 records in
    262144+0 records out
    4294967296 bytes (4.3 GB) copied, 154.362 s, 27.8 MB/s
  4. Change your colinux.conf file to use the new file system image:
    cobd0="c:\coLinux\Ubuntu-6.06.1.ext3.4gb"
  5. Switch back to the cmd.exe console, and start coLinux again
  6. root@ubuntu:~# apt-get install ext2resize
  7. root@ubuntu:~# ext2online /dev/cobd0
  8. Look at your available space:
    root@ubuntu:~# df
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/cobd0             4128448    411148   3507624  11% /
    varrun                  128096        36    128060   1% /var/run
    varlock                 128096         0    128096   0% /var/lock
    devshm                  128096         0    128096   0% /dev/shm
  9. Note, you may want to make a backup copy of your new expanded root file system image at this point in case things get hosed up later on. Be sure to halt coLinux first. Note, this will take even longer than the 1gb filesystem copy.

Upgrade to Ubuntu Hardy
Update 4/28/2008: When I first installed coLinux, Ubuntu Gutsy was the latest release. Because there was no direct upgrade path from Ubuntu 6.06 Dapper to Ubuntu 7.10 Gutsy, I had to incrementally upgrade from Dapper to Edgy, Edgy to Feisty, and Feisty to Gutsy. Later, when Hardy came out, I upgraded from Gutsy to Hardy. If you are upgrading to Dapper to Hardy for the first time, you can use the incremental method, or the direct method. Ubuntu supports a direct upgrade from 6.06 Dapper Drake to 8.04 Hardy Heron because both are Long Term Support (LTS) releases. See the Ubuntu upgrade documentation for more information. Note, I did not test the direct upgrade method.
  1. In coLinux:
    root@ubuntu:~# apt-get update
  2. root@ubuntu:~# apt-get install ubuntu-minimal
  3. root@ubuntu:~# apt-get install ubuntu-standard
  4. root@ubuntu:~# apt-get install xubuntu-desktop
  5. Edit your sources.list file and change "dapper" to "edgy"
    root@ubuntu:~# nano /etc/apt/sources.list
  6. root@ubuntu:~# apt-get update
  7. root@ubuntu:~# apt-get upgrade
  8. root@ubuntu:~# apt-get dist-upgrade
  9. Change "edgy" to "feisty" in /etc/apt/sources.list and repeat steps 6-8.
  10. Change "feisty" to "gutsy" in /etc/apt/sources.list and repeat steps 6-8.
  11. Change "gutsy" to "hardy" in /etc/apt/sources.list and repeat steps 6-8.
  12. Remove unneeded packages:
    root@ubuntu:~# apt-get autoremove

Disable some startup services
This is optional but may give you fewer warnings. In coLinux, go to the /etc/rc2.d directory and follow the instructions in the README file for disabling services. Here are some suggested services to disable:
  • acpid, acpi-support, apmd: used for power management
  • bluetooth: used for bluetooth
  • hotkey-setup: used to configure laptop hotkeys
  • pcmciautils: tools for PCMCIA cards (PC cards)
  • powernowd: controls CPU speed and voltage using the sysfs interface
  • vbesave: used to save the video card state

Configure Ubuntu
  1. Don't start gdm (the display manager which starts the X server)
    root@ubuntu:~# mv /etc/rc2.d/S13gdm /etc/rc2.d/K87gdm
  2. Set the timezone
    Start your Cygwin X server if it is not already running.
    root@ubuntu:~# time-admin &
    Set your timezone.
  3. Set the root password
    root@ubuntu:~# passwd
  4. Set local machine name:
    root@ubuntu:~# nano /etc/hostname
    root@ubuntu:~# nano /etc/hosts
  5. Create a user account
    root@ubuntu:~# adduser sofeng
    Note, adduser is a higher level command than useradd. adduser will set up a group and home directory.
  6. Set permissions on /dev/tty* (Note: not sure if this is needed)
    root@ubuntu:~# chgrp tty /dev/tty*
    Edit /etc/group to have the following line:
    tty:x:5:sofeng
  7. Make a pty device
    root@ubuntu:~# cd /dev
    root@ubuntu:~# MAKEDEV pty
  8. Add environment variables to .bash_profile
    Use the printenv command to list your environment variables.

Create a home filesystem image
  1. Create a 10GB blank file. In Cygwin,
    $ dd if=/dev/zero of=home-fs-image.10gb bs=1M count=10K
  2. Add the following to your c:\coLinux\colinux.conf file:
    cobd2="c:\coLinux\home-fs-image.10gb"
  3. Restart coLinux, then do the following:
    Create an ext2 file system on the new disk image
    root@ubuntu:~# mke2fs -F /dev/cobd2
    Turn this into an ext3 filesystem
    root@ubuntu:~# tune2fs -i 0 -j /dev/cobd4
  4. Mount the new filesystem to /home
    root@ubuntu:~# mv /home /home_backup
    root@ubuntu:~# mkdir /home
    root@ubuntu:~# mount /dev/cobd2 /home
    Check it worked:
    root@ubuntu:~# df
  5. To mount the filesystem automatically at startup, add the following line to your /etc/fstab:
    /dev/cobd2   /home   ext3   defaults   0 0

Setup the swap file
  1. Add the following to /etc/fstab:
    /dev/cobd1   swap   swap   defaults  0 0
  2. Initialize the swap partition
    root@ubuntu:~# sudo mkswap /dev/cobd1
  3. make use of swap now, without rebooting system:
    root@ubuntu:~# swapon -a
  4. Check that swap total is non-zero:
    root@ubuntu:~# free

Mount Windows filesystem as Samba share
  1. Setup Windows networking:
    • In Windows Explorer, right click on your C: drive and select "Properties"
    • Click on the "Sharing" tab
    • Select "Share this folder". Set the "Share name:" to "E$". Setup "Permissions" as necessary.
  2. create the location for your samba share:
    $ sudo mkdir -p /mnt/cdrive
  3. add the following to /etc/fstab:
    //yourcomputer.example.com/E$ /mnt/cdrive smbfs auto,user,rw,dir_mode=0777,file_mode=0777,uid=sofeng,gid=sofeng,credentials=/etc/smb-credentials 0 0
    /mnt/cdrive is the mount point
    smbfs means it is a samba filesystem type
    auto means it will be mounted at startup
    user means any user can mount it
    rw means it will be readable and writable
    file_mode=0777 specifies that files will be readable, writable, and executable by everyone (this replaces the deprecated fmask)
    dir_mode=0777 specifies that directories will be readable, writable, and executable by everyone (this replaces the deprecated dmask)
    uid=sofeng means the owner is sofeng
    gid=sofeng means the group is sofeng
    credentials=/etc/smb-credentials specifies the file that contains the user name and password
  4. create /etc/smb-credentials
    username=yourdomain\yourusername
    password=yourpassword
    $ sudo chmod 600 /etc/smb-credentails
  5. Mount the drive:
    $ sudo mount /mnt/cdrive

Set up wmii window manager
  1. Install wmii window manager
    apt-get install wmii
  2. I use emacs so the following steps are used to change the mod key.
  3. Download KeyTweak for Windows and remap your "Left Windows" key to "Context Menu".
  4. Use xmodmap to setup the "Menu" key to be "mod4". For my keyboard, my ~/.Xmodmap file looks like this:
    clear mod4
    keycode 115 =
    keycode 127 =
    keycode 117 = Super_L
    add mod4 = Super_L
    Then run xmodmap in one of your startup scripts (.xinitrc, .bash_profile, etc.)
    xmodmap ~/.Xmodmap

Backup on Linux: rsnapshot vs. rdiff-backup (vs. Time Machine)

Apple's release of Leopard and the included backup utility, Time Machine, has generated a lot of talk about backups recently. I will admit Time Machine is pretty cool and believe that it is a bit more than a glorified GUI on top of an existing *nix tool as some have claimed. However, the core functionality is very similar to the command-line tool, rsnapshot, which is itself based on a rsync script by Mike Rubel. Time Machine added a couple of features and a GUI to make it easy to use. Since I prefer the command line over GUIs most of the time anyways, rsnapshot seemed perfect for me.

To be complete, I researched a number of other backup utilities for Linux. Dirvish and flyback were out because I prefer the command line and they didn't seem to offer anything more than rsnapshot. Scripting rsync wouldn't get me anything more than rsnapshot either, plus it would be more work. In the end, I eliminated all but rsnapshot and another command line tool called rdiff-backup. Rdiff-backup has some advantages over rsnapshot (and Time Machine) because it stores compressed deltas of each version of a file instead of a complete copy of the file each time it changes. This is not a big deal for small files, but for large files that change often, it makes a significant difference. However, the big disadvantage of rdiff-backup, for me, was the inablility to perform different levels of backup, such as hourly, daily, weekly, monthly, etc. Depending on the needs of the user, this could negate the space saving advantage by needing to keep a large number of snapshots.

I ended up choosing rsnapshot over rdiff-backup because of this last reason. It seems rdiff-backup is closer to a version control tool and rsnapshot closer to a traditional backup solution. It would be great to create a hybrid of the two tools to gain the advantages of each. I started to dig into the source of rdiff-backup (python source makes me happy), but I didn't want to get too sidetracked with another project. For now, I am using Mercurial to version control my /etc and /home/.* config files and rsnapshot as a broader, general purpose backup tool.


Here is my comparison of rsnapshot and rdiff-backup:

Similar Features
  • both use an rsync-like algorithm to transfer data (rsnapshot actually uses rsync; rdiff-backup uses the python librsync library)
  • both can be used over ssh (though rsnapshot cannot push over ssh without some extra scripting)
  • both use a simple copy of the source for the current backup
Written inrsnapshot is written in Perl; rdiff-backup is written in Python and C
Sizerdiff-backup stores previous versions as compressed deltas to the current version similar to a version control system. rsnapshot uses actual files and hardlinks to save space. For small files, storage size is similar. For large files that change often, such as logfiles, databases, etc., rdiff-backup requires significantly less space for a given number of versions.
Speedrdiff-backup is slower than rsnapshot
Metadatardiff-backup stores file metadata, such as ownership, permissions, and dates, separately.
TransparencyFor rsnapshot, all versions of the backup are accessible as plain files. For rdiff-backup, only the current backup is accessible as plain files. Previous versions are stored as rdiff deltas.
Backup levelsrsnapshot supports multiple levels of backup such as monthly, weekly, and daily. rdiff-backup can only delete snapshots earlier than a given date; it cannot delete snapshots in between two dates.
CommunityBased on the number of responses to my post on the mailing lists (rsnapshot: 6, rdiff-backup: 0), rsnapshot has a more active community.

Reference: Chapter 7 in Backup & Recovery O'Reilly 2007

How to get a summary of disk usage with du

The following command gives me a listing of the sizes of all the directories in my root directory.

$ sudo du -chs /*

Here is the results of that command on my new (1 day old) Dell 530N Ubuntu desktop: (Note, the 5.0GB for the /home directory is due to the 5.0GB ubuntu-dell-reinstall.iso file. The rest of my /home directory is only 18MB.)

4.8M    /bin
18M     /boot
0       /cdrom
128K    /dev
9.1M    /etc
5.0G    /home
4.0K    /initrd
0       /initrd.img
154M    /lib
16K     /lost+found
12K     /media
4.0K    /mnt
13M     /opt
0       /proc
740K    /root
6.3M    /sbin
4.0K    /srv
0       /sys
80K     /tmp
2.1G    /usr
317M    /var
0       /vmlinuz
7.6G    total

To determine disk usage in a different directory, cd to that directory and run sudo du -chs *

Undoing changes by pppoeconf

I just got my new Dell with Ubuntu and thought I needed to configure something to get my new DSL service up and running so I ran pppoeconf. (It turns out I needed to register with ATT first. See my previous post.) So I needed to undo the changes I made with pppoeconf. Googling around a little, it appears there is no undo command for pppoeconf. However, I found the key configuration file was: /etc/network/interfaces. I looked at it, but wasn't sure which stuff to delete. I popped in my Ubuntu live installation disk (included with my Dell). Using the live CD, the internet worked great. So I looked at the /etc/network/interfaces while using the live CD and then edited my installed version to match.


Here is what my /etc/network/interfaces file looked like after running pppoeconf:

auto lo
iface lo inet loopback


auto dsl-provider
iface dsl-provider inet ppp
pre-up /sbin/ifconfig eth0 up # line maintained by pppoeconf
provider dsl-provider

auto eth0
iface eth0 inet manual

Here is the default /etc/network/interfaces on Ubuntu 7.10 Gutsy (only two lines):

auto lo
iface lo inet loopback

Setting up ATT Yahoo DSL with Ubuntu linux

I ordered new ATT/Yahoo DSL service the day before ordering my Dell 530N desktop with Ubuntu 7.10 Gutsy installed. ATT told me only Windows and Mac were supported for their DSL service. I guess they have an install CD that only runs on Windows or Mac. After some internet searching, I found that it is possible to set up the DSL service without the install CD. Here is the link which helped me with the process. The following is the summary of my steps on Ubuntu linux. I am using a SpeedStream DSL modem; this may or may not work with other modems.

  1. Connect the modem to everything (phone line to phone jack, ethernet cable to your PC, and, of course, the power adapter).
  2. Turn on the modem.
  3. On the underside of my modem was a yellow sticker that said "For Advanced Modem Configuration Go to: http://192.169.0.1"
  4. Go to http://192.169.0.1 in your web browser
  5. Enter the following login information:
  6. Enter in the Modem Access Code found on the bottom of your DSL modem.
  7. It will look like you are connected to the internet at this point, but you won't be able to get to Yahoo or Google. This login provides limited connectivity so you can get to the ATT registration site.
  8. Go to https://sbcreg.sbcglobal.net to get to the registration site. (If you are part of another domain, you will need to go to a different registration site. e.g. att.net customers go to http://helpme.att.net/register)
  9. Complete the registration process. You will need your DSL phone number. When you get to the Member ID and password confirmation page, you are done. Print/write down/memorize this information. You do not need to complete the rest of the registration process.
  10. Go back to http://192.169.0.1 in your web browser and enter in your new Member ID and password. (You will probably have to logout and then login again.)
  11. You should now be connected to the internet.

Note: this assumes the default Ubuntu 7.10 network configuration which automatically detects your active DSL connection. If you ran pppoeconf by mistake, see this post.

Dell 530N Ubuntu vs. build it yourself

I just received my new Dell 530N with Ubuntu 7.10 pre-loaded. I seriously considered building a PC myself but after weighing the pros and cons decided to go with the Dell. The main reason I chose the Dell was because it was the faster/easier option. The cost comparison was pretty similar. Hal's Notes has an article which compares the cost of the 530N vs. the cost of parts from newegg.com. Here is my very similar anaylsis.

My 530N was $349 (unfortunately i had to pay $30 for shipping) and included the following:

  • Intel Pentium Dual Core processor E2160 (1.80GHz,800FSB)
  • 1GB DDR2 SDRAM at 667MHz
  • 128MB NVIDIA GeForce 8300GS
  • 250GB SATA II Hard Drive (7200RPM)
  • 48X CDRW/DVD Combination Drive

The only thing I upgraded was the processor-- from the E2140 to the E2160 because I had read it was better for overclocking (if it turns out to be possible to overclock with Dell's motherboard).

For the build it myself option, I started with a budget configuration from The Tech Report. The total for that machine in November 2007 was $519. Adjusting the items to closer match the Dell 530N got me down to $409. See below.

  • LITE-ON Combo Black SATA Model DH-52C2S-04 - OEM
    $24.99
  • Antec NSK4480 Black/ Silver 0.8mm cold-rolled steel construction ATX Mid Tower Computer Case 380W Power Supply - Retail
    $79.95
  • Western Digital Caviar SE16 WD2500KS 250GB 7200 RPM SATA 3.0Gb/s Hard Drive - OEM
    $69.99
  • EVGA 128-P2-N428-LR GeForce 7200GS 512MB (128MB on Board) 64-bit GDDR2 PCI Express x16 Video Card - Retail
    $34.99
  • Kingston 1GB 240-Pin DDR2 SDRAM DDR2 667 (PC2 5300) Desktop Memory Model KVR667D2N5/1G - Retail
    $19.99
  • GIGABYTE GA-P35-DS3L LGA 775 Intel P35 ATX Intel Motherboard - Retail
    $99.99
  • Intel Pentium E2160 Allendale 1.8GHz LGA 775 65W Dual-Core Processor Model BX80557E2160 - Retail
    $78.99
  • Subtotal: $408.89

I didn't change the motherboard and case specified in article. I am guessing they are better than the Dell. Besides that, the Dell comes out cheaper. Note, that if I had upgraded Dell's components, say to 2GB RAM, a 320GB hard disk, a DVD burner, and a 256MB video card, the advantage goes out the window. I am guessing this is how Dell makes money-- lure the customer in with a low base price and charge extra for upgrades.

I went back and forth on the decision multiple times. One thing that almost made me build it myself was the much better chance of being able to overclock my processor with a standard motherboard. According to this article on hardwarezone.com, the E2160 can be overclocked to almost double and will still remain very stable. With standard motherboards, this is not too difficult to achieve. However, Dell's motherboards make it much more difficult. Hopefully for me, someone will figure it out and post an article about it.

PyQt: How to pass arguments while emitting a signal

I often forget how to do this so I'm documenting it here for future reference. If I want to emit a signal and also pass an argument with that signal, I can use the form self.emit(SIGNAL("mySignalName"), myarg). I connect the signal to a method in the usual way. To use the argument, I merely need to specify the argument in the method definition. What often confuses me is that I don't need to specify arguments in the connect statement. The example below emits a signal didSomething and passes two arguments, "important" and "information" to the update_label method.

import sys
import time
from PyQt4.QtCore import * 
from PyQt4.QtGui import * 

#################################################################### 
class MyWindow(QWidget): 
    def __init__(self, *args): 
        QWidget.__init__(self, *args)

        self.label = QLabel(" ")
        layout = QVBoxLayout()
        layout.addWidget(self.label)
        self.setLayout(layout)
        self.connect(self, SIGNAL("didSomething"),
                     self.update_label)
        self.do_something()

    def do_something(self):
        self.emit(SIGNAL("didSomething"), "important", "information")

    def update_label(self, value1, value2):
        self.label.setText(value1 + " " + value2)

####################################################################
if __name__ == "__main__": 
    app = QApplication(sys.argv) 
    w = MyWindow() 
    w.show() 
    sys.exit(app.exec_())

Conkeror: "Firefox for Emacs users"

I recently discovered Conkeror via Bill Clementson's article, Firefox for Emacs users. I must say it is pretty sweet. It's the type of thing that makes you have to get up and go to the bathroom because it's so exciting.

So I switched over to Emacs from Eclipse about 4 months ago. Though it has been slow learning everything in Emacs, I am starting to feel pretty comfortable now and actually feel uneasy when I'm using another editor. It is very nice to be able to do everything with the keyboard in Emacs. I can do all cursor movement, scrolling, and selection very nicely in Emacs. And all text manipulation commands are immediately accessible from the keyboard. Pair that with a bash command line running inside GNU screen, and I can do about 50% of my work without using a mouse. Of the other 50%, one big chunk is Microsoft Outlook which, as far as I know, I need because we are using an Exchange server. But the other big chunk is web browsing, which, up until now, has been very mouse dependent.

Enter Conkeror, the keyboard-driven web browser that "conkers" the web without a mouse. (I just made that up, btw. I do not know the origin of the name "Conkeror".) Conkeror is a Mozilla-based browser (e.g. Firefox) with an Emacs-like skin. Emacs users should feel at right home-- screen-stealing toolbars and menubars are gone. In their place is the modeline and mini-buffer. Commands are entered using M-x, switch buffers using C-x b, incremental search using C-s, and edit text using Emacs editing conventions. (Being a relative Emacs newbie, I actually rebound a lot of these to be more Windows/Firefox friendly. But I expect long time Emacs users would rejoice.)

In my opinion, the best part about Conkeror is the hyperlink navigation design. Hitting the f key while browsing highlights all possible hyperlinks (or focus areas) within the current view. Limiting the scope of choices to the current view is smart because you most likely won't want to follow a link that's not in the current view. To follow a link, there are two options. The first is to type in the link number which is overlaid in the top left corner of each link. The second method is to start typing any part of the link text and tab through the remaining selections. A different color highlight shows which link is currently selected. The second method is very fast-- however, the first method is more robust in that it can handle links with no text or a large number of links with similar text.

Conkeror is currently being actively developed and can be considered to be alpha stage software so there are issues and bugs. Also the documentation seems to be pretty sparse. However, to me, it looks like a really great tool with a potential for much more efficient and comfortable browsing. Kudos to the authors.

Will I ditch Firefox for Conkeror? That question remains to be answered. I hope the answer will be yes. I plan to use it for a few days and see if I run across any deal-breakers. Hopefully, with Conkeror, I can up my mouseless activity to around 70%.

Below are some notes on using Conkeror. I am using the snapshot XULRunner version of Conkeror on Windows XP.



How to set a proxy server

Assuming you have your proxy configured in Firefox already...

  • Type g about:config and hit ENTER in Conkeror.
  • Hit ENTER to promise you'll be careful.
  • Hit TAB to get to the "Filter:" box and type in "proxy".
  • Go to about:config in Firefox and filter on "proxy".
  • Make the relevant settings in Conkeror match Firefox. Make sure you set "network.proxy.type" in addition to the proxy servers. (If you were wondering, no, I don't know how to set these preferences without using the mouse.)


Here is my .conkerorrc file:

See also dotfiles.org for a few more.

// new bindings
define_key(default_global_keymap, "C-w", "kill-current-buffer");

// rebound from global.js
define_key(default_global_keymap, "h", "find-url");
define_key(default_global_keymap, "f1", default_help_keymap);
define_key(default_global_keymap, "b", "switch-to-buffer");
define_key(default_global_keymap, "C-f", "isearch-forward"); 
// note, i had to comment out the "C-f" binding in basic-commands.js to make this work

// rebound from isearch.js
define_key(isearch_keymap, "C-f", "isearch-continue-forward");

// rebound from normal.js
define_key(content_buffer_normal_keymap, "back_space", "go-back");
define_key(content_buffer_normal_keymap, "S-back_space", "go-forward");

// rebound from element.js
//define_key(content_buffer_normal_keymap, "k", "bookmark");

// rebound from basic-commands.js
define_key(content_buffer_normal_keymap, "home","beginning-of-line");
define_key(content_buffer_normal_keymap, "end","end-of-line");
define_key(content_buffer_normal_keymap, "C-home","cmd_scrollTop");    
define_key(content_buffer_normal_keymap, "C-end","cmd_scrollBottom");

// rebound from text.js
define_key(content_buffer_text_keymap, "home", "cmd_beginLine");
define_key(content_buffer_text_keymap, "end", "cmd_endLine");
define_key(content_buffer_text_keymap, "C-left", "cmd_wordPrevious");
define_key(content_buffer_text_keymap, "C-right", "cmd_wordNext");
define_key(content_buffer_text_keymap, "C-y", "cmd_redo");
define_key(content_buffer_text_keymap, "C-z", "cmd_undo");
define_key(content_buffer_text_keymap, "C-v", "cmd_paste");
define_key(content_buffer_text_keymap, "C-x", "cmd_cut");
define_key(content_buffer_text_keymap, "C-c", "cmd_copy");

// new for textarea.js
define_key(content_buffer_textarea_keymap, "home", "cmd_beginLine");
define_key(content_buffer_textarea_keymap, "end", "cmd_endLine");
define_key(content_buffer_textarea_keymap, "C-left", "cmd_wordPrevious");
define_key(content_buffer_textarea_keymap, "C-right", "cmd_wordNext");
define_key(content_buffer_textarea_keymap, "C-y", "cmd_redo");
define_key(content_buffer_textarea_keymap, "C-z", "cmd_undo");
define_key(content_buffer_textarea_keymap, "C-v", "cmd_paste");
define_key(content_buffer_textarea_keymap, "C-x", "cmd_cut");
define_key(content_buffer_textarea_keymap, "C-c", "cmd_copy");

// rebound for textarea.js
define_key(content_buffer_textarea_keymap, "C-home", "cmd_moveTop");
define_key(content_buffer_textarea_keymap, "C-end", "cmd_moveBottom");

// rebound from zoom.js
define_key(content_buffer_normal_keymap, "C-subtract", "zoom-out-text");
define_key(content_buffer_normal_keymap, "C-add", "zoom-in-text");

// webjumps 
add_delicious_webjumps ("saltycrane");
add_webjump("g", "www.google.com/search?q=%s");
add_webjump("d", "del.icio.us/saltycrane/%s");
add_webjump("blank", "about:blank");
add_webjump("reader", "www.google.com/reader");


How to search the Conkeror mailing list archive

Don't use the one linked from the Conkeror website. Instead use: http://www.nabble.com/MozDev---conkeror-f11304.html



How to get out of Adobe reader when visiting a pdf file

See this thread on the MozDev - conkeror mailing list archive.



How to copy a link

Use the c command.



add_command doesn't work!

Per the ChangeLog, it was renamed to interactive. I still can't figure out how to use message though.



How to uninstall XULRunner

Run the command xulrunner --unregister-user and then delete the xulrunner directory. If you deleted the xulrunner directory without unregistering, delete the registry items in HKEY_CURRENT_USER\Software\mozilla.org\GRE on Windows, or the directory ~/.gre.d on Linux.

Update 2008-09-11: I discovered Conkeror now has external editing capabilities. I can hit C-i in a text box and it will bring up Emacs. Problems: I set EDITOR and VISUAL to be emacsclient, however, when I spawn conkeror from wmii using MOD-p, and hit C-i in conkeror, it starts a new emacs server instead of connecting to my current one. However, if I just start conkeror from my terminal command line, it works correctly. Also, I don't know how to change the major mode of Emacs for editing the text boxes. Otherwise, good stuff.

Unaswered Questions

If you know the answer, please leave a comment.

 

1. How do I unpack a tarball to a specific directory?

I know I can uncompress a tar file in the current working directory with tar –xvf mytarfile.tar. Can I upack it to another directory instead?

 

 

How to scroll in GNU Screen

GNU Screen is great. But, after half a year of using it, I've only now figured out how to scroll into screen's buffer (courtesy of this tutorial). Normally, I just use SHIFT+PGUP to scroll up in urxvt's buffer. However, if I have two regions in screen, this doesn't work. To scroll in screen's buffer, I need to enter "copy mode". To do this, type C-a [ or C-a ESC. (The latter option works great for me because I bound the prefix key to the backtick key instead of C-a.) Once in "copy mode", I can scroll using the arrow keys or PGUP/PGDOWN keys. To exit "copy mode", I press ESC. Scrolling within screen's buffer instead of urxvt's also prevents me from losing my place in my scrollback buffer when I switch screen "windows" (terminal sessions).

Other useful commands
  • C-a S splits into two regions
  • C-a tab switches input focus to the next region
  • C-a X kills the current region
  • C-a :resize numlines resizes the current region to numlines lines.
Custom key bindings for scrolling (updated 2009-04-01)

If you don't want to hit C-a ESC PGUP everytime to page up, you can create keyboard shortcuts. I use CTRL+K to scroll up one line and META+K to scroll up one page (I know these are kind of weird key combinations). To create these shortcuts, I put the following in my .screenrc.

bindkey "^[k" eval "copy" "stuff ^b"  # enter copy mode and move up one page
bindkey "^k" eval "copy" "stuff k"    # enter copy mode and move up one line
bindkey -m "^[k" stuff ^b             # move up one page
bindkey -m "^k" stuff k               # move up one line

Now, when I'm in screen and press M-k, screen enters copy mode and scrolls up one page. Likewise, C-k enters copy mode and scrolls up one line. As before, to exit copy mode, hit ESC.

How to copy and paste in screen (added 2010-02-03)

Thanks to this GNU Screen cheat-sheet.

  • Enter scrollback mode using C-a [ or C-a ESC
  • Press the space bar to begin selecting text.
  • Move the cursor using h,j,k,l,C-b,C-f or arrow keys/PGUP/PGDOWN
  • Hit the space bar again to copy the selected text.
  • Hit ESC to exit scrollback mode.
  • Press C-a ] to paste.
How to open 2 terminals connected to the same session (Multi display mode) (updated 2012-04-05)

The -x option can be used to attach multiple terminals to the same screen session. Pretty cool! Open a terminal and create a screen session named "local"

$ screen -S local 

Start a second terminal, then attach it to the existing session:

$ screen -x local 
How to eliminate apparent delay when entering copy mode (updated 2012-04-05)

To get rid of the apparent delay when entering copy mode, I set msgminwait to 0 in my ~/.screenrc

msgminwait 0
My ~/.screenrc on github

Here is my ~/.screenrc on github.

See also

See also my post, How to start a long-running process in screen and detach from it and some "screen vs. tmux" links in the comments.