SaltyCrane: files_directorieshttps://www.saltycrane.com/blog/2011-12-28T11:06:05-08:00How to get the filename and it's parent directory in Python
2011-12-28T11:06:05-08:00https://www.saltycrane.com/blog/2011/12/how-get-filename-and-its-parent-directory-python/<pre class="python">import os.path
def get_file_with_parents(filepath, levels=1):
common = filepath
for i in range(levels + 1):
common = os.path.dirname(common)
return os.path.relpath(filepath, common)
print get_file_with_parents(
'/opt/xyzabc/etc/yaml/working/group1/vendor1.yaml', 1)
print get_file_with_parents(
'/opt/xyzabc/etc/yaml/working/group1/vendor1.yaml', 2)</pre>
<p>Results:</p>
<pre>group1/vendor1.yaml
working/group1/vendor1.yaml</pre>
How to remove ^M characters from a file with Python
2011-10-03T11:26:03-07:00https://www.saltycrane.com/blog/2011/10/how-remove-m-characters-from-file-python/<p>Use the following Python script to remove <code>^M</code> (carriage
return) characters from your file and replace them with newline
characters only. To do this in Emacs,
see <a href="/blog/2008/11/emacs-notes/#remove-ctrl-m-chars">my notes here</a>.</p>
<p>remove_ctrl_m_chars.py:</p>
<pre class="python">import os
import sys
import tempfile
def main():
filename = sys.argv[1]
with tempfile.NamedTemporaryFile(delete=False) as fh:
for line in open(filename):
line = line.rstrip()
fh.write(line + '\n')
os.rename(filename, filename + '.bak')
os.rename(fh.name, filename)
if __name__ == '__main__':
main()</pre>
<p>Run it</p>
<pre class="console">$ python remove_ctrl_m_chars.py myfile.txt </pre>
<h4>Documentation</h4>
<ul>
<li><a href="http://docs.python.org/library/functions.html#open">
Built-in Functions — <code>open()</code>
</a></li>
<li><a href="http://docs.python.org/library/stdtypes.html#bltin-file-objects">
Built-in Types — File Objects
</a></li>
<li><a href="http://docs.python.org/library/tempfile.html">
<code>tempfile</code> — Generate temporary files and directories
</a></li>
<li><a href="http://docs.python.org/library/stdtypes.html#str.rstrip">
String Methods — <code>str.rstrip()</code>
</a></li>
<li><a href="http://docs.python.org/library/os.html#os.rename">
Files and Directories — <code>os.rename()</code>
</a></li>
</ul>
Options for listing the files in a directory with Python
2010-04-19T16:57:04-07:00https://www.saltycrane.com/blog/2010/04/options-listing-files-directory-python/<p>I do a lot of sysadmin-type work with Python so I often need to
list the contents of directory on a filesystem. Here are 4 methods I've used so
far to do that. Let me know if you have any good alternatives. The examples
were run on my Ubuntu Karmic machine.
</p>
<h4 id="os-listdir">OPTION 1 -
<a href="http://docs.python.org/library/os.html#os.listdir">
<code><b>os.listdir()</b></code></a></h4>
<p>This is probably the simplest way to list the contents of a
directory in Python.</p>
<pre class="python">import os
dirlist = os.listdir("/usr")
from pprint import pprint
pprint(dirlist)</pre>
<p>Results:</p>
<pre>['lib',
'shareFeisty',
'src',
'bin',
'local',
'X11R6',
'lib64',
'sbin',
'share',
'include',
'lib32',
'man',
'games']</pre>
<h4 id="glob-glob">OPTION 2 -
<a href="http://docs.python.org/library/glob.html#glob.glob">
<code><b>glob.glob()</b></code></a></h4>
<p>This method allows you to use shell-style wildcards.</p>
<pre class="python">import glob
dirlist = glob.glob('/usr/*')
from pprint import pprint
pprint(dirlist)</pre>
<p>Results:</p>
<pre>['/usr/lib',
'/usr/shareFeisty',
'/usr/src',
'/usr/bin',
'/usr/local',
'/usr/X11R6',
'/usr/lib64',
'/usr/sbin',
'/usr/share',
'/usr/include',
'/usr/lib32',
'/usr/man',
'/usr/games']</pre>
<h4 id="unix-ls">OPTION 3 - Unix "ls" command using
<a href="http://docs.python.org/library/subprocess.html">
<code><b>subprocess</b></code></a></h4>
<p>This method uses your operating system's "ls" command. It allows you to sort
the output based on modification time, file size, etc. by passing these command-line
options to the "ls" command. The following example lists the 10 most recently
modified files in <code>/var/log</code>:</p>
<pre class="python">from subprocess import Popen, PIPE
def listdir_shell(path, *lsargs):
p = Popen(('ls', path) + lsargs, shell=False, stdout=PIPE, close_fds=True)
return [path.rstrip('\n') for path in p.stdout.readlines()]
dirlist = listdir_shell('/var/log', '-t')[:10]
from pprint import pprint
pprint(dirlist)</pre>
<p>Results:</p>
<pre>['auth.log',
'syslog',
'dpkg.log',
'messages',
'user.log',
'daemon.log',
'debug',
'kern.log',
'munin',
'mysql.log']</pre>
<h4 id="os-walk">OPTION 4 - Unix "find" style using
<a href="http://docs.python.org/library/os.html#os.walk">
<code><b>os.walk</b></code></a></h4>
<p>This method allows you to list directory contents recursively
in a manner similar to the Unix "find" command. It uses Python's
<code>os.walk</code>.</p>
<pre class="python">import os
def unix_find(pathin):
"""Return results similar to the Unix find command run without options
i.e. traverse a directory tree and return all the file paths
"""
return [os.path.join(path, file)
for (path, dirs, files) in os.walk(pathin)
for file in files]
pathlist = unix_find('/etc')[-10:]
from pprint import pprint
pprint(pathlist)</pre>
<p>Results:</p>
<pre>['/etc/fonts/conf.avail/20-lohit-gujarati.conf',
'/etc/fonts/conf.avail/69-language-selector-zh-mo.conf',
'/etc/fonts/conf.avail/11-lcd-filter-lcddefault.conf',
'/etc/cron.weekly/0anacron',
'/etc/cron.weekly/cvs',
'/etc/cron.weekly/popularity-contest',
'/etc/cron.weekly/man-db',
'/etc/cron.weekly/apt-xapian-index',
'/etc/cron.weekly/sysklogd',
'/etc/cron.weekly/.placeholder']</pre>
Monitoring a filesystem with Python and Pyinotify
2010-04-09T10:02:39-07:00https://www.saltycrane.com/blog/2010/04/monitoring-filesystem-python-and-pyinotify/<p><a href="http://trac.dbzteam.org/pyinotify">Pyinotify</a>
is a Python library for monitoring filesystem events
on Linux through the
<a href="http://en.wikipedia.org/wiki/Inotify">inotify</a>
Linux kernel subsystem.
It can monitor when a file is created, accessed, deleted,
modified, etc. For a full list of Pyinotify events see
<a href="http://trac.dbzteam.org/pyinotify/wiki/DocumentationPage">
the documentation</a>.
</p>
<h4 id="install">Install Pyinotify</h4>
<ul>
<li><a href="/blog/2010/02/how-install-pip-ubuntu/">Install pip</a></li>
<li>Install Pyinotify
<pre class="console">$ sudo pip install pyinotify</pre>
</li>
</ul>
<h4 id="example">Example</h4>
<pre class="python">import pyinotify
class MyEventHandler(pyinotify.ProcessEvent):
def process_IN_ACCESS(self, event):
print "ACCESS event:", event.pathname
def process_IN_ATTRIB(self, event):
print "ATTRIB event:", event.pathname
def process_IN_CLOSE_NOWRITE(self, event):
print "CLOSE_NOWRITE event:", event.pathname
def process_IN_CLOSE_WRITE(self, event):
print "CLOSE_WRITE event:", event.pathname
def process_IN_CREATE(self, event):
print "CREATE event:", event.pathname
def process_IN_DELETE(self, event):
print "DELETE event:", event.pathname
def process_IN_MODIFY(self, event):
print "MODIFY event:", event.pathname
def process_IN_OPEN(self, event):
print "OPEN event:", event.pathname
def main():
# watch manager
wm = pyinotify.WatchManager()
wm.add_watch('/var/log', pyinotify.ALL_EVENTS, rec=True)
# event handler
eh = MyEventHandler()
# notifier
notifier = pyinotify.Notifier(wm, eh)
notifier.loop()
if __name__ == '__main__':
main()</pre>
<p>Results:</p>
<pre>MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
OPEN event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-node.log
MODIFY event: /var/log/munin/munin-update.log
MODIFY event: /var/log/munin/munin-update.log
CLOSE_WRITE event: /var/log/munin/munin-update.log
OPEN event: /var/log/munin/munin-limits.log
MODIFY event: /var/log/munin/munin-limits.log
CLOSE_WRITE event: /var/log/munin/munin-limits.log
OPEN event: /var/log/munin/munin-graph.log
MODIFY event: /var/log/munin/munin-graph.log
CLOSE_WRITE event: /var/log/munin/munin-graph.log
OPEN event: /var/log/munin/munin-html.log
MODIFY event: /var/log/munin/munin-html.log
CLOSE_WRITE event: /var/log/munin/munin-html.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/auth.log
MODIFY event: /var/log/syslog
MODIFY event: /var/log/auth.log
...
</pre>
os.path.relpath() source code for Python 2.5
2010-03-31T10:30:01-07:00https://www.saltycrane.com/blog/2010/03/ospathrelpath-source-code-python-25/<p>Today I needed to use the
<a href="http://docs.python.org/library/os.path.html#os.path.relpath">
<code>os.path.relpath()</code></a> function. However this function
was introduced in Python 2.6 and I am using Python 2.5 for my project.
Luckily, James Gardner has written a version that works with Python 2.5
(on Posix systems (which mine is (Linux))). His <code>relpath</code>
function is part of his
<a href="http://jimmyg.org/work/code/barenecessities/index.html">BareNecessities</a>
package. You can view the documentation
<a href="http://jimmyg.org/work/code/barenecessities/0.2.5/manual.html#the-relpath-function">here</a>.
</p>
<p>Here is James Gardner's <code>relpath</code> function:</p>
<pre class="python">from posixpath import curdir, sep, pardir, join
def relpath(path, start=curdir):
"""Return a relative version of a path"""
if not path:
raise ValueError("no path specified")
start_list = posixpath.abspath(start).split(sep)
path_list = posixpath.abspath(path).split(sep)
# Work out how much of the filepath is shared by start and path.
i = len(posixpath.commonprefix([start_list, path_list]))
rel_list = [pardir] * (len(start_list)-i) + path_list[i:]
if not rel_list:
return curdir
return join(*rel_list)</pre>
A hack to copy files between two remote hosts using Python
2010-02-08T12:51:29-08:00https://www.saltycrane.com/blog/2010/02/hack-copy-files-between-two-remote-hosts-using-python/<p>I sometimes need to copy a file (such as a database dump) between two
remote hosts on EC2. Normally this involves a few steps: scp'ing the ssh keyfile
to Host 1, ssh'ing to Host 1, looking up the address for Host 2, then scp'ing the desired
file from Host 1 to Host 2.</p>
<p>I was excited to read in the man page that scp can copy files between two
remote hosts directly. However, it didn't work for me.
<a href="http://fixunix.com/ssh/258554-should-i-able-scp-between-two-remote-hosts.html#post700563">Apparently</a>,
running <code>scp host1:myfile host2:</code> is like running
<code>ssh host1 scp myfile host2:</code> so I still need the address of host2
and my ssh keyfile on host1.
</p>
<p>My inablility to let go of this small efficiency increaser, led me to
(what else?) write a Python script.
I know this is a hack so if you know of a better way of doing this, let
me know.
</p>
<p>The script parses my <code>~/.ssh/config</code> file to find
the ssh keyfile and address for host 2, uses scp to copy the ssh keyfile
to host 1, then runs the <code>ssh host1 scp ...</code> command with the
appropriate options filled in. The script captures all of the ssh
options for host 2 and passes them on the command line to <code>scp</code>
via the <code>-o</code> command-line option. Note, I only tested this
to set the <code>User</code> option– I don't know if all ssh options
will work.
</p>
<p><em>Warning: the script disables the StrictHostKeyChecking SSH option,
so you are more vunerable to a man-in-the-middle attack.</em></p>
<p><em>Update 2010-02-16:</em> I've found there is already a SSH config
file parser in the <a href="http://www.lag.net/paramiko/">paramiko</a>
library. The source can be viewed
<a href="http://github.com/robey/paramiko/blob/master/paramiko/config.py">
on github</a>.
</p>
<p><em>Update 2010-05-04:</em> I modified my code to use the paramiko
library and also allow command line options to be passed directly to
the scp command.
The latest code is available in my github repository
<a href="http://github.com/saltycrane/remote-tools">remote-tools</a>.
</p>
<pre class="python">import itertools
import os
import re
import sys
SSH_CONFIG_FILE = '/home/eliot/.ssh/config'
def main():
host1, path1 = sys.argv[1].split(':', 1)
host2, path2 = sys.argv[2].split(':', 1)
o = get_ssh_options(host2)
keyfile_remote = '/tmp/%s' % os.path.basename(o['identityfile'])
ssh_options = ' -o'.join(['='.join([k, v]) for k, v in o.iteritems()
if k != 'hostname' and k != 'identityfile'])
run('scp %s %s:%s' % (o['identityfile'], host1, keyfile_remote))
run('ssh %s scp -p -i %s -oStrictHostKeyChecking=no -o%s %s %s:%s' % (
host1, keyfile_remote, ssh_options, path1, o['hostname'], path2))
def get_ssh_options(host):
"""Parse ~/.ssh/config file and return a dict of ssh options for host
Note: dict keys are all lowercase
"""
def remove_comment(line):
return re.sub(r'#.*$', '', line)
def get_value(line, key_arg):
m = re.search(r'^\s*%s\s+(.+)\s*$' % key_arg, line, re.I)
if m:
return m.group(1)
else:
return ''
def not_the_host(line):
return get_value(line, 'Host') != host
def not_a_host(line):
return get_value(line, 'Host') == ''
lines = [line.strip() for line in file(SSH_CONFIG_FILE)]
comments_removed = [remove_comment(line) for line in lines]
blanks_removed = [line for line in comments_removed if line]
top_removed = list(itertools.dropwhile(not_the_host, blanks_removed))[1:]
goodpart = itertools.takewhile(not_a_host, top_removed)
return dict([line.lower().split(None, 1) for line in goodpart])
def run(cmd):
print cmd
os.system(cmd)
if __name__ == '__main__':
main()</pre>
<p>Here is an example <code>~/.ssh/config</code> file:</p>
<pre>Host testhost1
User root
Hostname 48.879.24.567
IdentityFile /home/eliot/.ssh/test_keyfile
Host testhost2
User root
Hostname 56.384.58.212
IdentityFile /home/eliot/.ssh/test_keyfile</pre>
<p>Here is an example run. It copies <code>/tmp/testfile</code> from
<code>testhost1</code> to the same path on <code>testhost2</code>.</p>
<pre>python scp_r2r.py testhost1:/tmp/testfile testhost2:/tmp/testfile</pre>
<p>Here is the console output:</p>
<pre style="overflow: auto">scp /home/eliot/.ssh/test_keyfile testhost1:/tmp/test_keyfile
test_keyfile 100% 1674 1.6KB/s 00:00
ssh testhost1 scp -p -i /tmp/test_keyfile -oStrictHostKeyChecking=no -ouser=root /tmp/testfile 56.384.58.212:/tmp/testfile</pre>
<p>One inconvenience is that it doesn't show the progress for the main transfer.
If anyone knows how I can fix this, please let me know.</p>
Iterating over lines in multiple Linux log files using Python
2010-01-23T00:07:51-08:00https://www.saltycrane.com/blog/2010/01/iterating-over-lines-multiple-linux-log-files-using-python/<p>I needed to parse through my Nginx log files to debug a problem. However, the logs
are separated into many files, most of them are gzipped, and I wanted the
ordering within the files reversed. So I abstracted the logic to handle this
into a function. Now I can pass a glob pattern such as <em>/var/log/nginx/cache.log*</em>
to my function,
and iterate over each line in all the files as if they were one file. Here is my
function. Let me know if there is a better way to do this.</p>
<p><em>Update 2010-02-24:</em>To handle multiple log files on a remote host, see
my <a href="http://github.com/saltycrane/remote-tools/blob/master/multilog.py">script
on github</a>.</p>
<pre class="python">import glob
import gzip
import re
def get_lines(log_glob):
"""Return an iterator of each line in all files matching log_glob.
Lines are sorted most recent first.
Files are sorted by the integer in the suffix of the log filename.
Suffix may be one of the following:
.X (where X is an integer)
.X.gz (where X is an integer)
If the filename does not end in either suffix, it is treated as if X=0
"""
def sort_by_suffix(a, b):
def get_suffix(fname):
m = re.search(r'.(?:\.(\d+))?(?:\.gz)?$', fname)
if m.lastindex:
suf = int(m.group(1))
else:
suf = 0
return suf
return get_suffix(a) - get_suffix(b)
filelist = glob.glob(log_glob)
for filename in sorted(filelist, sort_by_suffix):
if filename.endswith('.gz'):
fh = gzip.open(filename)
else:
fh = open(filename)
for line in reversed(fh.readlines()):
yield line
fh.close()</pre>
<p>Here is an example run on my machine. It prints the first 15 characters of
every 1000th line of all my syslog files.</p>
<pre class="python">for i, line in enumerate(get_lines('/var/log/syslog*')):
if not i % 1000:
print line[:15]</pre>
<p>File listing:</p>
<pre>$ ls -l /var/log/syslog*
-rw-r----- 1 syslog adm 169965 2010 01/23 00:18 /var/log/syslog
-rw-r----- 1 syslog adm 350334 2010 01/22 08:03 /var/log/syslog.1
-rw-r----- 1 syslog adm 18078 2010 01/21 07:49 /var/log/syslog.2.gz
-rw-r----- 1 syslog adm 16700 2010 01/20 07:43 /var/log/syslog.3.gz
-rw-r----- 1 syslog adm 18197 2010 01/19 07:52 /var/log/syslog.4.gz
-rw-r----- 1 syslog adm 15737 2010 01/18 07:45 /var/log/syslog.5.gz
-rw-r----- 1 syslog adm 16157 2010 01/17 07:54 /var/log/syslog.6.gz
-rw-r----- 1 syslog adm 20285 2010 01/16 07:48 /var/log/syslog.7.gz</pre>
<p>Results:</p>
<pre>Jan 22 23:57:01
Jan 22 14:09:01
Jan 22 03:51:01
Jan 21 17:35:01
Jan 21 14:37:33
Jan 21 08:35:01
Jan 20 22:12:01
Jan 20 11:56:01
Jan 20 01:41:01
Jan 19 15:18:01
Jan 19 04:53:01
Jan 18 18:35:01
Jan 18 08:40:01
Jan 17 22:10:01
Jan 17 11:32:01
Jan 17 01:05:01
Jan 16 14:27:01
Jan 16 04:01:01
Jan 15 17:25:01
Jan 15 08:50:01</pre>
Notes on working with files and directories in Python
2009-09-18T16:53:07-07:00https://www.saltycrane.com/blog/2009/09/notes-working-files-and-directories-python/Documentation:<br>
<ul>
<li><a href="http://docs.python.org/library/os.html#files-and-directories">
http://docs.python.org/library/os.html#files-and-directories</a></li>
<li><a href="http://docs.python.org/library/os.path.html">
http://docs.python.org/library/os.path.html</a></li>
<li><a href="http://docs.python.org/library/shutil.html">
http://docs.python.org/library/shutil.html</a></li>
<li><a href="http://docs.python.org/library/glob.html">
http://docs.python.org/library/glob.html</a></li>
<li><a href="http://docs.python.org/library/tarfile.html">
http://docs.python.org/library/tarfile.html</a></li>
</ul>
<h4>How to list files in a directory</h4>
<p>See my separate post: <a href="/blog/2010/04/how-list-contents-directory-python/">
How to list the contents of a directory with Python</a></p>
<h4>How to rename a file: os.rename</h4>
<p>Documentation: <a href="http://docs.python.org/library/os.html#os.rename">
http://docs.python.org/library/os.html#os.rename</a></p>
<pre class="python">import os
os.rename("/tmp/oldname", "/tmp/newname")</pre>
<h4>How to imitate mkdir -p</h4>
<pre class="python">import os
if not os.path.exists(directory):
os.makedirs(directory)</pre>
<h4>How to imitate cp -r (except copy only files including hidden dotfiles)</h4>
<p>What didn't work for my purpose:
<ul>
<li><a href="http://docs.python.org/library/shutil.html#shutil.copytree"><code>shutil.copytree</code></a></li>
<li><a href="http://serverfault.com/questions/3154/recursively-copying-hidden-files-linux">some other bash techniques</a></li>
</ul>
</p>
<pre class="python">import os
def _copy_dash_r_filesonly(src, dst):
"""Like "cp -r src/* dst" but copy files only (don't include directories)
(and include hidden dotfiles also)
"""
for (path, dirs, files) in os.walk(src):
for filename in files:
srcfilepath = os.path.join(path, filename)
dstfilepath = os.path.join(dst, os.path.relpath(srcfilepath, src))
dstdir = os.path.dirname(dstfilepath)
if not os.path.exists(dstdir):
run('mkdir -p %s' % dstdir)
run('cp -f %s %s' % (srcfilepath, dstfilepath))</pre>
Working with files and directories in Python
2008-04-14T14:52:00-07:00https://www.saltycrane.com/blog/2008/04/working-with-files-and-directories-in/<p>I often have a difficult time finding the appropriate functions to work with files
and directories in Python. I think one reason is because the Library Reference
puts seemingly related functions in two different places: <em>11 File and Directory
Access</em> and <em>14.1.4 Files and Directories</em>.</p>
<p>Section 11, <em>File and Directory Access</em> contains documentation
for useful functions such as
<code>os.path.exists</code> which checks if a path exists,
<code>glob.glob</code> which is useful for matching filenames using
the Unix-style <code>*</code> and <code>?</code> wildcards, and
<code>shutil.copy</code> which is similary to the Unix <code>cp</code>
command.</p>
<p>This section includes a total of 11 subsections for modules related
to file and directory access, but it does not contain basic commands
such as <code>os.chdir</code>, <code>os.listdir</code>, or <code>os.rename</code>.
These functions are documented in Section 14.1.4, <em>Files and
Directories</em>, instead.</p>
<p>Here is a summary of some of the functions I find useful. Check the
documentation for more complete and detailed information.</p>
<strong><a href="http://www.python.org/doc/2.5.2/lib/filesys.html">
11 File and Directory Access</a></strong><br />
<ul>
<li><a href="http://www.python.org/doc/2.5.2/lib/module-os.path.html">
<code>os.path</code></a> module:
<ul>
<li><code>exists</code> - checks if a path or file exists<br />
Example: <pre>import os.path
print os.path.exists("c:/Windows")</pre>
Results: <pre>True</pre>
</li>
<li><code>isfile</code> and <code>isdir</code> - test if the path is a
file or directory, respectively.<br />
Example: <pre>import os.path
print os.path.isfile("c:/Windows")
print os.path.isdir("c:/Windows")</pre>
Results: <pre>False
True</pre>
</li>
<li><code>getmtime</code> - returns the modification time of a path<br />
Example: <pre>import os.path
import time
mtime = os.path.getmtime("c:/Windows")
print time.gmtime(mtime)</pre>
Results: <pre>(2008, 4, 2, 15, 58, 39, 2, 93, 0)</pre>
</li>
</ul>
</li>
<li><a href="http://www.python.org/doc/2.5.2/lib/module-glob.html">
<code>glob</code></a> module:
<ul>
<li><code>glob</code>: returns a list of paths matching a Unix-style
glob pattern.<br />
Example: <pre>import glob
print glob.glob("c:/windows/*.bmp")</pre>
Results: <pre>['c:/windows\\Blue Lace 16.bmp', 'c:/windows\\Coffee Bean.bmp', 'c:/windows\\default.bmp', 'c:/windows\\FeatherTexture.bmp', 'c:/windows\\Gone Fishing.bmp', 'c:/windows\\Greenstone.bmp', 'c:/windows\\Prairie Wind.bmp', 'c:/windows\\Rhododendron.bmp', 'c:/windows\\River Sumida.bmp', 'c:/windows\\Santa Fe Stucco.bmp', 'c:/windows\\Soap Bubbles.bmp', 'c:/windows\\winnt.bmp', 'c:/windows\\winnt256.bmp', 'c:/windows\\Zapotec.bmp']</pre>
</li>
</ul>
</li>
<li><a href="http://www.python.org/doc/2.5.2/lib/module-shutil.html">
<code>shutil</code></a> module:
<ul>
<li><code>copy</code> - similar to Unix <code>cp</code></li>
<li><code>copy2</code> - similar to Unix <code>cp -p</code></li>
<li><code>copytree</code> - similar to Unix <code>cp -r</code></li>
<li><code>rmtree</code> - similar to Unix <code>rm -r</code></li>
</ul>
</li>
</ul>
<strong><a href="http://www.python.org/doc/2.5.2/lib/allos.html">
14 Generic Operating System Services</a> - >
<a href="http://www.python.org/doc/2.5.2/lib/module-os.html">
14.1 os --- Miscellaneous operating system interfaces</a> ->
<a href="http://www.python.org/doc/2.5.2/lib/os-file-dir.html">
14.1.4 Files and Directories</a></strong><br />
<ul>
<li><code>chdir</code> - change the current working directory</li>
<li><code>getcwd</code> - return a string representing the current
working directory</li>
<li><code>listdir</code> - return a list of the names of the entries
in the directory.</li>
<li><code>makedir</code> - create a directory</li>
<li><code>remove</code> - remove a file (this is identical to
<code>unlink</code>)</li>
<li><code>rename</code> - rename the file or directory</li>
<li><code>walk</code> - walks a directory tree (see my
<a href="http://www.saltycrane.com/blog/2007/03/python-oswalk-example/">
os.walk example</a>)</li>
</ul>
How to get current working directory in Python
2007-11-02T11:41:00-07:00https://www.saltycrane.com/blog/2007/11/how-to-get-current-working-directory-in/I always forget how to get the current working directory so here it is, mainly for my own reference.
<pre>import os
os.getcwd()
'C:\\Python25'</pre>
backing up with rsync
2007-08-31T13:55:00-07:00https://www.saltycrane.com/blog/2007/08/backing-up-with-rsync/<p>
Here is a python script using rsync to backup my Users directory in
Vista to an external hard drive.</p>
<pre class="python">import os
cmd = "rsync -avz --exclude '/AppData/'" + \
"/cygdrive/c/Users/saltycrane" + \
"/cygdrive/f/backup/Users"
os.system(cmd)</pre>
<h4 id="why-not-proprietary-backup">Why not to use the backup program that comes with your external hard drive:</h4>
<blockquote>Do not, whatever you do, feed your valuable data to a program that is going to save it in a file format that can only be read by that program, or by
that kind of computer. Because when the program can’t or the computer can’t,
<em>you’re out of options</em>.
--<a href="">Tim Bray</a>, <a href="http://www.tbray.org/ongoing/When/200x/2006/01/31/Data-Protection#p-5"><em>Protecting Your Data</em></a>
</blockquote>
<h4 id="why-rsync">Why to use rsync:</h4>
<p><kbd>rsync -essh -rtpvz</kbd> rocks.
--<a href="http://diveintomark.org/">Mark Pilgrim</a>,
<a href="http://diveintomark.org/archives/2004/05/01/essentials"><em>Essentials</em></a>
</p>
<h4 id="trailing-slashes">On trailing slashes</h4>
<p>from the rsync man page:
<blockquote> A trailing slash on the source changes this behavior to avoid creating an addi‐
tional directory level at the destination. You can think of a trailing / on a
source as meaning “copy the contents of this directory” as opposed to “copy the
directory by name”, but in both cases the attributes of the containing directory
are transferred to the containing directory on the destination. In other words,
each of the following commands copies the files in the same way, including their
setting of the attributes of /dest/foo:<br>
<br>
rsync -av /src/foo /dest<br>
rsync -av /src/foo/ /dest/foo<br>
</blockquote></p>
<h4 id="rsync-sudo">Use rsync with sudo on the remote host <em>(added 2011-09-07)</em></h4>
<p>Use the <code>--rsync-path</code> option. e.g.</p>
<pre class="console">$ rsync -avz --delete --rsync-path="sudo rsync" /tmp/something myhost:/some/path </pre>
Python os.walk example
2007-03-15T11:37:00-07:00https://www.saltycrane.com/blog/2007/03/python-oswalk-example/Here is a simple os.walk() example which walks your directory tree and returns the path, a list of directories, and a list of files:<br />
<pre class="python">import os
path = "c:\\python25"
i = 0
for (path, dirs, files) in os.walk(path):
print path
print dirs
print files
print "----"
i += 1
if i >= 4:
break</pre>
Here are the results:<br />
<pre>c:/python25
['DLLs', 'Doc', 'include', 'Lib', 'libs', 'tcl', 'Tools']
['LICENSE.txt', 'NEWS.txt', 'pylupdate4.exe', 'pyrcc4.exe', 'python.exe', 'pythonw.exe', 'pyuic4.bat', 'README.txt', 'temp.py', 'w9xpopen.exe']
----
c:/python25\DLLs
[]
['bz2.pyd', 'py.ico', 'pyc.ico', 'pyexpat.pyd', 'select.pyd', 'sqlite3.dll', 'tcl84.dll', 'tclpip84.dll', 'tk84.dll', 'unicodedata.pyd', 'winsound.pyd', '_bsddb.pyd', '_ctypes.pyd', '_ctypes_test.pyd', '_elementtree.pyd', '_hashlib.pyd', '_msi.pyd', '_socket.pyd', '_sqlite3.pyd', '_ssl.pyd', '_testcapi.pyd', '_tkinter.pyd']
----
c:/python25\Doc
[]
['Python25.chm']
----
c:/python25\include
[]
['abstract.h', 'asdl.h', 'ast.h', 'bitset.h', 'boolobject.h', 'bufferobject.h', 'cellobject.h', 'ceval.h', 'classobject.h', 'cobject.h', 'code.h', 'codecs.h', 'compile.h', 'complexobject.h', 'cStringIO.h', 'datetime.h', 'descrobject.h', 'dictobject.h', 'enumobject.h', 'errcode.h', 'eval.h', 'fileobject.h', 'floatobject.h', 'frameobject.h', 'funcobject.h', 'genobject.h', 'graminit.h', 'grammar.h', 'import.h', 'intobject.h', 'intrcheck.h', 'iterobject.h', 'listobject.h', 'longintrepr.h', 'longobject.h', 'marshal.h', 'metagrammar.h', 'methodobject.h', 'modsupport.h', 'moduleobject.h', 'node.h', 'object.h', 'objimpl.h', 'opcode.h', 'osdefs.h', 'parsetok.h', 'patchlevel.h', 'pgen.h', 'pgenheaders.h', 'pyarena.h', 'pyconfig.h', 'pydebug.h', 'pyerrors.h', 'pyexpat.h', 'pyfpe.h', 'pygetopt.h', 'pymactoolbox.h', 'pymem.h', 'pyport.h', 'pystate.h', 'pystrtod.h', 'Python-ast.h', 'Python.h', 'pythonrun.h', 'pythread.h', 'py_curses.h', 'rangeobject.h', 'setobject.h', 'sliceobject.h', 'stringobject.h', 'structmember.h', 'structseq.h', 'symtable.h', 'sysmodule.h', 'timefuncs.h', 'token.h', 'traceback.h', 'tupleobject.h', 'ucnhash.h', 'unicodeobject.h', 'weakrefobject.h']
----</pre>