I wrote previously about how to get stdout and stderr using
os.popen4. However, per the Python documentation, using the
subprocess module is preferred:
The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several other, older modules and functions, such as:
See the subprocess module documentation for more information.
Here is how to get stdout and stderr from a program using the subprocess module:
from subprocess import Popen, PIPE, STDOUT cmd = 'ls /etc/fstab /etc/non-existent-file' p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) output = p.stdout.read() print output
ls: cannot access /etc/non-existent-file: No such file or directory /etc/fstab
- How to use the bash shell with Python's subprocess module instead of /bin/sh — posted 2011-04-13
- How to capture stdout in real-time with Python — posted 2009-10-12
- How to use python and popen4 to capture stdout and stderr from a command — posted 2007-03-12
Here's a cleaner way:
from subprocess import Popen, PIPE cmd = 'blah' p = Popen(cmd, stdout=PIPE, stderr=PIPE) stdout, stderr = p.communicate()
That way you wait for the output until the process has actually completed.
Also, take a look at the Process objects in the cliutils package. (Disclaimer: I wrote it.) They let you do some cool things.
This looks like an interesting alternative. I just took my example from the Replacing os.popen* section in the docs.
There is error here: p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
Python says: NameError: name 'STDOUT' is not defined
If I use stderr=PIPE, it works.
This one is not good enough either: p = Popen(cmd, stdout=PIPE, stderr=PIPE)
It works fine with ls, rmdir etc. but with wget it seems to make errors whatever you do. But, the following seems to work fine with wget
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
Then, the following seems to work fine:
stdout, stderr = p.communicate()
You got "'STDOUT' is not defined" because you must either: from subprocess import STDOUT or you must refer to STDOUT as subprocess.STDOUT .
I'm surprised you had any success running ls and rmdir without shell=True. ls and rmdir are not programs, they are internal commands within the shell program.
wget, however, is a program. You can use shell=True, but you don't need to. Using shell=False (the default) will save you from having to start a new shell process (which is an expensive operation). When you submit a command with shell=True, you submit the whole command as one string. But when you submit the command with shell=False, you must issue the command as a list of strings, with the command name in the first element of the list, the first argument in the next list element, etc. For example:
wgproc = subprocess.Popen(['wget', '-r', '--tries=10', 'http://fly.srk.fer.hr/', '-o', 'log'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) (standardout, junk) = wgproc.communicate()
Hm. ls, rm, rmdir and others are programs. They live under /bin, just do a which ls.
And thanks for the useful post and comments :)
Thanks for the hint on subprocess.STDOUT.
Ha! Ron, you're an idiot.
How do you print the contents of stdout and stderr out in an easy to read fashion? if I do for line in stdout, it prints each character as separate line instead of printing each LINE on a separate line. Halp!
for line in p.stdout iterates over each line. See documentation here: https://docs.python.org/2/t... Make sure you're not calling
read() like I am doing in the blog post. Do this:
from subprocess import Popen, PIPE, STDOUT cmd = 'ls /etc/fstab /etc/non-existent-file' p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) for line in p.stdout: line = line.rstrip() print line
End ended up doing the (stdout, stderr ) = p.communicate() method, and then used the stdout.splitlines() function to get the output into a list of lines to loop through. that method worked for purposes.