Python Threading

sax — Mon, 29 Jun 2009 21:14:31 +0000

In order to speed up some of my scripts I’ve implemented threading using python. Some scripts now run in 1/10th the time they used to as a result. Here I will try to illustrate some of the key points, hopefully it will be useful to anyone.

#!/usr/bin/python
import sys
import subprocess
from threading import Thread
 
class MyThread(Thread):
    def __init__(self,server,cmd):
        Thread.__init__(self)
        self.server = server
        self.cmd = cmd
        self.status = -1
    def run(self):
        proc = subprocess.Popen('ssh %s %s' % (self.server,self.cmd),
                       shell=True,
                       stdin=subprocess.PIPE,
                       stdout=subprocess.PIPE,
                       stderr=subprocess.STDOUT,
                       )
        self.stdout, self.stderr = proc.communicate()
 
servers = ['shell1.example.net','ssh2.mhost.com','examplebox.net']
command = 'uptime'
cmdlist = []
for srv in servers:
    run = MyThread(srv,command)
    cmdlist.append(run)
    run.start()
 
for c in cmdlist:
    c.join()
    print "### %s:n%s" % (c.server,c.stdout),
print "## Done"

In my example we loop through servers[] and for each server[] we run the specified command on them, in their own thread! 3 servers in the above example, not a big deal, what if you have 20 servers? 40 servers? huge speed improvement.

The script can be easily extended to say… read the servers list in from a configuration file, or accept the ‘command’ from the command line arguments, (argv[1:]) etc.

Enjoy.

Chris Haynie » python

Python Threading