Check out the docs on the "multiprocessing" or "threading" module. In particular multiprocessing.Pool is handy for controlling the number of parallel things you

Check out the docs on the "multiprocessing" or "threading" module. In particular multiprocessing.Pool is handy for controlling the number of parallel things you have going on at once. # Assume we have functions GetUrls() that retrieves a list # of the urls we want to get, and Download(url) which # downloads the content of a url and sticks it in the # database. import multiprocessing pool = multiprocessing.Pool(processes=100) urls = GetUrls() pool.map(Download, urls) See: http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/threading.html http://docs.python.org/library/threading.html Also, if you're entering results into a database, the easiest way may be just to spawn multiple python processes from the command line.