Did you investigate the multiprocessing library? Also this kind of thing should be relatively light on the GIL if done correctly. The bzip2 module releases the

Did you investigate the multiprocessing library? Also this kind of thing should be relatively light on the GIL if done correctly. The bzip2 module releases the GIL (I assume?), as does file IO, which is most of the workload in your use case?