I haven't made a comparison with GPU vs CPU. On GPU, my epochs were averaging about 20 minutes to complete, regardless of how many I had running concurrently.

I haven't made a comparison with GPU vs CPU. On GPU, my epochs were averaging about 20 minutes to complete, regardless of how many I had running concurrently. There isn't any other hardware involved other than this one machine; I imagine there was some throttling involved, but I didn't notice it, really, but I think it's because my model isn't that complex. ------- When I decided to try to do this, I had about 40% GPU usage while training one epoch, and the CPU load was around 35%. When I added a parallel run, I was getting around 25% GPU load on the other process, and the CPU load went up about 10%. The third run, I was getting about 9% GPU load on the process and the CPU load went up by another 5%. ------- This isn't something that I plan on doing continuously either, but I was partly debugging some things and figured I could squeeze in some extra runs. ------- Thanks for the links to the distributed TensorFlow. I may eventually get around to doing that but at the moment I'm merely trying to validate an idea that I have at minimal cost, and happen to have this machine around practically doing nothing else.