Thats not an option when you want to do something like reinforcement learning with lock free updates. In that case, the networks are small enough that you want

Thats not an option when you want to do something like reinforcement learning with lock free updates. In that case, the networks are small enough that you want to use the CPU, but learning sensitive enough that you don't want multiple copies of the network getting out of sync. Then you absolutely need multiple cores sharing memory.