4 ms·
I heard the eGPU gets throttled memory wise because there are less pcie lanes. does this have a big impact on ML performance?
by edhu2017 8y ago
I heard the eGPU gets throttled memory wise because there are less pcie lanes. does this have a big impact on ML performance?
- loser777 8y agoI would hope not; any reasonable implementation would try to keep as much data on-board or on-chip as possible.
- edhu2017 8y agoEdit: This thread is helpful. https://www.reddit.com/r/MachineLearning/comments/6tyyyt/d_dual_gpu_pcie_lane_bottleneck_for_machine/ https://www.reddit.com/r/MachineLearning/comments/6tyyyt/d_d...
- loser777 8y agoYes, but the bulk of bandwidth should be eaten up by intermediate results rather than the input data e.g., for imagenet, 224x224x3 tensors are tiny compared to the intermediate activations that have to be moved around (and saved during training). Hopefully you would also only copy your model over once.
- edhu2017 8y agoI see, but I think for areas like Reinforcement Learning or sequence models though eGPUs would be significantly slower since you have to constantly shuttle new data into the GPU from your computer.
- rememberlenny 8y agoIt gets 70-80% performance. I’m not sure about throttling. It’s definitely not 1-to-1 based on my desktop, but it’s so much faster than the CPU.
- chrischen 8y agoWhen connecting on Windows the eGPU seems to be using a PCIe x8 slot which may limit the GPU.