5 ms·

This is really neat! I'm in a similar situation (Late 2013 Retina MacBook Pro 13") and have been doing some research on deep learning and neural networks. Has a

by Uehreka 10y ago

This is really neat! I'm in a similar situation (Late 2013 Retina MacBook Pro 13") and have been doing some research on deep learning and neural networks. Has anyone had success using eGPUs with things like CUDA and CUDNN? Is TB2's lower bandwidth a bottleneck that would keep you from using a really beefy eGPU for these purposes?

rincebrain 10y ago
TB2 is akin to using one of those x16 slots that's wired to x4 (or worse, since if you're using a Gen3 x16 GPU, you'd be running at Gen2 x4, which would be an eightfold reduction). It'll depend on your workload, but generally, it is going to be visibly slower than the "baseline" performance for any high-utilization workload.
- AlphaSite 10y ago
  Typically x8 can be used with near zero performance loss in many/most workloads, and withGen3 x4 should be fine again.
masklinn 10y ago
> Is TB2's lower bandwidth a bottleneck that would keep you from using a really beefy eGPU for these purposes? What limits you is the power input and cooling capabilities of your TB2 box. A 1080 has a 180W TDP (before O/C), so you need to feed it up to 180We and dissipate that much heat, a Titan X increases that to 250W. TFA went with a 1050 because it only has a 75W TDP, and that could be achieved with their enclosure (by swapping out the power brick, the "out of the box" version only handles 30W TDP). After that, it's mostly a matter of feeding the GPU enough stuff to do, giving it a small number of expensive tasks will utilise the limited bandwidth better than the reverse, which is why the efficiency of eGPU (TB2, but also TB3) increases as you increase rendering quality and the like (compared to internal) e.g. you might get 70% of internal at low settings and 90% at high.
Aeolos 10y ago
When using a single GPU, as is the case here, you will not be able to saturate a TB2 slot in any reasonable deep learning workload. For multiple GPUs, sure, PCIe performance matters, but for single GPUs it is not a bottleneck for any practical scenario I can think of. Right now, I am working with large volumetric datasets (think 100+GB) and even then I am only seeing ~200MB/s peak transfers, which are well within the capabilities of TB2. In my experience, large datasets are bottlenecked by the hard drive, which is not a problem for modern rMBPs. Edit: I am using a 2.5GB/s NVMe SSD, so the 200MB/s is the raw bandwidth used when mass-evaluating batches on the GPU. For training, I am seeing around 70-80MB/s sustained.
xenadu02 10y ago
The performance penalty for paging in data from system RAM is already high enough that most games are highly optimized to keep the necessary data in GPU RAM. Paging in random bits of data is more latency-sensitive than bandwidth-constrained. I'm not surprised that the penalty is ~25%. Using the eGPU to drive the internal panel means copying the frame buffer back to the internal GPU which is where the latency and performance penalty come from; I'm not sure how it is implemented, it might actually be copying twice (once to system RAM, then again to the discrete GPU RAM). Having more lanes over the TB connection would probably help here, though it might not eliminate the latency.
m_eiman 10y ago
The 13" would benefit a lot more than TFA I suppose, since then we'd upgrade from Intel GPU to "serious" GPU, and not just dGPU to eGPU?
masklinn 10y ago
Thinking about it more, addition to my other comment re. power input: above 75W graphics card don't draw power from just the PCIe slot (as it's limited to, you guessed it, 75W), so you end up needing a "PC" PSU with 6-pin and/or 8-pin PCIe Power Connectors to provide additional power to the GPU (which you'll still have to dissipate).

by Uehreka 10y ago

rincebrain 10y ago

TB2 is akin to using one of those x16 slots that's wired to x4 (or worse, since if you're using a Gen3 x16 GPU, you'd be running at Gen2 x4, which would be an eightfold reduction). It'll depend on your workload, but generally, it is going to be visibly slower than the "baseline" performance for any high-utilization workload.

AlphaSite 10y ago
Typically x8 can be used with near zero performance loss in many/most workloads, and withGen3 x4 should be fine again.

masklinn 10y ago

> Is TB2's lower bandwidth a bottleneck that would keep you from using a really beefy eGPU for these purposes? What limits you is the power input and cooling capabilities of your TB2 box. A 1080 has a 180W TDP (before O/C), so you need to feed it up to 180We and dissipate that much heat, a Titan X increases that to 250W. TFA went with a 1050 because it only has a 75W TDP, and that could be achieved with their enclosure (by swapping out the power brick, the "out of the box" version only handles 30W TDP). After that, it's mostly a matter of feeding the GPU enough stuff to do, giving it a small number of expensive tasks will utilise the limited bandwidth better than the reverse, which is why the efficiency of eGPU (TB2, but also TB3) increases as you increase rendering quality and the like (compared to internal) e.g. you might get 70% of internal at low settings and 90% at high.

Aeolos 10y ago

When using a single GPU, as is the case here, you will not be able to saturate a TB2 slot in any reasonable deep learning workload. For multiple GPUs, sure, PCIe performance matters, but for single GPUs it is not a bottleneck for any practical scenario I can think of. Right now, I am working with large volumetric datasets (think 100+GB) and even then I am only seeing ~200MB/s peak transfers, which are well within the capabilities of TB2. In my experience, large datasets are bottlenecked by the hard drive, which is not a problem for modern rMBPs. Edit: I am using a 2.5GB/s NVMe SSD, so the 200MB/s is the raw bandwidth used when mass-evaluating batches on the GPU. For training, I am seeing around 70-80MB/s sustained.

xenadu02 10y ago

The performance penalty for paging in data from system RAM is already high enough that most games are highly optimized to keep the necessary data in GPU RAM. Paging in random bits of data is more latency-sensitive than bandwidth-constrained. I'm not surprised that the penalty is ~25%. Using the eGPU to drive the internal panel means copying the frame buffer back to the internal GPU which is where the latency and performance penalty come from; I'm not sure how it is implemented, it might actually be copying twice (once to system RAM, then again to the discrete GPU RAM). Having more lanes over the TB connection would probably help here, though it might not eliminate the latency.

m_eiman 10y ago

The 13" would benefit a lot more than TFA I suppose, since then we'd upgrade from Intel GPU to "serious" GPU, and not just dGPU to eGPU?

masklinn 10y ago

Thinking about it more, addition to my other comment re. power input: above 75W graphics card don't draw power from just the PCIe slot (as it's limited to, you guessed it, 75W), so you end up needing a "PC" PSU with 6-pin and/or 8-pin PCIe Power Connectors to provide additional power to the GPU (which you'll still have to dissipate).