14 ms·

For RTX cards, water cooling probably is a must nowadays. Had a lot of cooling issues with my 3 2080Ti FE. Although managed to get it stablize at 85C, it is not

by liuliu 8y ago

For RTX cards, water cooling probably is a must nowadays. Had a lot of cooling issues with my 3 2080Ti FE. Although managed to get it stablize at 85C, it is not ideal. Also, getting as much as PCIe lanes is not a bad idea. The RTX cards only supports 2way NvLink, communication has to go through PCIe lanes between some. I am thinking about the next build with a motherboard like https://www.asrockrack.com/general/productdetail.asp?Model=EPYCD8-2T#Specifications https://www.asrockrack.com/general/productdetail.asp?Model=E... and Epyc 7401P. You have full 16 lanes per GPU and some room to spare for M2 SSDs. The 7401P also has a reasonable price between TR2950X and 2990WX, not sure why don’t more people have this setup for their workstations.

ericd 8y ago
Thanks for sharing. I've been planning on building a deep learning box with 4x2080's, but I've been concerned about the loss of the blower style coolers in such a dense configuration. Have you seen any good 2080 Ti's with blowers? Alternatively, I might just go for 2 Titans. The larger memory via 2 way NVLink and faster tensor cores might outweigh the loss of FP32 compute for my purposes. As for the Epyc, I've seen more people going for the TR2. Seems like the clock on the 7401P is lower, maybe that's why? Or maybe people are just less familiar with dealing with server hardware and are worried that they don't know what they don't know.
- sseveran 8y ago
  I have a box with 4 2080 ti's that are cooled with blowers and it has not been a problem. Temp's max out at 87C under full load.
  - ericd 8y ago
    Ah cool. What brand/model? Also, I heard somewhere that they start throttling at 80C, have you seen that at all?
    - sseveran 8y ago
      I believe they are evgas. I have not seen throttling.
- liuliu 8y ago
  Yeah. Frequency is higher, but not sure if you can overclock it. The biggest problem I saw on these EPYC chips (potentially) is that the PCIe lanes spread across 4 dies. So the performance of data going through host would be interesting to observe. This is a setup with some risks, so would like to hear more from the crowd the feasibility.
  - ericd 8y ago
    Ah yeah, the higher die count TR's had some interesting performance characteristics, according to Anandtech. One surprising result was that the inter-die communication silicon used more power than the cores themselves sometimes, IIRC. I'd probably check out that article, since I'm guessing some of it applies to the similar core-count Epycs.
rmac3 8y ago
Do you think water cooling is necessary for a dual RTX 2070 setup?
- liuliu 8y ago
  No. Two slots away is plenty.

by liuliu 8y ago

ericd 8y ago

Thanks for sharing. I've been planning on building a deep learning box with 4x2080's, but I've been concerned about the loss of the blower style coolers in such a dense configuration. Have you seen any good 2080 Ti's with blowers? Alternatively, I might just go for 2 Titans. The larger memory via 2 way NVLink and faster tensor cores might outweigh the loss of FP32 compute for my purposes. As for the Epyc, I've seen more people going for the TR2. Seems like the clock on the 7401P is lower, maybe that's why? Or maybe people are just less familiar with dealing with server hardware and are worried that they don't know what they don't know.

sseveran 8y ago
I have a box with 4 2080 ti's that are cooled with blowers and it has not been a problem. Temp's max out at 87C under full load.
- ericd 8y ago
  Ah cool. What brand/model? Also, I heard somewhere that they start throttling at 80C, have you seen that at all?
  - sseveran 8y ago
    I believe they are evgas. I have not seen throttling.
liuliu 8y ago
Yeah. Frequency is higher, but not sure if you can overclock it. The biggest problem I saw on these EPYC chips (potentially) is that the PCIe lanes spread across 4 dies. So the performance of data going through host would be interesting to observe. This is a setup with some risks, so would like to hear more from the crowd the feasibility.
- ericd 8y ago
  Ah yeah, the higher die count TR's had some interesting performance characteristics, according to Anandtech. One surprising result was that the inter-die communication silicon used more power than the cores themselves sometimes, IIRC. I'd probably check out that article, since I'm guessing some of it applies to the similar core-count Epycs.

rmac3 8y ago

Do you think water cooling is necessary for a dual RTX 2070 setup?

liuliu 8y ago
No. Two slots away is plenty.