3 ms·

I disagree that this is not an architecture for others. We are a research lab, and we're building a distributed cluster. Cheap infiniband. Lots of gtx1080Tis. I

I disagree that this is not an architecture for others. We are a research lab, and we're building a distributed cluster. Cheap infiniband. Lots of gtx1080Tis. It's not that expensive to have a 40-GPU cluster with 10 4U servers (about 100K Euro).

gm-conspiracy 9y ago
What mobo/CPU are you using? How much RAM in each server? Can you provide some more detailed specs? Thanks!
- jamesblonde 9y ago
  CPU is not that important. A motherboard that has PCIe 3.0 and 7 PCIx16 slots is ok for at least 3 GPUs. Each GPU takes typically two slots. Then, a >1400 Watts PSU. Here is a relatively cheap box that also has 8 disk slots (for hadoop): https://exxactcorp.com/index.php/solution/solu_detail/320 https://exxactcorp.com/index.php/solution/solu_detail/320
  - Retric 9y ago
    Some people are using water cooling setups to get 7 GPU's on a 7 PCI slot MB. You need to cut off the DVI that would otherwise occupy another slot, but overall it's not that time consuming and reduces your networking needs. EX: https://www.youtube.com/watch?v=9hsQmcSwGv0 https://www.youtube.com/watch?v=9hsQmcSwGv0
malux85 9y ago
Are you going to rent access to your cluster? :)
- jamesblonde 9y ago
  To users in Sweden at www.hops.site It provides Jupyter/Hadoop/Spark/Tensorflow and GPUs (as a resource in YARN).
jamesblonde 9y ago
You can see the potential scale-out you can get for Tensorflow here on this PR: https://github.com/tensorflow/tensorflow/issues/2916 https://github.com/tensorflow/tensorflow/issues/2916