4 ms·

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8 https://www.youtube.com/watch?v=OF3JYEIsjH8 Se

by liuliu 7y ago

I've been playing with custom-built 2080 Ti workstation for a while: https://www.youtube.com/watch?v=OF3JYEIsjH8 https://www.youtube.com/watch?v=OF3JYEIsjH8 Several issues: 1. electricity bill is still an issue, I've been paying anywhere between $500 to $1000 per month for this workstation (always have something to train). 2. something with a decent memory size (Titan RTX and RTX 8000) cost way too much; 3. once you reached a point of 4-2080Ti-is-not-fast-enough, power management and connectivity setup would be a nightmare. Would love to know other people's opinions on the on-prem setup, especially whether a consumer-grade 10Ghe is enough for connectivity-wise.

eyegor 7y ago
10gbe will depend on the workload. In general, I'd assume it's fine because it takes a parallel raid setup to saturate. Upgrading to 100gbe is pretty unreasonable cost wise unless you buy network gear from a back alley van dealer. Although once you reach 4 2080ti, you ought to consider switching to a titanium grade psu and rewiring if you're in a 100-120v country. If you're feeling cheap, just steal the phases from two different circuits. Last I looked, most psu operate around 5% lower efficiency on 115 vs 230.
- liuliu 7y ago
  I've encountered some latency issues with allreduce on transformer models due to vocabulary sizes when communication have to cross PCIe lanes. Increasing batch size helps a lot, but low-latency & high-throughput is universally more helpful to lift these minor concerns (I don't really want to care about my batch size to improve allreduce performance). Hence worried not only throughput, but also latency on consumer-grade 10ghe equipments.
- xyzzyz 7y ago
  If you're feeling cheap, just steal the phases from two different circuits. I chukled, but more seriously, if you can't rewire your house to get a normal 240V circuit, you should not be fucking around with hacks like above.
m0zg 7y ago
>> $500 to $1000 per month How much is your electricity? I currently run 12 GPUs in my garage pretty much non-stop. 4 GPUs per machine, 3 machines. Each machine is about 1.2KW on average (I can tell because each machine is connected through its own rack UPS), or 13.2 cents per hour, or $95/mo. Which, IMO, is not bad at all. That's less than $300 per month for 12 GPUs.
- liuliu 7y ago
  Sorry, it is 2-month billing cycle. We have around 30 cents per kwh I think.

by liuliu 7y ago

eyegor 7y ago

10gbe will depend on the workload. In general, I'd assume it's fine because it takes a parallel raid setup to saturate. Upgrading to 100gbe is pretty unreasonable cost wise unless you buy network gear from a back alley van dealer. Although once you reach 4 2080ti, you ought to consider switching to a titanium grade psu and rewiring if you're in a 100-120v country. If you're feeling cheap, just steal the phases from two different circuits. Last I looked, most psu operate around 5% lower efficiency on 115 vs 230.

liuliu 7y ago
I've encountered some latency issues with allreduce on transformer models due to vocabulary sizes when communication have to cross PCIe lanes. Increasing batch size helps a lot, but low-latency & high-throughput is universally more helpful to lift these minor concerns (I don't really want to care about my batch size to improve allreduce performance). Hence worried not only throughput, but also latency on consumer-grade 10ghe equipments.
xyzzyz 7y ago
If you're feeling cheap, just steal the phases from two different circuits. I chukled, but more seriously, if you can't rewire your house to get a normal 240V circuit, you should not be fucking around with hacks like above.

m0zg 7y ago

>> $500 to $1000 per month How much is your electricity? I currently run 12 GPUs in my garage pretty much non-stop. 4 GPUs per machine, 3 machines. Each machine is about 1.2KW on average (I can tell because each machine is connected through its own rack UPS), or 13.2 cents per hour, or $95/mo. Which, IMO, is not bad at all. That's less than $300 per month for 12 GPUs.

liuliu 7y ago
Sorry, it is 2-month billing cycle. We have around 30 cents per kwh I think.