4 ms·

This false dichotomy between colo and AWS is just making me exhausted 'cos I have been repeating this for so long: just rent a dedicated server. There surely ar

by chx 8y ago

This false dichotomy between colo and AWS is just making me exhausted 'cos I have been repeating this for so long: just rent a dedicated server. There surely are some cases where colo is the best choice but as the years, now a decade pass since I have tried to spread this, it makes less and less sense every year -- and it never did much in the first place. Maybe if you have several racks worth of equipment? I am not familiar with that size.

deleted 8y ago
[deleted]
mippie_moe 8y ago
Lambda Labs engineer here. Here's what we're trying to argue: Many benefits of running infrastructure in the cloud are lost for offline batch processing jobs. Training machine learning models doesn't require low response times, high availability, geographic proximity to clients, etc. Yet, with cloud, you're paying for all this extra infrastructure. The main benefits of cloud for tasks with machine learning type workloads are cost saving (if utilization is low) and no electrical set-up. On the other hand, cloud is extremely expensive for groups that require high base levels of GPU compute. The article is arguing that such groups can save a huge amount of money by moving infrastructure on-prem.
- chx 8y ago
  I feel I am talking to a brick wall. This often happens and this is what exhausts me. You are still arguing between cloud and colo / on-prem (even ignoring that on-prem and colo is very different but w/e) and what I am saying is that there is a third option neither the article nor your reply even acknowledges.
icelancer 8y ago
I have been hard-pressed to find Threadripper 2990wx dedicated servers with 1080ti/2080ti GPUs in them for rental, much less at a reasonable cost. Colo/on-prem was our only option for our mega transcoding server.
- chx 8y ago
  That's not a server CPU. You'd need to look for a dual EPYC 7371 server for similar performance -- 2990WX is 3-4.2GHZ 32 cores, 7371 is 3.1-3.8GHz 16 cores. I'd email webnx sales, they have some EPYC servers although not this very new CPU but they were always very flexible. It might be more economical if you front the price of the CPUs but the 7371 is very cheap for what it is, the two should be around $3k (I saw them as a special order on gamepc for 1600, the site name is very ironic because the monsters they sell are more workstation / server than gaming pc). I am not affiliated, just a very satisfied customer, I worked with them when I was the lead architect of a US Top 100 (per Quantcast) site. The 1080 Ti is certainly not a problem for them.
  - deleted 8y ago
    [deleted]

by chx 8y ago

deleted 8y ago

[deleted]

mippie_moe 8y ago

Lambda Labs engineer here. Here's what we're trying to argue: Many benefits of running infrastructure in the cloud are lost for offline batch processing jobs. Training machine learning models doesn't require low response times, high availability, geographic proximity to clients, etc. Yet, with cloud, you're paying for all this extra infrastructure. The main benefits of cloud for tasks with machine learning type workloads are cost saving (if utilization is low) and no electrical set-up. On the other hand, cloud is extremely expensive for groups that require high base levels of GPU compute. The article is arguing that such groups can save a huge amount of money by moving infrastructure on-prem.

chx 8y ago
I feel I am talking to a brick wall. This often happens and this is what exhausts me. You are still arguing between cloud and colo / on-prem (even ignoring that on-prem and colo is very different but w/e) and what I am saying is that there is a third option neither the article nor your reply even acknowledges.

icelancer 8y ago

I have been hard-pressed to find Threadripper 2990wx dedicated servers with 1080ti/2080ti GPUs in them for rental, much less at a reasonable cost. Colo/on-prem was our only option for our mega transcoding server.

chx 8y ago
That's not a server CPU. You'd need to look for a dual EPYC 7371 server for similar performance -- 2990WX is 3-4.2GHZ 32 cores, 7371 is 3.1-3.8GHz 16 cores. I'd email webnx sales, they have some EPYC servers although not this very new CPU but they were always very flexible. It might be more economical if you front the price of the CPUs but the 7371 is very cheap for what it is, the two should be around $3k (I saw them as a special order on gamepc for 1600, the site name is very ironic because the monsters they sell are more workstation / server than gaming pc). I am not affiliated, just a very satisfied customer, I worked with them when I was the lead architect of a US Top 100 (per Quantcast) site. The 1080 Ti is certainly not a problem for them.
- deleted 8y ago
  [deleted]