10 ms·

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. g

by harias 7y ago

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be easy. Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage.

icheishvili 7y ago
I don't think these are necessarily contradictory. With pytorch-transformers, you can use a full-blown BERT model like the best in the world. And yet, to make this novel and defensible, you would need to build on top of it and innovate significantly, which would require significant capital to achieve.
jjeaff 7y ago
>rare 10 minute peaks But is that really the use case here? I haven't worked in ML. But I'm not seeing where you are going to need to handle a 10 minute spike that requires a whole datacenter. A month's worth of a quad gpu instance on AWS could pay for a server with similar capacity in a few months of usage. And hardware is pretty resilient these days. Especially if you co-locate it in a datacenter that handles all the internet and power up time for you. And when something does go wrong, they offer "magic hands" service to go swap out hardware for you. Colocation is surprisingly cheap. As is leasing 'managed' equipment.
detaro 7y ago
> Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage. Do you need that for training workloads, and what percentage of a startups workload is training?
bsenftner 7y ago
I ran a small data cluster for years, the horsepower behind my startup. Other than the Chinese DDoS attacks, running the cluster was absolutely elementary. The idea that running a server or a band of servers is difficult is a bold faced lie. People have got to stop repeating the cloud propaganda.
sp527 7y ago
Training ML models usually doesn’t have the same uptime requirements as production systems. If your training goes down for a bit, it probably won’t make much difference to the underlying business, in most cases. That’s why the author found it glaringly obvious that it should be brought in-house. It’s often both the most costly and most “in-housable” compute work involved in these companies.

by harias 7y ago

icheishvili 7y ago

I don't think these are necessarily contradictory. With pytorch-transformers, you can use a full-blown BERT model like the best in the world. And yet, to make this novel and defensible, you would need to build on top of it and innovate significantly, which would require significant capital to achieve.

jjeaff 7y ago

>rare 10 minute peaks But is that really the use case here? I haven't worked in ML. But I'm not seeing where you are going to need to handle a 10 minute spike that requires a whole datacenter. A month's worth of a quad gpu instance on AWS could pay for a server with similar capacity in a few months of usage. And hardware is pretty resilient these days. Especially if you co-locate it in a datacenter that handles all the internet and power up time for you. And when something does go wrong, they offer "magic hands" service to go swap out hardware for you. Colocation is surprisingly cheap. As is leasing 'managed' equipment.

detaro 7y ago

> Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage. Do you need that for training workloads, and what percentage of a startups workload is training?

bsenftner 7y ago

I ran a small data cluster for years, the horsepower behind my startup. Other than the Chinese DDoS attacks, running the cluster was absolutely elementary. The idea that running a server or a band of servers is difficult is a bold faced lie. People have got to stop repeating the cloud propaganda.

sp527 7y ago

Training ML models usually doesn’t have the same uptime requirements as production systems. If your training goes down for a bit, it probably won’t make much difference to the underlying business, in most cases. That’s why the author found it glaringly obvious that it should be brought in-house. It’s often both the most costly and most “in-housable” compute work involved in these companies.