4 ms·
I would be curious to understand what the equivalent cluster solution would look like. A big selling point here is apparently that your researchers "don't have
by extr 5y ago
I would be curious to understand what the equivalent cluster solution would look like. A big selling point here is apparently that your researchers "don't have to understand the do's and don'ts of cluster scale computing", but if your budget is already in the millions, how big of an issue is that really? Is the barrier to entry for large scale NN training really that there aren't enough engineers with experience navigating the issues on commodity/cloud hardware?
Overall I guess I'm just confused at what scale of problems this is meant for. GPT-3 cost $5M to train, GPT-2 cost ~$50k. GPT-3 is way too big to fit into 40GB of SRAM, GPT-2 fits but isn't exactly bleeding edge. If this product is $2-5M, you could train GPT-2 40-100 times to make it cost effective, but now you're locked into the scale provided by this platform, and not all problems are GPT-2 sized. I'm not an expert so happy for someone else to chime in and correct me if I've gone wrong somewhere.
- blihp 5y agoGPT-2/3 is at the high end in terms of size. There's a whole universe of problems, and most of the market, below it. A great many business/research needs for at least the near future would fit into a CS-2 quite nicely.
- extr 5y agoYes I suppose you're right, GPT-2/3 aren't particularly representative of the "average" industry problem. Honestly what I'm jonesing for is an insider's take on the cost/benefit of a solution like this vs cloud, in terms of problem size, future flexibility, raw price, talent required, etc. Even just a ballpark "this is probably an order of magnitude more efficient vs cloud GPU compute, game changer" vs "we'd have to carefully consider and test performance, etc" But hey, that's the hard part right. Probably aren't many people total with experience with CS-1, let alone browsing HN :D