5 ms·

> The philosophy here seems to be “if we build it, they’ll buy it.” Supposedly Cerebras is already profitable, so it's hardly a situation where they are buildi

> The philosophy here seems to be “if we build it, they’ll buy it.” Supposedly Cerebras is already profitable, so it's hardly a situation where they are building something and hoping people buy it eventually. > That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and with an inferencing pipeline to generate samples. Again, based on the companies representations, Cebebras transparently supports Pytorch and Tensorflow, only requiring a few lines of changed code. Source: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-... (Dr. Cutress's video on TechTechPotato is also good).

sillysaurusx 6y ago
I’ve learned to be skeptical of such claims. TPUs made that same claim, but it’s hard to use it for real work, since you don’t have access to the 300GB TPU host memory / cpu, where you infeed training examples. There is a very specific test for “supports pytorch / tendorflow”: show an MLPerf imagenet resnet benchmark. It’s impossible to fake that. If you come anywhere close to TPUs in tensorflow, people (like me) will leap: https://mlcommons.org/en/training-normal-06/ https://mlcommons.org/en/training-normal-06/ Till then, it’s a “proof, please” type of situation.
- sanxiyn 6y ago
  I agree. I have high hope for Graphcore. They stated "We will be participating in MLPerf in 2021, starting with the first training submission in the spring". https://www.graphcore.ai/posts/graphcore-sets-new-ai-performance-standards-with-mk2-ipu-systems https://www.graphcore.ai/posts/graphcore-sets-new-ai-perform... (I would ignore all benchmarks in that post except the sentence quoted above however.)
- algo_trader 6y ago
  Software will lag new hardware. Duh. The question is whether this is solvable (given a couple of master devs) Cerebras itself commented that DL is not their first use case. They mentioned fluid dynamics instead. (I can find the reference if someone is interested). Cerebras also has a weird memory/compute ratio, so very hard for some uses. Somewhat off-topic. If u have standard keras/TF training code, the GPU to TPU is a smooth-ish transition. No? A re-write will squeeze another 20% but not 200%?

sillysaurusx 6y ago

I’ve learned to be skeptical of such claims. TPUs made that same claim, but it’s hard to use it for real work, since you don’t have access to the 300GB TPU host memory / cpu, where you infeed training examples. There is a very specific test for “supports pytorch / tendorflow”: show an MLPerf imagenet resnet benchmark. It’s impossible to fake that. If you come anywhere close to TPUs in tensorflow, people (like me) will leap: https://mlcommons.org/en/training-normal-06/ https://mlcommons.org/en/training-normal-06/ Till then, it’s a “proof, please” type of situation.

sanxiyn 6y ago
I agree. I have high hope for Graphcore. They stated "We will be participating in MLPerf in 2021, starting with the first training submission in the spring". https://www.graphcore.ai/posts/graphcore-sets-new-ai-performance-standards-with-mk2-ipu-systems https://www.graphcore.ai/posts/graphcore-sets-new-ai-perform... (I would ignore all benchmarks in that post except the sentence quoted above however.)
algo_trader 6y ago
Software will lag new hardware. Duh. The question is whether this is solvable (given a couple of master devs) Cerebras itself commented that DL is not their first use case. They mentioned fluid dynamics instead. (I can find the reference if someone is interested). Cerebras also has a weird memory/compute ratio, so very hard for some uses. Somewhat off-topic. If u have standard keras/TF training code, the GPU to TPU is a smooth-ish transition. No? A re-write will squeeze another 20% but not 200%?