2 ms·
Sure. You only have to design a chip, design an assembly language, design a compiler, design the kernels, design a parallelization framework, design a server sy
by dragontamer 4y ago
Sure. You only have to design a chip, design an assembly language, design a compiler, design the kernels, design a parallelization framework, design a server system to load-balance tasks, and then rework the pytorch/tensorflow code to use your new faster custom primitives that no one else has.
-----
Except step 1: "design a chip", is already something on the order of hundreds-of-megabucks of investment
- baq 4y agooh absolutely. but they're already there, looks like.
- dragontamer 4y agoIf they were already there, they'd have ResNet 50 benchmarks to share. And the other tidbit is that it has to be better than the off the shelf NVidia chip (A100 today, and GH100 in a few months). I think Dojo might be faster than A100, but GH100 will be 1.5 nodes ahead, that's a big process advantage... ------- If GH100 is faster in practice, then all the work has been somewhat wasted.