4 ms·
AI chip making? I can train an AI on my intel laptop if I whish. If I need more CPU power, i can rent some. The genie is out of the bottle and the only way is f
by nuancebydefault 4y ago
AI chip making? I can train an AI on my intel laptop if I whish. If I need more CPU power, i can rent some.
The genie is out of the bottle and the only way is forward. The latest worldwide race.
- avital 4y agoThis isn't accurate. The bottleneck in very-large-scale-training BY FAR is communication between devices. If you have a million CPUs, the communication cost will be significantly higher than a thousand A100s (perhaps in the order of 100x or even more). So this is only possible to replicate with very dense and high compute chips with extremely fast interconnect.
- nuancebydefault 4y agoThanks for providing this insight. Is A100 the only platform? Can we pause/resume all such platforms simultaneously?