2 ms·
I see the DIGITs box as mainly for inference, not training. It allows me to load a fairly large model (e.g. 70B llama or 12B flux) and run it locally at decent
by kadushka 2y ago
I see the DIGITs box as mainly for inference, not training. It allows me to load a fairly large model (e.g. 70B llama or 12B flux) and run it locally at decent speeds.
- richardw 2y agoThen surely far simpler custom chips are the eventual model, like happened with crypto? Groq, Etched etc. In that universe, Nvidia has absolutely no moat and a thousand chips are coming.
- kadushka 2y agoMaybe. For me to consider buying some unknown hardware it needs be a lot better than Nvidia. Like double the speed, half the power, and half the price. And the software better be rock solid and tested on all popular models. Many startups have been trying for several years now, and eventually someone will succeed, but it's not easy. Even AMD hasn't been able to pull it off.