3 ms·
One person can implement a transformer model from scratch in a weekend. Hardware is not the valuable part of machine learning. Data and how it is used are. The
by bob1029 1y ago
One person can implement a transformer model from scratch in a weekend. Hardware is not the valuable part of machine learning. Data and how it is used are.
The "magic of AI" doesn't live inside an Nvidia GPU. There are billions of dollars of marketing being deployed to convince you it does. As soon as the market realizes that nvidia != magic AI box, the music should stop pretty quickly.
- tehjoker 1y agoThat's true, but without the kind of horsepower provided by modern hardware, even though I'm skeptical that it's all needed, especially given DeepSeek's amazing results, AI would be nearly impossible. There are some important innovations on the algorithm / network structure side, but all these ideas are only able to be tried because the hardware supports it. This stuff has been around for decades.
- chermi 1y agoDeepseek required existing models that required the horsepower.
- tehjoker 1y agoThat was claimed but never proven. I personally don't believe the American companies making this claim. I suspect they made this up to protect their valuations when they were hideously embarassed and lost a trillion dollars in equity.
- chermi 1y agoUmm, part of it does. It necessary but not sufficient, at least to achieve it on the timescales we've seen. Scale is part of the "magic".