3 ms·
>I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and r
by pixl97 3y ago
>I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and research purposed without costing more tha n my kidneys.
At least for the time being there is no magic workaround for the amount of FLOPS and memory bandwidth needed. When you look at neural network architectures they tend to be massively parallel, and building out at that scale in hardware is going to cost a lot.
And, when the hardware does show up, at least with the scaling laws we're seeing at this time, the groups that have massive amounts of this lower powered/accellerated hardware are going to be ahead of those without budgets.