3 ms·
I don't get why more than 2 years after the ChatGPT release moment there is not a plethora of high-mem matrix-matrix matrix-vector hardware for the high-end ava
by semessier 2y ago
I don't get why more than 2 years after the ChatGPT release moment there is not a plethora of high-mem matrix-matrix matrix-vector hardware for the high-end available. Both high bandwidth and commodity DRAM. Both operations are very well understood for dense and sparse cases. There were FPGAs and ASICs early on but nothing really caught on relative to the GPU behemoth which is tons of silicon on the die that is not needed for base matmult. Hence, it's unbelievable how Nvidia continues to charging for memory to be one of the highest value companies in the world.
- dinfinity 2y agoI believe this is mainly due to everything ML/AI optimizing for CUDA, with even AMD cards (which are very similar to Nvidia cards) unable to compete due to lack of proper support for CUDA.
- semessier 2y agothis was/is the chip opportunity of the century. Even more optimized than the still general purpose nvidia cards. And no matrix mult is abstracted away for decades, don't need CUDA. So a chip would likely be much much easier than a mixed signal chip with the Apple C1 being on the high end of nightmare in comparison.
- brador 2y agoGPUs are a managed duopoly through agreement. Nvidia takes the high end, AMD the low end. Notice they do not compete. This maximises returns for shareholders of either and both.