4 ms·
Or just develop the next wave of chips designed for specifically transformer-based architectures (and ternary computing), and bypass the needs for GPUs and CUDA
by dogcomplex 2y ago
Or just develop the next wave of chips designed for specifically transformer-based architectures (and ternary computing), and bypass the needs for GPUs and CUDA altogether
- Zambyte 2y agoThat would be betting against other architectures like Mamba, which does not seem like an obviously good bet to make yet. Maybe it is though.
- dogcomplex 2y agoYou're right, there are a number of avenues that are viable alternatives to the gpu monopoly. I like the fact that these can be made with just mass-printed multiplication (and in ternary computing's case - addition) gates which require little more than 10 year old tech which is already widely distributed.