4 ms·
> a 70B model would fit on an RTX 4090 With this technique, it will not, because it uses a custom FPGA, not consumer GPUs
by dustypotato 2y ago
> a 70B model would fit on an RTX 4090
With this technique, it will not, because it uses a custom FPGA, not consumer GPUs
- dwallin 2y agoThey tested their models on both a conventional gpu as well as a custom FPGA.