3 ms·
Well, the sanctions are an attempt at denying China access to more powerful GPUs like the A100, so it seems the sanctions may actually accelerate research into
by 01100011 4y ago
Well, the sanctions are an attempt at denying China access to more powerful GPUs like the A100, so it seems the sanctions may actually accelerate research into running LLMs on lower performing HW.
- kkielhofner 4y agoWhat's so weird about that (not surprising because lawmakers/politics) is that A100s are actually terrible at inference. Assuming the model can fit in RAM my 3090 handily beats them across the board at everything I've tried. My 4090 blows them away. So by the most commonly accepted definition of "performance" the GPUs banned in China are actually /lower/ performance. The only thing this ban does is slow down from scratch training of very large models. There's a lot of effort on finetuning these large models on < 80GB VRAM hardware but significantly more effort on "running" them on < 24GB VRAM hardware (3090/4090 and lower). In the end your point stands - between any efforts by those impacted by sanctions or just the hobbyist/low-mid range commercial applications research into cramming these huge models into "consumer" hardware are accelerating dramatically.
- riku_iki 4y ago3090 is not allowed to be used in datacenters by nvidia, which China likely doesn't care about, so they are in advantage at the end of the day.
- kkielhofner 4y agoInterestingly that (in)famous language is only included in the driver EULA for the Nvidia Windows driver, not the Linux driver (last I looked).