3 ms·
How is it possible to run this model on 2x 4090s ? I thought that 4090s were "nerfed" and nvlink support removed - https://www.windowscentral.com/hardware/comp
by sireat 3y ago
How is it possible to run this model on 2x 4090s ?
I thought that 4090s were "nerfed" and nvlink support removed - https://www.windowscentral.com/hardware/computers-desktops/nvidia-kills-off-nvlink-on-rtx-4090 https://www.windowscentral.com/hardware/computers-desktops/n...
- Timshel 3y agoProbably this : https://huggingface.co/docs/accelerate/usage_guides/big_modeling https://huggingface.co/docs/accelerate/usage_guides/big_mode...
- MacsHeadroom 3y agoWithout NVLink different layers are loaded onto individual cards and only one card often has to wait for the other card, slowing down generation. It's still faster than CPU offloading. You can even mix and match GPUs like a 4090 + 3060.