3 ms·
Fair enough - I wouldn't recommend going with a 5K GPU for home use either. 3090s or 4090s! I have 2 4090s personally, which is perfect for pretty serious 7B f
by buildbot 3y ago
Fair enough - I wouldn't recommend going with a 5K GPU for home use either. 3090s or 4090s!
I have 2 4090s personally, which is perfect for pretty serious 7B fine-tuning and inference, and doing development work on smaller stuff before scaling to larger runs in the cloud.
At work anything less than 8 GPUS per run is small time stuff - we sometimes scale up to 128 or 256 GPUs for some runs.