4 ms·
Pardon me for asking a bunch of consumer-gpu questions: 1) is there any architectural difference between the 3090 and the 4090? Are there models that will run
by mlsu 3y ago
Pardon me for asking a bunch of consumer-gpu questions:
1) is there any architectural difference between the 3090 and the 4090? Are there models that will run on 3 series that will not run on 4 series cards due to feature incompatibility? or is it just raw speed/power efficiency?
2) And; on raw speed. My understanding is that the 4090 is roughly 2x faster than the 3090, right? Are there any things in particular where the 4090 is much much faster (such as, big chip architecture changes that really accelerate some operations)?
3) And somewhat relatedly, if you have a 2x 3090 rig, is that about as fast as a single 4090?
4) And; if you have a 2x 3090 rig, are you able to train larger models? My understanding is that training/finetuning requires significantly more VRAM (and that is because inference can be done with quantization, whereas training must be done at full fp16 precision, right?) But can you train e.g. a model that requires 48gb of space with 2 3090s on the same rig, or do you need a large card with 48gb, like an a4000?
- deleted 3y ago[deleted]
- coolspot 3y ago> Are there models that will run on 3 series that will not run on 4 series cards due to feature incompatibility? No, it is all the same matrix multiplication. > if you have a 2x 3090 rig, is that about as fast as a single 4090? Yes, but double the VRAM > But can you train e.g. a model that requires 48gb of space with 2 3090s on the same rig, or do you need a large card with 48gb, like an a4000? Yes, 2x 3090 would be able to handle anything that a 48Gb card can. For training you will need space for weights and gradients. If using Adam optimizer, that would be 2x-4x model size. Plus weights, plus activations, plus inputs times batch size. So a 24Gb card can train approximately a 3B model without compromises. I am not a lawyer.
- quickthrower2 3y agoI am not your data scientist