48 gb suffice for 4-bit inference and q-lora training of a 70b model. ~80 GB allows you to push it to 8-bit (which is nice of course), but full precision finetu

48 gb suffice for 4-bit inference and q-lora training of a 70b model. ~80 GB allows you to push it to 8-bit (which is nice of course), but full precision finetuning is completely out of reach either way. Though you're right of course that pcie will totally suffice for this case.