4 ms·
What I hoped for was a list of RAM requirements. Models are often presented as if it's irrelevant how many 80GB A100s you need. But in reality, trying out even
by uniqueuid 4y ago
What I hoped for was a list of RAM requirements. Models are often presented as if it's irrelevant how many 80GB A100s you need. But in reality, trying out even inference often has a hard threshold based on the hardware you have at hand. And that hardware still is predominantly <= 24GB (3090 TI, A5000, A30).
Does anybody know of such a list? I.e. models with their respective minimum RAM requirements?