3 ms·
The difference is there was a lot of work done with the multiple networks S.D. uses to make them fit into <8GB of GPU memory before they released it. In this ca
by mk_stjames 4y ago
The difference is there was a lot of work done with the multiple networks S.D. uses to make them fit into <8GB of GPU memory before they released it. In this case, and the # of parameters for this level of GPT is in the hundreds of GB (I think I read an estimate about ~340GB) meaning that there isn't a single consumer GPU that can even load the model. You need a rack of A100's even to do any inference.