Not to be flippant, but paying for access to an API is going to be way more cost-effective than buying sufficient hardware to run this locally. You might try wa

Not to be flippant, but paying for access to an API is going to be way more cost-effective than buying sufficient hardware to run this locally. You might try waiting to see if someone distills this model down to a smaller one, or sparsifies it by deleting a bunch of parameters and fine-tuning, or comes up with a 1.5-bit version. But you're going to need 2x RAM to run this one even very optimistically. 4x would give better quality outputs. And it will still be fairly slow unless you throw more GPU hardware at it.