There's a sweet spot for running MoE models, though. If you need the entire model in VRAM but only need to retrieve a part of it per token, trading more memory

There's a sweet spot for running MoE models, though. If you need the entire model in VRAM but only need to retrieve a part of it per token, trading more memory for less bandwidth can be a win. I have a 4090, and given the MoE trend, I'd be more tempted to purchase a Strix Halo next than a 5090.