For the output performance (quality and tokens/second), Mixtral 8x7B seems to require less VRAM. But it is still hard to make it fit, even with a lot of quantiz

For the output performance (quality and tokens/second), Mixtral 8x7B seems to require less VRAM. But it is still hard to make it fit, even with a lot of quantization, within the GPU RAM of most discreet consumer GPUs. Perhaps a smaller base model like Phi-2 could bring the VRAM requirements down, but the MoE will bring the output quality up from Phi-2.