I am betting on a combination of quantization and speculative sampling with a distilled smaller set of models: https://arxiv.org/pdf/2302.01318.pdf https://arxi

I am betting on a combination of quantization and speculative sampling with a distilled smaller set of models: https://arxiv.org/pdf/2302.01318.pdf https://arxiv.org/pdf/2302.01318.pdf