You can use it as the assistant model to a large model, it's called speculative sampling. You generate text with the small model and validate with the large one

You can use it as the assistant model to a large model, it's called speculative sampling. You generate text with the small model and validate with the large one, ensuring no deviation occurs. Speedup of 2.5x https://arxiv.org/abs/2302.01318 https://arxiv.org/abs/2302.01318 (DeepMind)