I've seen some very recent research proposing just that: https://arxiv.org/pdf/2305.05033.pdf https://arxiv.org/pdf/2305.05033.pdf . Of course still far from pr

I've seen some very recent research proposing just that: https://arxiv.org/pdf/2305.05033.pdf https://arxiv.org/pdf/2305.05033.pdf . Of course still far from production, but interesting that in some cases it actually also improves latency.