I think the biggest architectural change here is them doing different compute for prefill & decode, with pretty much architecture from this microsoft research w

I think the biggest architectural change here is them doing different compute for prefill & decode, with pretty much architecture from this microsoft research work from 2024 https://arxiv.org/abs/2405.05254 https://arxiv.org/abs/2405.05254, very exciting stuff!