This got to be one of the major goals for the next gen generative models. Those "big" models cannot be reliably used on critical tasks thanks to various failure

This got to be one of the major goals for the next gen generative models. Those "big" models cannot be reliably used on critical tasks thanks to various failure modes. Even for those non-critical tasks, the latency is typically high enough to be unusable for many potential applications. If it can be 99.9% done in-device, this has a potential to drastically change the user experience in many directions.