I wonder if there’s some “critical mass” for ML compute on device, that may make this kind of devices significantly more useful & usable. E.g., running 70B or

I wonder if there’s some “critical mass” for ML compute on device, that may make this kind of devices significantly more useful & usable. E.g., running 70B or even larger model on device should allow to cut interactive response time, and should alleviate privacy concerns while enabling incomparably greater personalization (eg using all of your emails and chats as context); but will also have significant battery life implications unless there’s some breakthrough in specialized ML ASICs. In any case, this is where part of that 1B of raised funds should be spent. But to get to the “critical mass”, this looks more of a 100B+ effort.