I believe at least Google is starting to do edge inference—take a look at the pixel 8 line-up they just announced. It doesn't seem to be emphasized as much, but

I believe at least Google is starting to do edge inference—take a look at the pixel 8 line-up they just announced. It doesn't seem to be emphasized as much, but the tensor G3 chip certainly has builtin inference.