NPUs, at least for apple, are built for relatively small scale efficient background processing. Theyre taking care of extracting structured metadata from freete

NPUs, at least for apple, are built for relatively small scale efficient background processing. Theyre taking care of extracting structured metadata from freetext, recognizing people & text in photos, indexing file contents, etc. they do this really well with crazy low power draw. Theyre generally not even a target for current inference frameworks ala ollama, pytorch, etc. I did see one prototype someone built using private ios APIs a few months back. They were able to run a reasonable SLM about an order of magnitude slower than GPU but with a tiny power draw.