3 ms·
This Neural Engine seems useless for LLMs. Trapped in the wrong architecture
by brcmthrowaway 4mo ago
This Neural Engine seems useless for LLMs. Trapped in the wrong architecture
- satvikpendem 4mo agoApple is releasing CoreAI which is supposed to be optimized for LLMs and the transformer architecture.
- mfro 4mo agoI've done some basic testing of the CoreAI framework (using Apple's official 'llm-runner' and officially supported .coreai converted models) and seen no noticable performance increase between standard MLX or GGUF with llama.cpp. I'd love to see some thorough benchmarks from someone though.
- htk 3mo agoThe idea is that it uses a lot less power than the GPU.