4 ms·
Apple is releasing CoreAI which is supposed to be optimized for LLMs and the transformer architecture.
by satvikpendem 4mo ago
Apple is releasing CoreAI which is supposed to be optimized for LLMs and the transformer architecture.
- mfro 4mo agoI've done some basic testing of the CoreAI framework (using Apple's official 'llm-runner' and officially supported .coreai converted models) and seen no noticable performance increase between standard MLX or GGUF with llama.cpp. I'd love to see some thorough benchmarks from someone though.
- htk 4mo agoThe idea is that it uses a lot less power than the GPU.