3 ms·

Today the Mini tops out at 48GB. Gotta go to the Studio to get 64GB.

• aurareturn 3mo ago
  Don't buy the Mini or Studio. Both have the M4 which lacks the Neural Accelerators, making prompt processing ~3-4x slower.
  • mortenjorck 3mo ago
    I assume those don't just work automatically with an off-the-shelf gguf. What do you need in your local inference stack to take advantage of M5's neural accelerators?
    • aurareturn 3mo ago
      They do work with llama.cpp and MLX automatically.
    • wren6991 3mo ago
      Apple muddied the waters by calling them "neural accelerators" but it seems like what they actually added in the M5 generation is tensor instructions for the existing GPU cores. It's not a separate accelerator like the ANE. llama.cpp's Metal backend does use them when they're available.
  • 2Gkashmiri 3mo ago
    Apple Mac Studio (M3 Ultra Chip/28 CPU, 60 GPU/96 GB/1 TB How is this config?