3 ms·

I wonder if the mlx optimized would run on 64gb mac

• CharlesW 1y ago
  LM Studio's heuristics (which I've found to be pretty reliable) suggest that a 3-bit quantization (~50 GB) should work fine.
  • qafy 1y ago
    You can fine tune the amount of unified memory reserved for the system vs GPU, just search up `sysctl iogpu.wired_limit_mb`. On my 64gb mac mini the default out of the box is only like ~44gb available to the GPU (i forget the exact number), but tuning this parameter should help you run models that are a little larger than that.