3 ms·

Thanks! So this could actually be worth running on a mid-specced recentish Mac laptop?

• daemonologist 2y ago
  Macs are a bit odd because you're using system memory (so, more but slower) and Apple's APIs, but it's definitely worth a try. I'd guess >= M2 Pro + >= 24 GB RAM would do okay. Also maybe try an MLX version? (e.g. https://huggingface.co/justinmeans/DeepCoder-14B-Preview-mlx-4Bit https://huggingface.co/justinmeans/DeepCoder-14B-Preview-mlx...)
  • apothegm 2y ago
    Sweet. Thank you so much!