4 ms·
I’ve got a 64gb Mac M2. All of the openllm models seem to hang on startup or on API calls. I got them working through GCP colab. Not sure if it’s a configura
by benjaminwootton 3y ago
I’ve got a 64gb Mac M2. All of the openllm models seem to hang on startup or on API calls. I got them working through GCP colab. Not sure if it’s a configuration issue or if the hardware just isn’t up to it?
- benreesman 3y agoValiant et al work great on my 64Gb Studio at Q4_K_M. Happy to answer questions.
- wahnfrieden 3y agoTry llama.cpp with Metal (critical) and GGUF models from TheBloke Or wait another month or so for https://ChatOnMac.com https://ChatOnMac.com