4 ms·
Running with a Local LLM on a Mac M1, this completely locked up my system for minutes. I tried to let it run, because the progress bar was ticking every now and
by 2024throwaway 3y ago
Running with a Local LLM on a Mac M1, this completely locked up my system for minutes. I tried to let it run, because the progress bar was ticking every now and then, but after 10 minutes I gave up and killed it.
- tarruda 3y agoI wouldn't recommend unless you got at least 16gb ram (though possibly more is needed depending on what model is used).
- 2024throwaway 3y agoI do have 16gb ram.
- raxxorraxor 3y agoYou are running the wrong model. Get a smaller one Macs are usually ok with their performance since the GPU has access to main memory. A decent Nvidia GPU with lots of memory is still king though. I run codelama:7b on a MacBook air, and even auto complete is partially usable.