3 ms·
If you want to try Llama 2 on a Mac and have Homebrew (or Python/pip) you may find my LLM CLI tool interesting: https://simonwillison.net/2023/Aug/1/llama-2-mac
by simonw 3y ago
If you want to try Llama 2 on a Mac and have Homebrew (or Python/pip) you may find my LLM CLI tool interesting: https://simonwillison.net/2023/Aug/1/llama-2-mac/ https://simonwillison.net/2023/Aug/1/llama-2-mac/
- petulla 3y agoWhat's the inference time without gpu?
- lm2s 3y agoIt might the time mentioned at the bottom of the page since the author isn't sure that the GPU is being used: >How to speed this up—right now my Llama prompts often take 20+ seconds to complete.
- bravura 3y agoDoes it support Metal / MPS acceleration?
- SOLAR_FIELDS 3y agoI’ve gotten mine running with FastChat - they have a Metal/MPS option. Sadly 7b is not very good for SQL tasks. I think even with RAG it would struggle.