2 ms·
>Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT. Note that I never said "open source" just "open models". As in, I
by ramesh31 3y ago
>Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT.
Note that I never said "open source" just "open models". As in, I can now actually build things with GPT3 capability that run locally. I couldn't care less about the model code.
>Llama 2 still requires multiple kilowatts of electricity and $10,000+ of compute hardware per inference session.
I'm running llama-2-7b-chat on my 8GB M1 Mac right now with llama.cpp. Completions are instant, and essentially at GPT3 levels of accuracy.
The higher param models require up to 64GB RAM, but it's all CPU based.
- gyrovagueGeist 3y agoIf you compiled llama.cpp with Metal I wouldn’t say its ~all CPU based. But it is still incredible! I need to set this up myself this weekend :)
- ramesh31 3y agohttps://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML