6 ms·

Would you please describe the basics of this process? I am interested in doing this and my hardware is similarly limited. No dedicated GPU, but 24g ram and soli

by eth0up 2y ago

Would you please describe the basics of this process? I am interested in doing this and my hardware is similarly limited. No dedicated GPU, but 24g ram and solid CPU.

nejsjsjsbsb 2y ago
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama docker exec -it ollama ollama run llama3.2:1b Have a chat at the CLI Then try bigger models if that is too fast for you :)
BMc2020 2y ago
Alpaca is in the Pop!_shop (what is called the store on most other distros) so it's just a matter of clicking install. Inside it it has a list of several models, I downloaded a small model, but even a small model is very big, 4 Gigs. Some are over 100GB. When one is downloaded just click on new chat. https://github.com/Jeffser/Alpaca https://github.com/Jeffser/Alpaca GPT4ALL on the cinnamon mint box I first installed Easy Flatpak then installed it from there. That I had a little different goal, I had a lot of pdf and epub books I wanted to index. I downloaded a small model again, a 4.34G Llama 3 8B Instruct. In the local docs I put one pdf 'Github for Dummies' and asked it to index. It took about 30 minutes IIRC. (don't start with too many, it could be days indexing. You can add docs later and re-index so you can build your database slowly) Then instead of looking up in the book how to revert a commit, you chat with the model and ask it how to revert a commit. My take is the model is the human language interface for queries, and that's all. Bear in mind almost everyone knows more about this me, I just wanted to see what all the fuss was about. You can also do similar what I did with gpt4all online for free as long as you have a google account. Go to notebooklm dot google dot com and start uploading your documents.
- eth0up 2y ago
  Thanks for the info.

6 ms·

Would you please describe the basics of this process? I am interested in doing this and my hardware is similarly limited. No dedicated GPU, but 24g ram and soli

by eth0up 2y ago

Would you please describe the basics of this process? I am interested in doing this and my hardware is similarly limited. No dedicated GPU, but 24g ram and solid CPU.

nejsjsjsbsb 2y ago
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama docker exec -it ollama ollama run llama3.2:1b Have a chat at the CLI Then try bigger models if that is too fast for you :)
BMc2020 2y ago
Alpaca is in the Pop!_shop (what is called the store on most other distros) so it's just a matter of clicking install. Inside it it has a list of several models, I downloaded a small model, but even a small model is very big, 4 Gigs. Some are over 100GB. When one is downloaded just click on new chat. https://github.com/Jeffser/Alpaca https://github.com/Jeffser/Alpaca GPT4ALL on the cinnamon mint box I first installed Easy Flatpak then installed it from there. That I had a little different goal, I had a lot of pdf and epub books I wanted to index. I downloaded a small model again, a 4.34G Llama 3 8B Instruct. In the local docs I put one pdf 'Github for Dummies' and asked it to index. It took about 30 minutes IIRC. (don't start with too many, it could be days indexing. You can add docs later and re-index so you can build your database slowly) Then instead of looking up in the book how to revert a commit, you chat with the model and ask it how to revert a commit. My take is the model is the human language interface for queries, and that's all. Bear in mind almost everyone knows more about this me, I just wanted to see what all the fuss was about. You can also do similar what I did with gpt4all online for free as long as you have a google account. Go to notebooklm dot google dot com and start uploading your documents.
- eth0up 2y ago
  Thanks for the info.