3 ms·
No thanks. I'll run privately self-hosted llama on my phone. Sharing data with OpenAI it's *irresponsible step* at this time of development.
by transformi 3y ago
No thanks. I'll run privately self-hosted llama on my phone.
Sharing data with OpenAI it's *irresponsible step* at this time of development.
- brunoqc 3y ago> I'll run privately self-hosted llama on my phone. any guides on how to do that?
- transformi 3y agoYes! You have two options: 1- Find some port for it to run locally on GitHub (it will be quantized and not that useful right now, till 2024-till Qualcomm will ship their phones). - see https://github.com/Bip-Rep/sherpa https://github.com/Bip-Rep/sherpa *The better one* 2- Host the version on a server on the cloud (or locally on your computer with tunneling) using ooga-booga (with --api flag), and communicate with that self-hosted llama, directly. This way you won't be limited to 7/13B versions but can run the 70B...
- darkmuck 3y agoMlc-llm for Android