3 ms·

Ask HN: Is there a good alternative for a local ChatGPT

11 points by ludovicianul 4y ago

I would like to have something as similar as possible to ChatGPT, but deployed in a closed environment. Feed it my specific data and query it afterwards. What would you recommend?

OhNoNotAgain_99 4y ago
[dead]
gostsamo 4y ago
Keep in mind that the number of parameters on this thing goes to hundreds of billions. So, start with a local datacenter and continue from there. Facebook's Lama which is supposed to be both similar in performance and much reduced in size has only 65 billion params and weights around 220gb (as far as I remember from an article). You will need a beefy server to run this and it would better be shared or the server will be idle most of the time.
- ludovicianul 4y ago
  I've exaggerated a bit when saying close to ChatGPT. But my focus is getting something which already bundles the complexity, feed it data and start using it. Without running 100 seps to reach that state.
  - gostsamo 4y ago
    Well, you can try the hugging face models. I'm not sure if someone has already built an consumer level app on them, but at least they provide trained models for the most popular stuff out there in an easy to pip install packages. Alternatively, you can check Eutheler AI though I haven't worked with anything of theirs. I just know that they were trying to be some kind of Open AI alternative. 1. https://www.eleuther.ai/about/ https://www.eleuther.ai/about/
    - ludovicianul 4y ago
      Thanks!
dstala 4y ago
https://huggingface.co/ https://huggingface.co/
amrb 4y ago
A web UI for running Large Language Models https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui
ad404b8a372f2b9 4y ago
There is a thread on /g/ about running your own LLM chatbots with a lot of good info.
labarilem 4y ago
This might help you: https://www.hpc-ai.tech/blog/colossal-ai-chatgpt https://www.hpc-ai.tech/blog/colossal-ai-chatgpt

3 ms·

Ask HN: Is there a good alternative for a local ChatGPT

11 points by ludovicianul 4y ago

I would like to have something as similar as possible to ChatGPT, but deployed in a closed environment. Feed it my specific data and query it afterwards. What would you recommend?

OhNoNotAgain_99 4y ago
[dead]
gostsamo 4y ago
Keep in mind that the number of parameters on this thing goes to hundreds of billions. So, start with a local datacenter and continue from there. Facebook's Lama which is supposed to be both similar in performance and much reduced in size has only 65 billion params and weights around 220gb (as far as I remember from an article). You will need a beefy server to run this and it would better be shared or the server will be idle most of the time.
- ludovicianul 4y ago
  I've exaggerated a bit when saying close to ChatGPT. But my focus is getting something which already bundles the complexity, feed it data and start using it. Without running 100 seps to reach that state.
  - gostsamo 4y ago
    Well, you can try the hugging face models. I'm not sure if someone has already built an consumer level app on them, but at least they provide trained models for the most popular stuff out there in an easy to pip install packages. Alternatively, you can check Eutheler AI though I haven't worked with anything of theirs. I just know that they were trying to be some kind of Open AI alternative. 1. https://www.eleuther.ai/about/ https://www.eleuther.ai/about/
    - ludovicianul 4y ago
      Thanks!
dstala 4y ago
https://huggingface.co/ https://huggingface.co/
amrb 4y ago
A web UI for running Large Language Models https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui
ad404b8a372f2b9 4y ago
There is a thread on /g/ about running your own LLM chatbots with a lot of good info.
labarilem 4y ago
This might help you: https://www.hpc-ai.tech/blog/colossal-ai-chatgpt https://www.hpc-ai.tech/blog/colossal-ai-chatgpt