4 ms·

If all you’ve done is download the model and perform basic prompts then I understand why you think this. There is a lot more going on behind Bard and GPT than a

If all you’ve done is download the model and perform basic prompts then I understand why you think this. There is a lot more going on behind Bard and GPT than a chat window passing the inputs to the model. Edit for clarity: You’re comparing a platform (Bard, GPT) to a model (llama, etc). The majority of folks playing with local models are missing the platform. In order to close the gap, you need to hook up the local models to LangChain and build up different workflows for different use cases. Consequently, this is also when you start hitting the limits of consumer hardware. It’s easy to download a torrent, double click the binary and pass some simple prompts into the basic model. Once you add memory, agents, text splitters, loaders, vector db, etc, is when the value of a high end GPU paired with a capable CPU + tons of memory becomes evident. This still requires a lot of technical experience to put together a solution beyond running the examples in their docs.

alsodumb 3y ago
All the things you mentioned make it a platform, but even as a model, none of the smaller open-source models come close to GPT 3.5 or 4 in my experience. You can test it by using the GPT3.5 or 4 with their API. They outputs are waaaay better than anything I get from the open source models.
- Art9681 3y ago
  I am not doubting you and my experience has been the same. My current home lab has a pretty good Jupyter server where I experiment with different local models vs GPT using LangChain and the simple chains can achieve some impressive parity with GPT3.5 depending on the use case and local model. Things do break down when we I do more complex things due to compute capacity. Im still running all of the local models on CPU mind you. I have not gotten to the point of testing on a high end GPU yet but based on what ive seen so far, it won't take much more to run smaller local models that are good enough. This is they key. On the client side, we want smaller more focused models. This is what the post linked in this thread hints at and I agree. We are months, if not weeks, if not days...and maybe hours (at this pace!) where those smaller more domain specific models are common. Still, they won't solve the issues I mentioned above. You will likely need to build your own platform around it, or pay exorbitant fees to host it in the Cloud.
crazyedgar 3y ago
Are you sure? I have yet to see any evidence that anyone at all (including Google) has built a model (or a "platform" as you prefer to refer to them) that can follow instructions as well as 50% of ChatGPT, let alone GPT-4. I don't think any amount of work in LangChain and vector databases is enough to fix this: you really need a strong base model that is trained to align with human intentions well. Of course if you just want a bot that can answer free-form simple questions, then maybe people can't tell the difference. Just give them some real work to do and it becomes glaringly obvious.
- fzliu 3y ago
  Vector databases such as Milvus are only there to help reduce/minimize hallucinations rather than get rid of them completely. Until we have a model architecture that can perform completion from the _prompt only_ rather than pre-training data, hallucinations will always be present.

alsodumb 3y ago

All the things you mentioned make it a platform, but even as a model, none of the smaller open-source models come close to GPT 3.5 or 4 in my experience. You can test it by using the GPT3.5 or 4 with their API. They outputs are waaaay better than anything I get from the open source models.

Art9681 3y ago
I am not doubting you and my experience has been the same. My current home lab has a pretty good Jupyter server where I experiment with different local models vs GPT using LangChain and the simple chains can achieve some impressive parity with GPT3.5 depending on the use case and local model. Things do break down when we I do more complex things due to compute capacity. Im still running all of the local models on CPU mind you. I have not gotten to the point of testing on a high end GPU yet but based on what ive seen so far, it won't take much more to run smaller local models that are good enough. This is they key. On the client side, we want smaller more focused models. This is what the post linked in this thread hints at and I agree. We are months, if not weeks, if not days...and maybe hours (at this pace!) where those smaller more domain specific models are common. Still, they won't solve the issues I mentioned above. You will likely need to build your own platform around it, or pay exorbitant fees to host it in the Cloud.

crazyedgar 3y ago

Are you sure? I have yet to see any evidence that anyone at all (including Google) has built a model (or a "platform" as you prefer to refer to them) that can follow instructions as well as 50% of ChatGPT, let alone GPT-4. I don't think any amount of work in LangChain and vector databases is enough to fix this: you really need a strong base model that is trained to align with human intentions well. Of course if you just want a bot that can answer free-form simple questions, then maybe people can't tell the difference. Just give them some real work to do and it becomes glaringly obvious.

fzliu 3y ago
Vector databases such as Milvus are only there to help reduce/minimize hallucinations rather than get rid of them completely. Until we have a model architecture that can perform completion from the _prompt only_ rather than pre-training data, hallucinations will always be present.