4 ms·

Won’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be? The tools you’re talking about like

Won’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be? The tools you’re talking about like math, wiki, or search are already built as plugins on ChatGPT. I see so many AI apps being built, but I think ChatGPT will be general enough to cover 85-90% of use-cases using the chat UI.

fbrncci 3y ago
It's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, legal and security reasons need a proprietary solution and won't be able to interface with any third party APIs, plugins or frameworks.
- theturtletalks 3y ago
  I completely agree with you on AI being extensively integrated into existing apps and being leveraged that way. But most of the AI apps I see are just like a “skin” on ChatGPT API. I do think there is value in a universal chat UI that can connect to GPT-3 and other models.
  - fbrncci 3y ago
    I think that is a phase, which will come to pass, eventually most of the plugins and apps which are going to be popular, will likely run their own models or use open source models. Because the API calls for complex applications to OpenAI are currently far from economical. In the end you will have to charge the user, and that is going to be the crux. As a sole developer, doing loads of experiments on less than 50 documents, I am already crossing 50$ in API calls within a month. I can already run llama.cpp, but its just not good enough; but the cost would effectively be 0$ (not counting my hardware).
chaxor 3y ago
The most important aspect of langchain is NOT using OpenAI for the LM. The most useful aspect of using langchain is to use it with Galpaca (or vicuna/koala/etc) to spin up an assistant for your home. This way, you can push all of your files through it - even petabytes or terabytes of files, at a fraction of the cost - and have it organize things for you. No privacy problems, no extreme costs, just ease of use, low latency, offline, blazingly fast beauty. That's at least the trajectory. Meta may soon release an improvement to Galactica similar to Galpaca (GeorgiaTech attempt) more officially (perhaps with more multimodal focus), which will likely improve upon the llama based models even further. ChatGPT is just one model among many here, and it's not even the first to use RLHF (Deepmind, as usual, was a bit earlier). The simple task of downloading Redpamajas/thePile/etc and getting a vector db for it locally, and enhancing it with local files effectively brings a local Google to everyone, and it may only require a decent spinning disk HD for the DB storage with the typical langchain LLM setup to have a completely local 'jarvis'-like assistant. (Sure, I know some people care about 'news'-like info that requires connectivity, but most things don't)
- 0xDEF 3y ago
  The vast majority of people building LM apps with (or without) LangChain are using OpenAI. I sincerely hope local LM tech like Galpaca (or vicuna/koala/etc) succeed but I don't understand why we are collectively pretending they are currently anywhere near gpt-3.5-turbo both in terms of speed and quality. Honestly the local models feel more like first generation BERT/GPT-1 models that have been fine-tuned for QA using RLHF.
  - chaxor 3y ago
    The models now are probably closer to GPT-2, and gets very close to GPT-3 with decent hardware (larger param models) and effort. They're certainly worse when it comes to 'knowledge' or memorized parts of the training data, but a decent bit of this is ameliorated by having "the pile" + other data locally available for reference. There are some issues that arise from not having decent priors due to that lack of knowledge, which may or may not be important for the given task. A (perhaps somewhat bad) example may be: if you ask me "what is a good one-liner in bash for parsing xml from a stream", I may give you an answer using xmlstarlet. However, this may not be the best answer - since Xalan can handle XSLT version 3, but xmlstarlet can't (XSLTv3 handles streams). So if looking up information in the database, some things may be slightly missed like that - but this behavior would be close to what ChatGPT offers (ChatGPT is quite awful in this way most of the time). You are right that it would miss GPT-4 by a good bit though in these cases, but most people aren't using GPT-4 for this anyway. Ultimately both can be used. OpenAI can do things you really want it for, such as a programming assistant, or things that may require much more "reasoning" (not a proper word, but conveys the message) capabilities. Local models can do the really useful base work of completely re-organizing or re-encoding files to free up space if you set it to do so, integrating with a HomeAssistant system, setting up a HomeAssistant system if you want one, answering your vocal 'Alexa/Siri'-like questions completely offline, setting up backup solutions for all your computers, setting up servers that perform more tasks that you may want - essentially a complete personal assistant. OpenAI shouldn't be needed for this, and it is highly desired to not have them do any of this (due to costs and the number of credentials it would give to them).
hospitalJail 3y ago
We cannot give my company's information to OpenAI/MS. No legal paperwork will change this. This information is so important, it is only on offline computers.

fbrncci 3y ago

It's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, legal and security reasons need a proprietary solution and won't be able to interface with any third party APIs, plugins or frameworks.

theturtletalks 3y ago
I completely agree with you on AI being extensively integrated into existing apps and being leveraged that way. But most of the AI apps I see are just like a “skin” on ChatGPT API. I do think there is value in a universal chat UI that can connect to GPT-3 and other models.
- fbrncci 3y ago
  I think that is a phase, which will come to pass, eventually most of the plugins and apps which are going to be popular, will likely run their own models or use open source models. Because the API calls for complex applications to OpenAI are currently far from economical. In the end you will have to charge the user, and that is going to be the crux. As a sole developer, doing loads of experiments on less than 50 documents, I am already crossing 50$ in API calls within a month. I can already run llama.cpp, but its just not good enough; but the cost would effectively be 0$ (not counting my hardware).

chaxor 3y ago

The most important aspect of langchain is NOT using OpenAI for the LM. The most useful aspect of using langchain is to use it with Galpaca (or vicuna/koala/etc) to spin up an assistant for your home. This way, you can push all of your files through it - even petabytes or terabytes of files, at a fraction of the cost - and have it organize things for you. No privacy problems, no extreme costs, just ease of use, low latency, offline, blazingly fast beauty. That's at least the trajectory. Meta may soon release an improvement to Galactica similar to Galpaca (GeorgiaTech attempt) more officially (perhaps with more multimodal focus), which will likely improve upon the llama based models even further. ChatGPT is just one model among many here, and it's not even the first to use RLHF (Deepmind, as usual, was a bit earlier). The simple task of downloading Redpamajas/thePile/etc and getting a vector db for it locally, and enhancing it with local files effectively brings a local Google to everyone, and it may only require a decent spinning disk HD for the DB storage with the typical langchain LLM setup to have a completely local 'jarvis'-like assistant. (Sure, I know some people care about 'news'-like info that requires connectivity, but most things don't)

0xDEF 3y ago
The vast majority of people building LM apps with (or without) LangChain are using OpenAI. I sincerely hope local LM tech like Galpaca (or vicuna/koala/etc) succeed but I don't understand why we are collectively pretending they are currently anywhere near gpt-3.5-turbo both in terms of speed and quality. Honestly the local models feel more like first generation BERT/GPT-1 models that have been fine-tuned for QA using RLHF.
- chaxor 3y ago
  The models now are probably closer to GPT-2, and gets very close to GPT-3 with decent hardware (larger param models) and effort. They're certainly worse when it comes to 'knowledge' or memorized parts of the training data, but a decent bit of this is ameliorated by having "the pile" + other data locally available for reference. There are some issues that arise from not having decent priors due to that lack of knowledge, which may or may not be important for the given task. A (perhaps somewhat bad) example may be: if you ask me "what is a good one-liner in bash for parsing xml from a stream", I may give you an answer using xmlstarlet. However, this may not be the best answer - since Xalan can handle XSLT version 3, but xmlstarlet can't (XSLTv3 handles streams). So if looking up information in the database, some things may be slightly missed like that - but this behavior would be close to what ChatGPT offers (ChatGPT is quite awful in this way most of the time). You are right that it would miss GPT-4 by a good bit though in these cases, but most people aren't using GPT-4 for this anyway. Ultimately both can be used. OpenAI can do things you really want it for, such as a programming assistant, or things that may require much more "reasoning" (not a proper word, but conveys the message) capabilities. Local models can do the really useful base work of completely re-organizing or re-encoding files to free up space if you set it to do so, integrating with a HomeAssistant system, setting up a HomeAssistant system if you want one, answering your vocal 'Alexa/Siri'-like questions completely offline, setting up backup solutions for all your computers, setting up servers that perform more tasks that you may want - essentially a complete personal assistant. OpenAI shouldn't be needed for this, and it is highly desired to not have them do any of this (due to costs and the number of credentials it would give to them).

hospitalJail 3y ago

We cannot give my company's information to OpenAI/MS. No legal paperwork will change this. This information is so important, it is only on offline computers.