38 ms·

One killer feature is OpenAI’s vector database. I was surprised you can throw gigabytes of documents at it and chatgpt can see it all. It’s hard to simulate tha

One killer feature is OpenAI’s vector database. I was surprised you can throw gigabytes of documents at it and chatgpt can see it all. It’s hard to simulate that via context window. That’s not necessarily a moat, but OpenAI is still shipping important features. I wonder how hard it is for Claude et al to replicate.

brigadier132 2y ago
They are outsourcing that to Qdrant, anyone can replicate it.
Xenoamorphous 2y ago
What do you mean, does it do anything other than the usual RAG?
- sillysaurusx 2y ago
  Whatever it does, it’s indistinguishable from stuffing all the documents into the context window. Or at least I haven’t seen it fail yet.
  - KeplerBoy 2y ago
    Stuffing everything into the context window fails horribly every time i try it. The model just doesn't seem to be able to really process the entire input.
- ramraj07 2y ago
  I’m also confused what they’re talking about.. Does OpenAI have some feature I’m not aware of?
  - dpkirchner 2y ago
    There's a way to upload documents that can be referred to in chat. I think they call it custom GPTs (which seems like a poor name): https://help.openai.com/en/articles/8554397-creating-a-gpt https://help.openai.com/en/articles/8554397-creating-a-gpt
    - singularity2001 2y ago
      Wait if this is what OP is referring to I am even more confused because the last time I built a custom GPT it was horribly slow and very inaccurate to look up uploaded information also you certainly couldn't upload gigabytes of your own PDFs. Did anything change??
      - sillysaurusx 2y ago
        I’m referring to this specifically: https://platform.openai.com/docs/assistants/tools/file-search https://platform.openai.com/docs/assistants/tools/file-searc...
prng2021 2y ago
When these companies are getting literally multiple billions of dollars of funding thrown at them, I can't think of a single feature that can be a moat. If it's truly a feature that leapfrogs competitors, there's just no way that someone like Anthropic spending $1B on engineering resources can't replicate the same exact feature that engineers elsewhere implemented.
- Frost1x 2y ago
  If it’s simply an engineering feat I agree. Very, very, very often people tend to mix up science being done in technology spaces with engineering. Just because it’s being done in software and perhaps even without rigor doesn’t mean you’re not doing something novel and quite different at a fundamental perspective than someone else. Sometimes it could just be luck of the draw in an implementation approach that gets you there but that’s not always the case. I’ve worked in scientific computing for awhile now and there are countless subtle decisions often done in the implementation phase that, from a theoretical perspective, aren’t definitively answered by the science working behind the scenes. There’s often a gap in knowledge and implementers either hit those gaps by trial and error, luck, or insight. That’s my opinion, at least. So I don’t think it’s always “just an implementation problem” some will claim, as if the science is well understood and solved. Perhaps it is, but from my experience that tends to not be the case.
  - danielmarkbruce 2y ago
    This 100%. And LLMs and the applications around them (even something as simple seeming as ChatGPT) have more subtle decisions than any other type of software that I've ever seen. Everyone claims there isn't a moat, I bet there is.
  - oakst 2y ago
    Really interesting perspective
  - prng2021 2y ago
    Yep I agree with that. I will say though that people, teams, and even entire companies (like what happened at Inflection) get poached everyday so maintaining a moat that way is tough. Also, even though it could happen in the future, is OpenAI's lead due to a moat of scientists with ideas so novel that no other AI company can compete? Certainly not because even though ChatGPT took the world by storm, numerous other companies built LLMs in a very short time span that now perform at very similar levels, both subjectively and based on benchmarks.
jazzyjackson 2y ago
I'm sure OpenAI has some secret sauce that makes their RAG better than others, but LMStudio did ship the feature in 0.3.0 last week https://lmstudio.ai/blog/lmstudio-v0.3.0 https://lmstudio.ai/blog/lmstudio-v0.3.0
aunty_helen 2y ago
It’s not hard, we’re doing it with postgres and azure blob storage.
TrevorJ 2y ago
there are third party vector databases though
__rito__ 2y ago
I used FAISS at work in the beginning of this year, and it was fantastic. Company I worked for has massive private medical datasets and will never agree to non-local models or methods. FAISS [0] is wonderful. Give it a try. You can work with FAISS with LangChain, llamaindex and the like. [0]: https://ai.meta.com/tools/faiss https://ai.meta.com/tools/faiss
- sillysaurusx 2y ago
  Thank you! I’ve been wondering about something like this. Much appreciated.

brigadier132 2y ago

They are outsourcing that to Qdrant, anyone can replicate it.

Xenoamorphous 2y ago

What do you mean, does it do anything other than the usual RAG?

sillysaurusx 2y ago
Whatever it does, it’s indistinguishable from stuffing all the documents into the context window. Or at least I haven’t seen it fail yet.
- KeplerBoy 2y ago
  Stuffing everything into the context window fails horribly every time i try it. The model just doesn't seem to be able to really process the entire input.
ramraj07 2y ago
I’m also confused what they’re talking about.. Does OpenAI have some feature I’m not aware of?
- dpkirchner 2y ago
  There's a way to upload documents that can be referred to in chat. I think they call it custom GPTs (which seems like a poor name): https://help.openai.com/en/articles/8554397-creating-a-gpt https://help.openai.com/en/articles/8554397-creating-a-gpt
  - singularity2001 2y ago
    Wait if this is what OP is referring to I am even more confused because the last time I built a custom GPT it was horribly slow and very inaccurate to look up uploaded information also you certainly couldn't upload gigabytes of your own PDFs. Did anything change??
    - sillysaurusx 2y ago
      I’m referring to this specifically: https://platform.openai.com/docs/assistants/tools/file-search https://platform.openai.com/docs/assistants/tools/file-searc...

prng2021 2y ago

When these companies are getting literally multiple billions of dollars of funding thrown at them, I can't think of a single feature that can be a moat. If it's truly a feature that leapfrogs competitors, there's just no way that someone like Anthropic spending $1B on engineering resources can't replicate the same exact feature that engineers elsewhere implemented.

Frost1x 2y ago
If it’s simply an engineering feat I agree. Very, very, very often people tend to mix up science being done in technology spaces with engineering. Just because it’s being done in software and perhaps even without rigor doesn’t mean you’re not doing something novel and quite different at a fundamental perspective than someone else. Sometimes it could just be luck of the draw in an implementation approach that gets you there but that’s not always the case. I’ve worked in scientific computing for awhile now and there are countless subtle decisions often done in the implementation phase that, from a theoretical perspective, aren’t definitively answered by the science working behind the scenes. There’s often a gap in knowledge and implementers either hit those gaps by trial and error, luck, or insight. That’s my opinion, at least. So I don’t think it’s always “just an implementation problem” some will claim, as if the science is well understood and solved. Perhaps it is, but from my experience that tends to not be the case.
- danielmarkbruce 2y ago
  This 100%. And LLMs and the applications around them (even something as simple seeming as ChatGPT) have more subtle decisions than any other type of software that I've ever seen. Everyone claims there isn't a moat, I bet there is.
- oakst 2y ago
  Really interesting perspective
- prng2021 2y ago
  Yep I agree with that. I will say though that people, teams, and even entire companies (like what happened at Inflection) get poached everyday so maintaining a moat that way is tough. Also, even though it could happen in the future, is OpenAI's lead due to a moat of scientists with ideas so novel that no other AI company can compete? Certainly not because even though ChatGPT took the world by storm, numerous other companies built LLMs in a very short time span that now perform at very similar levels, both subjectively and based on benchmarks.

jazzyjackson 2y ago

I'm sure OpenAI has some secret sauce that makes their RAG better than others, but LMStudio did ship the feature in 0.3.0 last week https://lmstudio.ai/blog/lmstudio-v0.3.0 https://lmstudio.ai/blog/lmstudio-v0.3.0

aunty_helen 2y ago

It’s not hard, we’re doing it with postgres and azure blob storage.

TrevorJ 2y ago

there are third party vector databases though

__rito__ 2y ago

I used FAISS at work in the beginning of this year, and it was fantastic. Company I worked for has massive private medical datasets and will never agree to non-local models or methods. FAISS [0] is wonderful. Give it a try. You can work with FAISS with LangChain, llamaindex and the like. [0]: https://ai.meta.com/tools/faiss https://ai.meta.com/tools/faiss

sillysaurusx 2y ago
Thank you! I’ve been wondering about something like this. Much appreciated.