5 ms·

>knowledge base (files) What is it actually doing with the files you upload? Is it just pasting the full text into the prompt? Or is it doing something RAG-l

>knowledge base (files) What is it actually doing with the files you upload? Is it just pasting the full text into the prompt? Or is it doing something RAG-like and dynamically retrieving some subset based on the query?

simonw 3y ago
This is undocumented (frustrating) but it looks like it's chunking them, running embeddings on the chunks and storing the results in a https://qdrant.tech/ https://qdrant.tech/ vector database. We know it's Qdrant because an error message leaked that detail: https://twitter.com/altryne/status/1721989500291989585 https://twitter.com/altryne/status/1721989500291989585 It only applies that mechanism to some file types though - PDFs and .md files for example. Other file formats that you upload are stored and made available to Code Interpreter but are not embedded for vector search.
- IanCal 3y ago
  I've had good success putting code examples in a single txt file for our custom framework, and it seems to use that neatly for generating code. I'm surprised you've not had much success with them, I gave an assistant my wife's PhD thesis and while the API was working initially it seemed alright.
- wddkcs 3y ago
  Does that error message really confirm qdrant for Chat? It's just failing to index a file called 'qdrant', and I don't see any further proof offered in that thread.
  - simonw 3y ago
    That's good enough evidence for me. They're clearly running a vector store (you can get further hints at that by spying on the JSON using browser DevTools). Qdrant is a very good vector store - it's powering all of the new Twitter features as of a few weeks ago. Seems much more likely to me that they're using Qdrant than this is a weird error message coincidence.
- jjwiseman 3y ago
  Interestingly, when I used an .md extension, GPT would write python code to try to pull parts out to answer queries (which worked miserably), but when I used .txt (for the same files), it seemed to put it in the vector store.
- mmq 3y ago
  Actually msft CEO mentioned in his presentation that OAI moved to Azure's vector search and AI search services for ChatGPT.
- saliagato 3y ago
  I think they are running their RAG on Microsoft's Azure Cognitive Search. Satya mentioned something about it in the yesterday Microsoft Ignite keynote
NathanielLovin 3y ago
If the file is short, they put the full text into the prompt. If it's longer, they use some sort of RAG with qdrant, it appears to be top-1 with context expansion, but nobody's knows for sure how they're doing the chunking.
avereveard 3y ago
I've seen gpts with a SQLite file that is loaded by the code interpreter to provide structured data as needed I've seen gpts playing games with code that was attached to the prompt generating the turn results I've seen gpt using API for converting YouTube to text to provide summaries There's a lot of things that can be done, even if it's not the most dev friendly experience

simonw 3y ago

This is undocumented (frustrating) but it looks like it's chunking them, running embeddings on the chunks and storing the results in a https://qdrant.tech/ https://qdrant.tech/ vector database. We know it's Qdrant because an error message leaked that detail: https://twitter.com/altryne/status/1721989500291989585 https://twitter.com/altryne/status/1721989500291989585 It only applies that mechanism to some file types though - PDFs and .md files for example. Other file formats that you upload are stored and made available to Code Interpreter but are not embedded for vector search.

IanCal 3y ago
I've had good success putting code examples in a single txt file for our custom framework, and it seems to use that neatly for generating code. I'm surprised you've not had much success with them, I gave an assistant my wife's PhD thesis and while the API was working initially it seemed alright.
wddkcs 3y ago
Does that error message really confirm qdrant for Chat? It's just failing to index a file called 'qdrant', and I don't see any further proof offered in that thread.
- simonw 3y ago
  That's good enough evidence for me. They're clearly running a vector store (you can get further hints at that by spying on the JSON using browser DevTools). Qdrant is a very good vector store - it's powering all of the new Twitter features as of a few weeks ago. Seems much more likely to me that they're using Qdrant than this is a weird error message coincidence.
jjwiseman 3y ago
Interestingly, when I used an .md extension, GPT would write python code to try to pull parts out to answer queries (which worked miserably), but when I used .txt (for the same files), it seemed to put it in the vector store.
mmq 3y ago
Actually msft CEO mentioned in his presentation that OAI moved to Azure's vector search and AI search services for ChatGPT.
saliagato 3y ago
I think they are running their RAG on Microsoft's Azure Cognitive Search. Satya mentioned something about it in the yesterday Microsoft Ignite keynote

NathanielLovin 3y ago

If the file is short, they put the full text into the prompt. If it's longer, they use some sort of RAG with qdrant, it appears to be top-1 with context expansion, but nobody's knows for sure how they're doing the chunking.

avereveard 3y ago

I've seen gpts with a SQLite file that is loaded by the code interpreter to provide structured data as needed I've seen gpts playing games with code that was attached to the prompt generating the turn results I've seen gpt using API for converting YouTube to text to provide summaries There's a lot of things that can be done, even if it's not the most dev friendly experience