4 ms·
Interesting, but I am a bit disappointed that this release doesn't include fine-tuning on an enterprise corpus of documents. This only looks like a slightly mor
by fdeage 3y ago
Interesting, but I am a bit disappointed that this release doesn't include fine-tuning on an enterprise corpus of documents. This only looks like a slightly more convenient and privacy-friendly version of ChatGPT. Or am I missing something?
- idopmstuff 3y agoAt the bottom, in their coming soon section: "Customization: Securely extend ChatGPT’s knowledge with your company data by connecting the applications you already use"
- fdeage 3y agoI saw it, but it only mentions "applications" (whatever that means) and not bare documents. Does this mean companies might be able to upload, say, PDFs, and fine-tune the model on that?
- idopmstuff 3y agoYeah, I'll be curious to see what it means by this. Could be a few things, I think: - Codebases - Documents (by way of connection to your Box/SharePoint/GSuite account) - Knowledgebases (I'm thinking of something like a Notion here) I'm really looking forward to seeing what they come up with here, as I think this is a truly killer use case that will push LLMs into mainstream enterprise usage. My company uses Notion and has an enormous amount of information on there. If I could ask it things like "Which customer is integrated with tool X" (we keep a record of this on the customer page in Notion) and get a correct response, that would be immensely helpful to me. Similar with connecting a support person to a knowledgebase of answers that becomes incredibly easy to search.
- mediaman 3y agoPretty unlikely. Generally you don't use fine-tuning for bare documents. You use retrieval augmented generation, which usually involves vector similarity search. Fine-tuning isn't great at learning knowledge. It's good at adopting tone or format. For example, a chirpy helper bot, or a bot that outputs specifically formatted JSON. I also doubt they're going to have a great system for fine-tuning. Successful fine-tuning requires some thought into what the data looks like (bare docs won't work), at which point you have technical people working on the project anyway. Their future connection system will probably be in the format of API prompts to request data from an enterprise system using their existing function fine-tuning feature. They tried this already with plugins, and they didn't work very well. Maybe they'll come up with a better system. Generally this works better if you write your own simple API for it to interface with which does a lot of the heavy lifting to interface with the actual enterprise systems, so the AI doesn't output garbled API requests so much.
- kenjackson 3y agoWhen I first started working with GPT I was disappointed in this. I thought like the previous commentor that I could fine tune by adding documents and it would add it to the "knowledge" of GPT. Instead I had to do what you suggest is vector similarity search, and add the relevant text to the prompt. I do think an open line of research is some way for users to just add arbitrary docs in an easy way to the LLM.
- fdeage 3y agoYes, this would definitely be a game changer for almost all companies. Considering how huge the market is, I guess it's pretty difficult to do, or it would be done already. I certainly don't expect a nice drag-and-drop interface to put my Office files and then ask questions about it coming in 2023. Maybe 2024?
- tempestn 3y agoThat would be the absolute game-changer. Something with the "intelligence" of GPT-4, but it knows the contents of all your stuff - your documents, project tracker, emails, calendar, etc. Unfortunately even if we do get this, I expect there will be significant ecosystem lock-in. Like, I imagine Microsoft is aiming for something like this, but you'd need to use all their stuff.
- r_thambapillai 3y agoThere are great tools that do this already in a support-multiple-ecosystems kind of way! I'm actually the CEO of one of those tools: Credal.ai - which lets you point-and-click connect accounts like O365, Google Workspace, Slack, Confluence, e.t.c, and then you can use OpenAI, Anthropic etc to chat/slack/teams/build apps drawing on that contextual knowledge: all in a SOC 2 compliant way. It does use a Retrieval-Augmented-Generation approach (rather than fine tuning), but the core reason for that is just that this tends to actually offer better results for end users than fine tuning on the corpus of documents anyway! Link: https://www.credal.ai/ https://www.credal.ai/
- xyst 3y agoGreat now chatgpt can train on outdated documents from the 2000s, provide more confusion to new people, and give us more headaches
- grrowl 3y agoAzure-hosted GPT already lets you "upload your own documents" in their playground; it seems to be similar to how ChatGPT GPT-4 Code Interpreter handles file uploads.
- BoorishBears 3y agoYou don't fine-tune on a corpus of documents to give the model knowledge, you use retrieval. They support uploading documents to it for that via that code interpreter, and they're adding connectors to applications where the documents live, not sure what more you're expecting.
- fdeage 3y agoYes, but what if they are very large documents that exceed the maximum context size, say, a 200-page PDF? In that case won't you be forced to do some form of fine-tuning, in order to avoid a very slow/computationally expensive on-the-fly retrieval? Edit: spelling
- Difwif 3y agoTypical retrieval methods break up documents into chunks and perform semantic search on relevant chunks to answer the question.
- BoorishBears 3y agoFine-tuning the LLM in the way that you're mentioning is not even an option: as a practical rule fine-tuning the LLM will let you do style transfer, but you knowledge recall won't improve (there are edge cases, but none apply to using ChatGPT) That being said you can use fine tuning to improve retrieval, which indirectly improves recall. You can do things like fine tune the model you're getting embeddings from, fine tune the LLM to craft queries that better match a domain specific format, etc. It won't replace the expensive on-the-fly retrieval but it will let you be more accurate in your replies. Also retrieval can be infinitely faster than inference depending on the domain. In well defined domains you can run old school full text search and leverage the LLMs skill at crafting well thought out queries. In that case that runs at the speed of your I/O.
- jrpt 3y agoWe have >200 page PDFs at https://docalysis.com/ https://docalysis.com/ and there's on-the-fly retrieval. It's not more computationally expensive than something like searching one's inbox (I'd image you have more than 200 pages worth of emails in your inbox).
- gopher_space 3y agoRetrieval Augmented Generation would be something to check out. There was a good intro on the subject posted here a week or 3 ago.
- internet101010 3y agoThis is one of the reasons we decided to go with Databricks. Embed all the things for RAG during ETL.