4 ms·

I think that the ChatGPT model (at least chatgpt-3.5-turbo) has gotten impressively better at this. - In my most recent tests, it will tell you when the data y

I think that the ChatGPT model (at least chatgpt-3.5-turbo) has gotten impressively better at this. - In my most recent tests, it will tell you when the data you've provided doesn't match the task (instead of inventing an answer out of thin air). - It will also add (unprompted) comments/notes before or after the result to disclaim a plausible reason why certain choices have been made or the answer isn't complete. You have to take into account that not everyone wants the model to not hallucinate. There is a lot of competing pressure: - Some people would like the model to say "As an AI model trained by OpenAI, I am not qualified to provide an answer because this data is not part of my training set" or something similar, because they want it to only talk when it's sure of the data. (I personally think this use case - using LLMs as search engines/databases of truth - is deeply flawed and not what LLMs are for ; but for a large enough GPT-n, it would work perfectly fine. There is a model size where the model would indeed contain the entire uncompressed Internet, after all) - Some people want the model to never give such a denial, and always provide an answer in the require format, even if it requires the model to "bullshit" or improvise a bit. An example is, if that as a business user I provide the model with a blog article and ask for metadata in a JSON structure, I want the model to NEVER return "As an AI model..." and ALWAYS return valid JSON, even if the metadata is somewhat shaky or faulty. Most apps are more tolerant to BS than they are to empty/invalid responses. That's the whole reason behind all of those "don't reply out of character/say you are an AI" prompts you see floating around (which, in my experience, are completely useless and do not affect the result one bit) So the reinforcement is constantly going in those opposite directions.

vidarh 4y ago
I agree it's better, but it's still awful at it. But I think it's one of the low hanging fruits to improve, though, just by reinforcing the limits of what it knows. With respect to the competing draws here, I'm not sure they necessarily compete that much. E.g. being able to ask it to speculate but explain justifications or being able to provide a best guess or being able to ask it to just be as creative as it wants, would be sufficient. Alternatively one that knows how to embed indication of what it knows to be true vs. what is speculation or pure fiction. Of course we can "just" also train different models with different levels of / types of reinforced limits. Having both a "anything goes" model and a constrained version that knows what it does know might both be useful for different things.

vidarh 4y ago

I agree it's better, but it's still awful at it. But I think it's one of the low hanging fruits to improve, though, just by reinforcing the limits of what it knows. With respect to the competing draws here, I'm not sure they necessarily compete that much. E.g. being able to ask it to speculate but explain justifications or being able to provide a best guess or being able to ask it to just be as creative as it wants, would be sufficient. Alternatively one that knows how to embed indication of what it knows to be true vs. what is speculation or pure fiction. Of course we can "just" also train different models with different levels of / types of reinforced limits. Having both a "anything goes" model and a constrained version that knows what it does know might both be useful for different things.