3 ms·

> Search companies like Google have discovered that it is very important to modify search results in various countries based on court judgments, and OpenAI is g

> Search companies like Google have discovered that it is very important to modify search results in various countries based on court judgments, and OpenAI is going to find that the same is true for ChatGPT. ChatGPT will get things like this wrong, everyone knows that, but there needs to be a way to tell it, "Don't get THIS one wrong." This is very insightful, thank you. I'm extremely curious about how this will be achieved from the technical point of view. It's not clear to me that ChatGPT has the concept "a an american lawyer named X" (as opposed to just guessing the next token), so how will it be filtered out?

btilly 4y ago
They need to figure out how to do that. :-) I would suggest that they have a search database of things to avoid, and a specialized ChatGPT that can find things in that database. Any statement it wants to make is passed by the specialized watchdog with, "Is there anything here that we can't say?" If the specialized watchdog says yes and creates a citation to the rule. Then a tool pulls the rule ACTUALLY in the database up, and ChatGPT compares with the statement to say whether it REALLY applies (without this step the watchdog could hallucinate rules that don't exist!), and then corrects the original ChatGPT before it actually says anything. This probably would work fairly well in practice.
sacred_numbers 4y ago
I can think of a few ways: 1. The ChatGPT web search plugin becomes standard protocol for every prompt. If you ask a factual question ChatGPT will first look up an answer with a search engine, then use the results to craft an answer. This could also be implemented using a knowledge database created by scraping the web that is occasionally updated rather than actually performing a web search every time. This also allows OpenAI to shift the blame to whatever source it found rather than ChatGPT if it does find libelous content. 2. OpenAI will just add a disclaimer to every response (or at least every response that appears to be asking a factual question) saying that asking factual questions is unreliable. 3. If there is a blacklist of topics that require a disclaimer or cause the LLM to refuse to answer, it can generate an answer, check it against the blacklist using embeddings/semantic search and either re-generate an answer or generate a refusal before showing the answer to the user. My best guess is that it will be a combination of 1 and 2. I have always maintained that LLMs are very unlikely to develop into AGI on their own, but are very likely to be a critical piece of an AGI. The most recent research into scaling laws (Chinchilla, Llama) finds significant improvements from scaling data size much farther than parameter size, so memorizing facts within the parameters will become less and less feasible. This is actually ideal, though, since you want your model parameters to encode language and reasoning patterns, not memorize facts. If it's memorizing facts you either need more data or better (i.e. deduplicated) data. I'm not an expert, though, and I'm too lazy for a research review, so please don't sue me for libel if my facts are out of date.

btilly 4y ago

They need to figure out how to do that. :-) I would suggest that they have a search database of things to avoid, and a specialized ChatGPT that can find things in that database. Any statement it wants to make is passed by the specialized watchdog with, "Is there anything here that we can't say?" If the specialized watchdog says yes and creates a citation to the rule. Then a tool pulls the rule ACTUALLY in the database up, and ChatGPT compares with the statement to say whether it REALLY applies (without this step the watchdog could hallucinate rules that don't exist!), and then corrects the original ChatGPT before it actually says anything. This probably would work fairly well in practice.

sacred_numbers 4y ago

I can think of a few ways: 1. The ChatGPT web search plugin becomes standard protocol for every prompt. If you ask a factual question ChatGPT will first look up an answer with a search engine, then use the results to craft an answer. This could also be implemented using a knowledge database created by scraping the web that is occasionally updated rather than actually performing a web search every time. This also allows OpenAI to shift the blame to whatever source it found rather than ChatGPT if it does find libelous content. 2. OpenAI will just add a disclaimer to every response (or at least every response that appears to be asking a factual question) saying that asking factual questions is unreliable. 3. If there is a blacklist of topics that require a disclaimer or cause the LLM to refuse to answer, it can generate an answer, check it against the blacklist using embeddings/semantic search and either re-generate an answer or generate a refusal before showing the answer to the user. My best guess is that it will be a combination of 1 and 2. I have always maintained that LLMs are very unlikely to develop into AGI on their own, but are very likely to be a critical piece of an AGI. The most recent research into scaling laws (Chinchilla, Llama) finds significant improvements from scaling data size much farther than parameter size, so memorizing facts within the parameters will become less and less feasible. This is actually ideal, though, since you want your model parameters to encode language and reasoning patterns, not memorize facts. If it's memorizing facts you either need more data or better (i.e. deduplicated) data. I'm not an expert, though, and I'm too lazy for a research review, so please don't sue me for libel if my facts are out of date.