5 ms·
The TLDR of the article is essentially that any time a ChatGPT conversation strays to the factual (in this case a list of data ethics books), the quality of the
by lsy 4y ago
The TLDR of the article is essentially that any time a ChatGPT conversation strays to the factual (in this case a list of data ethics books), the quality of the data (whether those books exist and are written by the stated authors) is extremely low.
Ultimately this is where ChatGPT and similar projects seem most likely to hit a wall. Text generators are statistical machines, but the relationship between statistics and truth is merely probabilistic and not identical. There is no way to train ChatGPT to say things that are true, because the number of true and false statements in the world are far too numerous and too specific to ever reliably delineate. At best, we will be stuck fact-checking ChatGPT using enormously complex knowledge bases and asking it to try again until something doesn't raise a red flag, at which point it may still not be true.
For most business purposes this is unacceptable. We will have machines that can generate untruths in a way that contravene common sense, so long as those untruths don't obviously contradict whatever local database is being consulted for grounding. And the mechanisms required to re-ground the GPT output even to that unhappy state will be so complex in their own right that businesses might as well just implement a similarly complex system to handle the issue directly, without calling out to a complex, licensed GPT. Not sure whether there is a way around this.
- visarga 4y agoOne approach to fixing factual errors is to use two rounds of LLM interaction. I forgot the name of the paper. Say you ask "What is the height of Everest?" 1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer 2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet 3. do a second pass to rewrite the answer with the LLM using search_snippet in the prompt Basically, the incorrect phrase candidate_answer is very good at matching the correct answer in search engines. It is like a template tuned to extract the desired facts. A search engine check could also verify when the searched fact has no references.
- ptx 4y agoHow would you apply this approach to the article's example, the response to the question "Who is Daragh O Brien from Castlebridge", where I count at least 15 separate statements of fact? Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work. Also: Is the generated response really "very good at matching the correct answer"? I suppose it would work because the search engine's language processing cancels out the useless parts that were generated by the AI (sort of a "human ABI", analogous to the C ABI?) but a more direct query (e.g. "height of everest") would likely be just as effective.
- lsy 4y agoYes, this is the essential issue. Any system competent to fully ground and check factual statements in a stream of arbitrary text will be phenomenally more complex than the original LLM, and usually will be able to answer queries directly, at which point one wonders what the LLM is adding. I think at best, if we can somehow identify all factual statements that need to be cross-referenced and then offload them to a knowledge base (dubious), we are left this kind of mad-libs connective flow that the LLM has created which approximates the essay style of a human writer. I'm not certain that has much practical value besides allowing for a form of undetectable plagiarism to be published as though it were free-form writing.
- kerpotgh 4y ago[dead]