3 ms·
What is the current state of "correspondence between fed text and output text"? I.e. how much, when fed in the training e.g. that "Spain invested 6000$ in Colum
by mdp2021 3y ago
What is the current state of "correspondence between fed text and output text"? I.e. how much, when fed in the training e.g. that "Spain invested 6000$ in Columbus' first voyage", LLMs will repeat that notion exactly?
This, without taking into account reasoning and consistence. And already this notion that I picked randomly is not without issues: dollars how computed? And, it is not difficult to state that Columbus reached the Caribbeans in 1492; more complex to "decide" the year of the siege of Troy out of the many dates proposed.
But already at the simplified level of determined clear notions: if LLMs are told that "A is B", and in absence of inconsistency in the training corpus, what is the failure rate (i.e. then outputting something critically different)?
> ways to check up
Some LLMs work as search engines, outputting not just their tentative answer but linked references. A reasonably safe practice at this stage is to use LLMs that way: ask then use the output to check the reference.