3 ms·
LLMs have hard time to reason about adjacent topics. I have a favorite question to a LLM that can be (and is) learned from papers from 2010-2012 (well before a
by thesz 2y ago
LLMs have hard time to reason about adjacent topics.
I have a favorite question to a LLM that can be (and is) learned from papers from 2010-2012 (well before advent of LLMs) and I am keeping asking it for two years now.
LLMs are able to cite relevant papers with "word-by-word" accuracy, they remember them quite well. Every paper on the subject has all relevant definitions in them. Yet, LLMs cannot combine adjacent definitions to come up with the ultimate solution to my question.
The question: "implement blocked clause decomposition in Haskell."
Google "blocked clause decomposition" for papers on subject.
Have fun.
Over time, LLM's seem to lose the ability to even approach the solution to this question in a first answer. They need more and more attention and correction nowadays.
I see it as a knowledge collapse mentioned in paper I provided a link to. Instead of an answer I get a gamified pretense of a helping hand and we all do.