4 ms·

For all that LLMs produce good performance, it's still just predictive texting. If you can get the hallucination rate on that down to 0% without using anything

For all that LLMs produce good performance, it's still just predictive texting. If you can get the hallucination rate on that down to 0% without using anything else, I'll be extremely shocked. Now, layer a few sanity checks on top of an LLM, especially some clever thing we haven't invented yet, and I'll totally believe it - the task is absolutely doable, I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources.

ben_w 2y ago
Conditional on it still sounding like you mean Transformers rather than LLMs, we're pretty close to agreement. But even Transformers really are not just preditive text. IIRC the original Google usage, Attention is All You Need era, was for translation; and while I would indeed characterise the first few OpenAI/GPT models as "autocomplete on steroids", that changed with InstructGPT, which was the first time I saw them transforming requests into actions, in the form of creating a very simple web game. > I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources. I currently think there is no such thing as "knowledge" in reality, that such a state is as unrealisable as counting to infinity, that all we can really have are beliefs of varying certainty; in this regard, 100% can never happen in any system including humans — but also, I wouldn't say an AI is "hallucinating" if the error rate was similar to that of a human. Likewise, I find it really weird how a neural network with the complexity of a mid-sized rodent is able to transform prompts in the most used languages into mostly-correct source code in most programming languages — this is not a thing I would have expected, given the observable lack of employment opportunities for rodents* in software engineering departments. I could be wrong about both, of course. * other than furries, who are everywhere ;)

ben_w 2y ago

Conditional on it still sounding like you mean Transformers rather than LLMs, we're pretty close to agreement. But even Transformers really are not just preditive text. IIRC the original Google usage, Attention is All You Need era, was for translation; and while I would indeed characterise the first few OpenAI/GPT models as "autocomplete on steroids", that changed with InstructGPT, which was the first time I saw them transforming requests into actions, in the form of creating a very simple web game. > I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources. I currently think there is no such thing as "knowledge" in reality, that such a state is as unrealisable as counting to infinity, that all we can really have are beliefs of varying certainty; in this regard, 100% can never happen in any system including humans — but also, I wouldn't say an AI is "hallucinating" if the error rate was similar to that of a human. Likewise, I find it really weird how a neural network with the complexity of a mid-sized rodent is able to transform prompts in the most used languages into mostly-correct source code in most programming languages — this is not a thing I would have expected, given the observable lack of employment opportunities for rodents* in software engineering departments. I could be wrong about both, of course. * other than furries, who are everywhere ;)