3 ms·

This is a good point. While LLMs being incapable of reliably doing a simple task that’s been doable by computers since the punch card days is an important consi

This is a good point. While LLMs being incapable of reliably doing a simple task that’s been doable by computers since the punch card days is an important consideration for anyone that might be thinking about using them for anything other than as a toy, this fact is uninteresting because of Reasons

space_fountain 2y ago
LLMs can clearly solve problems that computers up to now couldn't. They can't solve all problems and this should definitely be a cautionary note to anyone who wants to use them as an artificial general intelligence, but this take seems no different to someone looking at a punchcard computer and going, it can't even recognize typos or categorize images, what good is this? We've already had human computers who can do everything these can do, and can recognize images and notice typos
- evilduck 2y ago
  Also humans would revert to explicitly using an algorithm and external storage like a sheet of paper with tally marks or a spreadsheet or even a computer program if you scale the question up to a full sheet of text or a whole book or a collection of books (we probably do it at a single word size too, but it's more intuitive than explicit behavior for most folks when the count sum is around 8 or less). LLMs can't effectively execute algorithms similarly in their context, nor can they memorize new data or facts it was given without providing it tools like function calling or embeddings. If you give LLMs tool calling and storage mechanisms then counting letters in words becomes pretty damn reliable.
- jrflowers 2y ago
  > going, it can't even recognize typos or categorize images, what good is this? No one said that LLMs aren’t good for anything. I pointed out — in response to another poster downplaying mention of a well-known and undisputed limitation that LLMs often have — that it is valid to consider these well-known and undisputed limitations if one is considering using them for anything other than a toy. It is downright silly to discourage discussion of well-known and undisputed limitations! The only reason for that can only be entirely emotional as there is genuinely nothing tangible to be gained by being steadfast in silence about a fact that isn’t up for debate.
  - space_fountain 2y ago
    I think somehow there were a series of miscommunications. This kind of sub token manipulation task is hard for an LLM for somewhat predictable reasons. Knowing those limitations are important, but don't come up too often in practical circumstances. Outside of contrived examples counting the number of letters in a long word is pretty rare. I took your response to be arguing against a message I'd read to be saying something like the above. Especially when you basically seemed to be saying that limitations like this are important in everything but toy applications. It's uninteresting because it doesn't point towards larger problems with their use in the kind of application they're being used for and are intended for unlike prompts that point to weakness in logic or propensity to hallucinate.
Nevermark 2y ago
Try counting the number of your red retina cells that are firing while you look at a painting. Don’t need to be exact as firing is statistical, just give us a good average. Hard? You can’t count? Computers count pixels no problem. So weird you can’t. Dementia? Not an AGI? /h —- This is what is happening. Here are the “Reasons”. In your vision system, the raw information from individual retina signals is munged into a different representation before reaching a level where you have flexible processing. Likewise, in LLMs, letters are munged into tokens before LLMs “see” them. When they sometimes get that “simple” question right, it’s actually a bit of an amazing feat. Given how they are constructed. —- Now try counting R’s as you read at a normal rate, or listen to someone speak. You can’t do that either, during normal processing. When we add spelling to LLMs training examples, they will do it easily. Just as you learned to do it, only after special lessons, after you had already learned to listen and speak. Spelling is its own special practiced skill, in humans and LLMs.
- jrflowers 2y ago
  > Try counting the number of your red retina cells that are firing during while you look at a painting. This analogy makes sense because everybody could count their red retina cells until a couple years ago when the new painting paradigm arose, and also counting red retinal cells is a good analogy for being able to see simple objects that have always been distinguishable. It is fascinating how tapping the “Do Not Use LLMs For Computation If The Results Need To Be Reliably Better Than A Random Output” sign invites explanations of why that fact is actually Cool and Good
  - Nevermark 2y ago
    Ask anyone who has not specifically learned to spell, to count R’s while you speak. You learned to listen and speak words before you could spell. Imagine if nobody had actually shown you written words? Or they were speaking another dialect but expecting you to count R’s in standard English? LLMs are not trained on words in the form of letters. They process and generate the words in the form of tokens. Pre- and post-processing systems converts letters to tokens and the reverse, without their ability to access that processing. Spelling, for both us and LLMs, requires specific training/lessons. > It is fascinating how tapping the “Do Not Use LLMs For Computation If The Results Need To Be Reliably Better Than A Random Output” sign invites explanations of why that fact is actually Cool and Good Also fascinating: People who hallucinate/confabulate ridiculous straw man rationales for people they disagree with, unaware they are filling in gaps in their knowledge regarding other people’s actual reasoning and the actual subject at hand. So LLM! Such unreliable processing! Perhaps, start posting a reliability disclaimer?

space_fountain 2y ago

LLMs can clearly solve problems that computers up to now couldn't. They can't solve all problems and this should definitely be a cautionary note to anyone who wants to use them as an artificial general intelligence, but this take seems no different to someone looking at a punchcard computer and going, it can't even recognize typos or categorize images, what good is this? We've already had human computers who can do everything these can do, and can recognize images and notice typos

evilduck 2y ago
Also humans would revert to explicitly using an algorithm and external storage like a sheet of paper with tally marks or a spreadsheet or even a computer program if you scale the question up to a full sheet of text or a whole book or a collection of books (we probably do it at a single word size too, but it's more intuitive than explicit behavior for most folks when the count sum is around 8 or less). LLMs can't effectively execute algorithms similarly in their context, nor can they memorize new data or facts it was given without providing it tools like function calling or embeddings. If you give LLMs tool calling and storage mechanisms then counting letters in words becomes pretty damn reliable.
jrflowers 2y ago
> going, it can't even recognize typos or categorize images, what good is this? No one said that LLMs aren’t good for anything. I pointed out — in response to another poster downplaying mention of a well-known and undisputed limitation that LLMs often have — that it is valid to consider these well-known and undisputed limitations if one is considering using them for anything other than a toy. It is downright silly to discourage discussion of well-known and undisputed limitations! The only reason for that can only be entirely emotional as there is genuinely nothing tangible to be gained by being steadfast in silence about a fact that isn’t up for debate.
- space_fountain 2y ago
  I think somehow there were a series of miscommunications. This kind of sub token manipulation task is hard for an LLM for somewhat predictable reasons. Knowing those limitations are important, but don't come up too often in practical circumstances. Outside of contrived examples counting the number of letters in a long word is pretty rare. I took your response to be arguing against a message I'd read to be saying something like the above. Especially when you basically seemed to be saying that limitations like this are important in everything but toy applications. It's uninteresting because it doesn't point towards larger problems with their use in the kind of application they're being used for and are intended for unlike prompts that point to weakness in logic or propensity to hallucinate.

Nevermark 2y ago

Try counting the number of your red retina cells that are firing while you look at a painting. Don’t need to be exact as firing is statistical, just give us a good average. Hard? You can’t count? Computers count pixels no problem. So weird you can’t. Dementia? Not an AGI? /h —- This is what is happening. Here are the “Reasons”. In your vision system, the raw information from individual retina signals is munged into a different representation before reaching a level where you have flexible processing. Likewise, in LLMs, letters are munged into tokens before LLMs “see” them. When they sometimes get that “simple” question right, it’s actually a bit of an amazing feat. Given how they are constructed. —- Now try counting R’s as you read at a normal rate, or listen to someone speak. You can’t do that either, during normal processing. When we add spelling to LLMs training examples, they will do it easily. Just as you learned to do it, only after special lessons, after you had already learned to listen and speak. Spelling is its own special practiced skill, in humans and LLMs.

jrflowers 2y ago
> Try counting the number of your red retina cells that are firing during while you look at a painting. This analogy makes sense because everybody could count their red retina cells until a couple years ago when the new painting paradigm arose, and also counting red retinal cells is a good analogy for being able to see simple objects that have always been distinguishable. It is fascinating how tapping the “Do Not Use LLMs For Computation If The Results Need To Be Reliably Better Than A Random Output” sign invites explanations of why that fact is actually Cool and Good
- Nevermark 2y ago
  Ask anyone who has not specifically learned to spell, to count R’s while you speak. You learned to listen and speak words before you could spell. Imagine if nobody had actually shown you written words? Or they were speaking another dialect but expecting you to count R’s in standard English? LLMs are not trained on words in the form of letters. They process and generate the words in the form of tokens. Pre- and post-processing systems converts letters to tokens and the reverse, without their ability to access that processing. Spelling, for both us and LLMs, requires specific training/lessons. > It is fascinating how tapping the “Do Not Use LLMs For Computation If The Results Need To Be Reliably Better Than A Random Output” sign invites explanations of why that fact is actually Cool and Good Also fascinating: People who hallucinate/confabulate ridiculous straw man rationales for people they disagree with, unaware they are filling in gaps in their knowledge regarding other people’s actual reasoning and the actual subject at hand. So LLM! Such unreliable processing! Perhaps, start posting a reliability disclaimer?