3 ms·

I'm not sure if, in the case of humans, that example falls on making up facts for the logical analysis (as you imply) or in the assumed common context for langu

by j1elo 3y ago

I'm not sure if, in the case of humans, that example falls on making up facts for the logical analysis (as you imply) or in the assumed common context for language processing (the "tool" we socially use to make language efficient: by not having to explain absolutely everything). If you told me that thing about the box, I'd assume you meant a box that doesn't leak, not because of making that fact up in my brain, but because you're verbally describing a system to me, and it's implied you'll tell me the most relevant features of the system in order to transmit the idea of it from your mind to mine. Otherwise I'd be right to complain that you are maliciously hiding important information in your description. Problem is, of course, we don't have a common preestablished language context with an LLM, at least not any more than what is the prevalent one in its training materials ("Ask" vs. "Guess" cultures, etc).

vidarh 3y ago
So you are making things up. Including making up rules about which things it's ok for you to make up. They might be reasonable things to make up, but you're still inferring them to plug holes in the description based on what seems to be the most probably interpretation. And that is pretty much exactly what an LLM does: It predicts the most likely item of data to fill in. That their training is as of yet insufficient to make them predict reasonable things in all situations should be far less surprising than how quickly we've gotten to a point where we can communicate well enough with them that these conversations are meaningful at all. One of the fascinating things about these discussions to me is that they reveal an astonishing amount of assumptions about human thought processes that appear to be just marginally above LLM hallucinations in rigor.
- j1elo 3y ago
  > So you are making things up. Including making up rules about which things it's ok for you to make up. I guess yeah, that's correct :) But it's also not an artifact, but within the very definition of human interaction. We need to be able to plug holes. Otherwise conversation evolves into a list of guarding caveats after each and every message. OTOH Twitter is a great example of how different cultures develop different assumptions about what is reasonable to fill the gaps in with. It's notorious for how people just take whatever wrong and unintended meaning they want from controversial tweets. Some times on purpose, granted, but lots of other times not really. I concur it's a very interesting topic!
  - vidarh 3y ago
    > We need to be able to plug holes. Otherwise conversation evolves into a list of guarding caveats after each and every message. Absolutely. So the big question is to what extent what LLMs do is this, but without (yet) having been sufficiently trained on what is reasonable to fill in unknown gaps with, vs. whether there is a bigger issue. I'm pretty hopeful that it's down to more training on when "I don't know" should be the most likely output, but I think we really don't know until more work has been done on it...

by j1elo 3y ago

vidarh 3y ago

So you are making things up. Including making up rules about which things it's ok for you to make up. They might be reasonable things to make up, but you're still inferring them to plug holes in the description based on what seems to be the most probably interpretation. And that is pretty much exactly what an LLM does: It predicts the most likely item of data to fill in. That their training is as of yet insufficient to make them predict reasonable things in all situations should be far less surprising than how quickly we've gotten to a point where we can communicate well enough with them that these conversations are meaningful at all. One of the fascinating things about these discussions to me is that they reveal an astonishing amount of assumptions about human thought processes that appear to be just marginally above LLM hallucinations in rigor.

j1elo 3y ago
> So you are making things up. Including making up rules about which things it's ok for you to make up. I guess yeah, that's correct :) But it's also not an artifact, but within the very definition of human interaction. We need to be able to plug holes. Otherwise conversation evolves into a list of guarding caveats after each and every message. OTOH Twitter is a great example of how different cultures develop different assumptions about what is reasonable to fill the gaps in with. It's notorious for how people just take whatever wrong and unintended meaning they want from controversial tweets. Some times on purpose, granted, but lots of other times not really. I concur it's a very interesting topic!
- vidarh 3y ago
  > We need to be able to plug holes. Otherwise conversation evolves into a list of guarding caveats after each and every message. Absolutely. So the big question is to what extent what LLMs do is this, but without (yet) having been sufficiently trained on what is reasonable to fill in unknown gaps with, vs. whether there is a bigger issue. I'm pretty hopeful that it's down to more training on when "I don't know" should be the most likely output, but I think we really don't know until more work has been done on it...