7 ms·

Caveat lector: I'm analysing this as a linguist. Not as a programmer. (Unless you count a bunch of bash scripts as "programming".) >Let us start first with des

Caveat lector: I'm analysing this as a linguist. Not as a programmer. (Unless you count a bunch of bash scripts as "programming".) >Let us start first with describing what we call the “missing text phenomenon” (MTP), that we believe is at the heart of all challenges in natural language understanding. The missing text is not the heart of the challenge; it's just a surface issue. The actual problem is deeper - machines treat words as if they got intrinsic meaning, when the meaning is actually negotiated between speaker and hearer at the moment of the utterance. This might sound complicated so, as an example, let's say I told you: "I got a cat. That wug sleeps on my bed." You don't know what's a "wug" (it's a nonsensical word BTW), or why I'm calling my cat a "wug". So you're missing info, just like the machine, right? And yet you're able to parse the second sentence just fine, based on the first. And if we keep chatting about my cats, any mention of the word "wug" will represent the feline from the picture. We effectively negotiated the word "wug" to represent that cat. A machine does not do that; instead it'll give words intrinsic, non-negotiable meanings. It won't have "wug" in its dataset, so it won't understand that sentence. I don't believe that machine learning will solve this issue _on its own_, but it could once we're able to simulate that negotiation of meanings. >Was Aristotle really Aristotle? If I saw this sentence "in the wild", I'd immediately interpret it as someone questioning the historicity of Aristotle, and claiming his works were made by someone else. It shows that the question is not as redundant as it might look like. > The trophy did not fit in the suitcase because it was too // 1a. small // 1b. big This specific example doesn't show it, but that ambiguity is an issue even for human beings, as shown by "I dropped the trophy on the table. Now it's broken." (What's broken - the trophy, or the table?) In fact, it's such a big issue that languages often have a bunch of resources to minimise those situations, as gender agreement and distal/proximal pronouns.

shoo 5y ago
> A machine does not do that; instead it'll give words intrinsic, non-negotiable meanings. It won't have "wug" in its dataset, so it won't understand that sentence. > I don't believe that machine learning will solve this issue _on its own_, but it could once we're able to simulate that negotiation of meanings. Fascinating example. Personally, I don't reckon there's any inherent reason that forces all machines to give words intrinsic non-negotiable meanings. The machine is merely fitting or predicting based on some model that someone has cooked up, subject to some particular modelling assumptions. If someone can cook up a new model then the machine will be able to negotiate wug-meanings too. (perhaps some kind of hierarchical or parallel model that incorporates a baseline model assuming intrinsic meanings learned from training data, but overlays some additional wug-negotiation subsystem that can kick in when it encounters a new word or starts to infer that a word is being used in a rather nonstandard way). In the world of parsing text for programming languages (which should have well-defined grammars, hopefully an incredibly well defined and easy problem, unless the language is insane/c++) in order for a computer to output helpful error messages when a programmer types something incoherent, a helpful parser needs to be carefully built to handle all kinds of weird crap and attempt to make some kind of sense out of it ("there's no 'ffo' defined. Did you mean 'foo' or this other thing 'bar' that is sitting right here defined the current context?").
- lvxferre 5y ago
  The way I see this working is by adding a second layer of machine learning to the process. The first layer would work as usual (associate words with utterance-independent meanings), but the second layer would compare those meanings with the utterance in question, shave most (or all) of them out, and either add a new meaning or reinforce an old one. But again, I'm no programmer. I might be saying something that is already done, or unfeasible for some reason.
  - shoo 5y ago
    (this is also not my field) I was thinking about this a little more: In some cases (such as your excellent wug example) it might be possible to heuristically associate the word "wug" to a probable contextual meaning "your cat" from a shortlist of candidate objects ("you (conversing)", "me (also conversing)", "cat", "bed") without understanding that cats are things that sleep on beds, and some kind of educated guess that "that wug" doesn't refer to the speaker themselves! In other cases, figuring this out might require a fair bit of side knowledge about the meanings of what is being discussed, and how they relate to each other, to almost logically deduce what entity is being labelled, given knowledge of how those entities work. That might be dramatically harder to get get a machine to do well. Perhaps a machine being able to to this would be a leap forward in machine intelligence from today's ML "stochastic parrots".
- tjs8rj 5y ago
  Who cares if the machine understands wug? Maybe the issue is more to do with trying to teach language learning as a end on its own. Nobody studies language without purpose, there’s always a next step. If you’re friend is telling you about their cat and what a wug it is, the goal is to conversate, make the other feel heard, entertain them, kill time until something next happens, impress them, and so on. Most of these don’t require even picking up on “wug”, they require understanding the other’s state of mind: “my new cat is so cute - I love them!”. Getting caught on solving wug seems like an uneccessary triviality, more important is the tone they say it with, they body language, your past interactions with them. You could form a proper reaction if you only heard every other word and had those other details. Humans don’t rely on deciphering “wug” - they don’t need to. The whole interaction is just negotiating mental states, intentions, and actions. Deciphering “wug” is unnecessary.
kumarvvr 5y ago
An awesome explanation. I guess then, for machine learning to succeed in natural language interpretation, we ought to have another layer of abstraction that learns meanings from contexts and the general situation. Extending on your idea, I think, not only the verbal context, but physical location / time of day / historical context / current events / social group etc all add their own flavor to the variety of meanings on single word can have. For example, if I am walking in a park and there is a strong gust of wind, I might say "whoa, that's a strong one". The listener, if physically present along with the speaker, will immediately get it. Adding language context is one challenge. Adding the context of environment is a completely different challange.
michannne 5y ago
> machines treat words as if they got intrinsic meaning, when the meaning is actually negotiated between speaker and hearer at the moment of the utterance. IMO, and I'm not a linguist, I've always felt we need to treat conversations as transactions, with each speaker acting as a party and continually updating some contract over time, with each speaker's goal to use conversational tactics i.e. clarification, questioning, answering a question, diverting the conversation, humor/jokes, lies, as a means to "guess" the contract the opposing parties are operating under, then using the best guess of the contract to later retrieve information relevant to the system when it has some confidence in the answers it can expect. Something like an exceptionally complex game mixed with financial system theory, like I "bet" that you're going to keep talking about your cat, and maybe that bet is wrong, so I can ask a question, "What do you mean by wug?", but the act of asking that question will cost me some points, maybe you can think of it as though we're at a secret club and you use a phrase meant to represent a key and I'm supposed to respond with the proper answer. Even though I'm asking a question, I'm giving you information that you can use to update your best guess of the contract that allows you to assume I'm not part of the in-group of the club and I shouldn't be allowed in. I've thought about a system designed like this for a bit but I suppose the biggest challenge has been how do you treat a conversation as a game considering neither party may "win" in any reasonable time frame. We could go years chatting and I could never fully predict what your answer might be because your usage of conversational tactics may be refined over years by speaking to multiple people of backgrounds similar to mine. So if I'm a cop, you might know not to admit to a crime when you figure out I'm a cop. Right now, I don't think ML can do that, not because it's impossible but because I think conversations are as difficult as predicting the stock market. Fortunately it seems solvable, not everyone is expected to win every conversation in their lifetime
visarga 5y ago
GPT-3 can figure out a nonsensical word and use it in the same example. The prompt is between quotes, everything else is generated. > "I got a cat. That wug sleeps on my bed." On every night. He’s really nice. But he isn’t small. > I got a dog. That wug sleeps at my house. On every night. She’s really nice. But she isn’t small. ~~~ Another one in dialogue format. > Human: I got a cat. That wug sleeps on my bed. > AI: It has a great personality. > Human: Define a wug based on the previous lines. > AI: A wug is a small, impulsive, and clumsy creature.
- gattilorenz 5y ago
  Not having access to GPT-3, can you prompt "Is a wug a cat?"
- goatlover 5y ago
  What sort of additional prompting would you need to do to get it to understand that cats aren't clumsy, but do enjoy knocking things over?
- lvxferre 5y ago
  Can GPT-3 figure out that a common word is being used outside its usual context, so its usual meanings don't apply? Let's say my example was instead "I got a cat. That little orc sleeps on my bed." I wonder how GPT-3 would deal with it. Maybe it would think "cat" and "orc" refer to different objects/beings? Or associate "cat" with features that clearly wouldn't fit a cat like "green", "humanoid", or "fantasy monster"?
  - goatlover 5y ago
    Clumsy already doesn't fit with cat, assuming GPT-3 is using wug for cat. Cats are more athletic than we are. They are very finely tuned hunters. But they need a lot of stimulation and so house cats will play a lot, and cause silly disturbances for humans.
abecedarius 5y ago
fwiw, I just pasted your example into GPT-3, ran it twice, and both times the continuation acted like 'wug' meant a cat. (Once as the cat's name, once generically.)
PeterisP 5y ago
> "I dropped the trophy on the table. Now it's broken." (What's broken - the trophy, or the table?) We could not solve this kind of problems (Winograd schemas) automatically a few years ago, but now ML techniques can solve them at about the same error rate as humans.

shoo 5y ago

> A machine does not do that; instead it'll give words intrinsic, non-negotiable meanings. It won't have "wug" in its dataset, so it won't understand that sentence. > I don't believe that machine learning will solve this issue _on its own_, but it could once we're able to simulate that negotiation of meanings. Fascinating example. Personally, I don't reckon there's any inherent reason that forces all machines to give words intrinsic non-negotiable meanings. The machine is merely fitting or predicting based on some model that someone has cooked up, subject to some particular modelling assumptions. If someone can cook up a new model then the machine will be able to negotiate wug-meanings too. (perhaps some kind of hierarchical or parallel model that incorporates a baseline model assuming intrinsic meanings learned from training data, but overlays some additional wug-negotiation subsystem that can kick in when it encounters a new word or starts to infer that a word is being used in a rather nonstandard way). In the world of parsing text for programming languages (which should have well-defined grammars, hopefully an incredibly well defined and easy problem, unless the language is insane/c++) in order for a computer to output helpful error messages when a programmer types something incoherent, a helpful parser needs to be carefully built to handle all kinds of weird crap and attempt to make some kind of sense out of it ("there's no 'ffo' defined. Did you mean 'foo' or this other thing 'bar' that is sitting right here defined the current context?").

lvxferre 5y ago
The way I see this working is by adding a second layer of machine learning to the process. The first layer would work as usual (associate words with utterance-independent meanings), but the second layer would compare those meanings with the utterance in question, shave most (or all) of them out, and either add a new meaning or reinforce an old one. But again, I'm no programmer. I might be saying something that is already done, or unfeasible for some reason.
- shoo 5y ago
  (this is also not my field) I was thinking about this a little more: In some cases (such as your excellent wug example) it might be possible to heuristically associate the word "wug" to a probable contextual meaning "your cat" from a shortlist of candidate objects ("you (conversing)", "me (also conversing)", "cat", "bed") without understanding that cats are things that sleep on beds, and some kind of educated guess that "that wug" doesn't refer to the speaker themselves! In other cases, figuring this out might require a fair bit of side knowledge about the meanings of what is being discussed, and how they relate to each other, to almost logically deduce what entity is being labelled, given knowledge of how those entities work. That might be dramatically harder to get get a machine to do well. Perhaps a machine being able to to this would be a leap forward in machine intelligence from today's ML "stochastic parrots".
tjs8rj 5y ago
Who cares if the machine understands wug? Maybe the issue is more to do with trying to teach language learning as a end on its own. Nobody studies language without purpose, there’s always a next step. If you’re friend is telling you about their cat and what a wug it is, the goal is to conversate, make the other feel heard, entertain them, kill time until something next happens, impress them, and so on. Most of these don’t require even picking up on “wug”, they require understanding the other’s state of mind: “my new cat is so cute - I love them!”. Getting caught on solving wug seems like an uneccessary triviality, more important is the tone they say it with, they body language, your past interactions with them. You could form a proper reaction if you only heard every other word and had those other details. Humans don’t rely on deciphering “wug” - they don’t need to. The whole interaction is just negotiating mental states, intentions, and actions. Deciphering “wug” is unnecessary.

kumarvvr 5y ago

An awesome explanation. I guess then, for machine learning to succeed in natural language interpretation, we ought to have another layer of abstraction that learns meanings from contexts and the general situation. Extending on your idea, I think, not only the verbal context, but physical location / time of day / historical context / current events / social group etc all add their own flavor to the variety of meanings on single word can have. For example, if I am walking in a park and there is a strong gust of wind, I might say "whoa, that's a strong one". The listener, if physically present along with the speaker, will immediately get it. Adding language context is one challenge. Adding the context of environment is a completely different challange.

michannne 5y ago

> machines treat words as if they got intrinsic meaning, when the meaning is actually negotiated between speaker and hearer at the moment of the utterance. IMO, and I'm not a linguist, I've always felt we need to treat conversations as transactions, with each speaker acting as a party and continually updating some contract over time, with each speaker's goal to use conversational tactics i.e. clarification, questioning, answering a question, diverting the conversation, humor/jokes, lies, as a means to "guess" the contract the opposing parties are operating under, then using the best guess of the contract to later retrieve information relevant to the system when it has some confidence in the answers it can expect. Something like an exceptionally complex game mixed with financial system theory, like I "bet" that you're going to keep talking about your cat, and maybe that bet is wrong, so I can ask a question, "What do you mean by wug?", but the act of asking that question will cost me some points, maybe you can think of it as though we're at a secret club and you use a phrase meant to represent a key and I'm supposed to respond with the proper answer. Even though I'm asking a question, I'm giving you information that you can use to update your best guess of the contract that allows you to assume I'm not part of the in-group of the club and I shouldn't be allowed in. I've thought about a system designed like this for a bit but I suppose the biggest challenge has been how do you treat a conversation as a game considering neither party may "win" in any reasonable time frame. We could go years chatting and I could never fully predict what your answer might be because your usage of conversational tactics may be refined over years by speaking to multiple people of backgrounds similar to mine. So if I'm a cop, you might know not to admit to a crime when you figure out I'm a cop. Right now, I don't think ML can do that, not because it's impossible but because I think conversations are as difficult as predicting the stock market. Fortunately it seems solvable, not everyone is expected to win every conversation in their lifetime

visarga 5y ago

GPT-3 can figure out a nonsensical word and use it in the same example. The prompt is between quotes, everything else is generated. > "I got a cat. That wug sleeps on my bed." On every night. He’s really nice. But he isn’t small. > I got a dog. That wug sleeps at my house. On every night. She’s really nice. But she isn’t small. ~~~ Another one in dialogue format. > Human: I got a cat. That wug sleeps on my bed. > AI: It has a great personality. > Human: Define a wug based on the previous lines. > AI: A wug is a small, impulsive, and clumsy creature.

gattilorenz 5y ago
Not having access to GPT-3, can you prompt "Is a wug a cat?"
goatlover 5y ago
What sort of additional prompting would you need to do to get it to understand that cats aren't clumsy, but do enjoy knocking things over?
lvxferre 5y ago
Can GPT-3 figure out that a common word is being used outside its usual context, so its usual meanings don't apply? Let's say my example was instead "I got a cat. That little orc sleeps on my bed." I wonder how GPT-3 would deal with it. Maybe it would think "cat" and "orc" refer to different objects/beings? Or associate "cat" with features that clearly wouldn't fit a cat like "green", "humanoid", or "fantasy monster"?
- goatlover 5y ago
  Clumsy already doesn't fit with cat, assuming GPT-3 is using wug for cat. Cats are more athletic than we are. They are very finely tuned hunters. But they need a lot of stimulation and so house cats will play a lot, and cause silly disturbances for humans.

abecedarius 5y ago

fwiw, I just pasted your example into GPT-3, ran it twice, and both times the continuation acted like 'wug' meant a cat. (Once as the cat's name, once generically.)

PeterisP 5y ago

> "I dropped the trophy on the table. Now it's broken." (What's broken - the trophy, or the table?) We could not solve this kind of problems (Winograd schemas) automatically a few years ago, but now ML techniques can solve them at about the same error rate as humans.