6 ms·

The comments in interpretability read like science fiction to me. There's paragraphs on DV3 explaining other models and itself and the emergent properties that

The comments in interpretability read like science fiction to me. There's paragraphs on DV3 explaining other models and itself and the emergent properties that appear with bigger models. There's so much commented out related to functional explainability and counterfactual generations. "we asked DV3 for an explanation. DV3 replied that it detected sarcasm in the review, which it interpreted as a sign of negative sentiment. This was a surprising and reasonable explanation, since sarcasm is a subtle and subjective form of expression that can often elude human comprehension as well. However, it also revealed that DV3 had a more sensitive threshold for sarcasm detection than the human annotator, or than we expected -- thereby leading to the misspecification. To verify this explanation, we needed to rewrite the review to eliminate any sarcasm and see if DV3 would revise its prediction. We asked DV3 to rewrite the review to remove sarcasm based on its explanation. When we presented this new review to DV3 in a new prompt, it correctly classified it as positive sentiment, confirming that sarcasm was the cause of the specification error." The published paper instead says "we did not test for the ability to understand sarcasm, irony, humor, or deception, which are also related to theory of mind" . The main conclusion I took away from this is "the remarkable emergence of what seems to be increasing functional explainability with increasing model scale". I can see the reasoning for why OpenAI decided not to publish any more details about the size or steps to reproduce their model. I assumed we would need a much bigger model to see these level of "human" understanding from LLMs. I can respect Meta, Google, and OpenAI's decision, but I hope this accelerates the research into truly open source models. Interacting with these models shouldn't be locked behind corporate doors.

PaulDavisThe1st 4y ago
> "we did not test for the ability to understand sarcasm" I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm. The same is true of irony. I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog. Dennett further speculates that our own self-experience of intention is a matter of turning the same explanatory model upon our own behavior, but that's an extension that isn't directly relevant to this speculation about language models.
- knaik94 4y ago
  The authors go into more detail about their reasoning and the nuances between mechanistic and functional explainability. The authors only said related not required. No clear reason was given for why the commented sections stayed commented out.
- nextaccountic 4y ago
  > I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog. That's also a way to frame evolution and literally all of biology.
- paganel 4y ago
  > It requires some association between various stylistic elements and the concept of sarcasm. That sounds very mechanic-ist, including Dennett's theory (I haven't read Dennett, to be exact, I'm going by your explanation). Which I guess it's par for the course when talking about a scaled up Mechanical Turk concoction like this GPT thing is. It won't "create" any Radio Erevan jokes anytime soon, that's for sure, though.
- PeterisP 4y ago
  > I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm. I am quite certain that detecting sarcasm can't be done based on stylistic elements" and requires some (even if implicit) estimation of what the author is thinking that contrasts with what is being said. E.g. a relevant example I have actually seen in doing sentiment analysis on tweets to evaluate how customers perceive a company is "#CompanyName Got my order delivered in just under three hours. Thank you for great service! thumbsup-emoji" - now is this a positive review or sarcasm? And the thing is, you can't tell from the message by itself, you need an understanding of the customers' expectation (guess you might call it "a theory of mind") that for a pizza chain this is obviously sarcasm, but for a web store that sells some electronics the same thing would actually mean a fast delivery and great service. IMHO sarcasm detection is mostly about 'world knowledge' about what the implied expectations are, and not about stylistic elements at all.
meghan_rain 4y ago
Wait so they tested if it detects sarcasm, it failed, and then they were like "let's pretend this never happened" and wrote "We did not test for the ability to understand sarcasm"?
- famouswaffles 4y ago
  It didn't fail ?
- salad-tycoon 4y ago
  There was a conflict of opinions with D3 and human annotator. The quote above at least does not indicate who was right or wrong, merely noting that the machine had a more sensitive sarcasm detector. So I guess maybe it’s saying the human was wrong after all?
  - ChatGTP 4y ago
    If you read the IMDB review, there is no sarcasm in it. Maybe more like appreciation with some humour thrown in? Are we all a bit confused and second guessing our own use and interpretation of language now ?
    - knaik94 4y ago
      I got confused about whether the review was positive or negative too. The part of the review where it says "he must have never in his life seen a flick about any small towns" could be classified as sarcastic. The review isn't very clear in terms of prose and conclusion. The reviewer clearly says "you should watch the movie", but after re-reading it a few times, I would consider that as sarcasm too. I can't say for sure whether the reviewer enjoyed it themselves.
      - ChatGTP 4y ago
        Sorry I wasn't clear, I mean appreciation that he does not living in a horrible small town with monsters etc. Not exactly for the movie. "he must have never in his life seen a flick about any small towns" That's probably the least sarcastic sentence for me, because it's just a reinforcement of the opening statement: "I am so happy not to live in an American small town. Because whenever I'm shown some small town in the States it is populated with all kinds of monsters among whom flesh hungry zombies, evil aliens and sinister ghosts are most harmless." I don't think it's a "positive review" I think it's neither, it's fairly neutral, and the author kind of suggests the reader watches the movie. Out of interest I asked someone who has no idea about ChatGPT-4 and it's apparent sarcasm detection abilities and they didn't think it was sarcastic, albeit a bit 'weird' and poorly written. Confirmation bias? We could say more, however, there are more important things to do...
djmips 4y ago
In fact this is starting to feel more like an ARG than real life. https://en.wikipedia.org/wiki/Alternate_reality_game https://en.wikipedia.org/wiki/Alternate_reality_game

PaulDavisThe1st 4y ago

> "we did not test for the ability to understand sarcasm" I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm. The same is true of irony. I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog. Dennett further speculates that our own self-experience of intention is a matter of turning the same explanatory model upon our own behavior, but that's an extension that isn't directly relevant to this speculation about language models.

knaik94 4y ago
The authors go into more detail about their reasoning and the nuances between mechanistic and functional explainability. The authors only said related not required. No clear reason was given for why the commented sections stayed commented out.
nextaccountic 4y ago
> I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog. That's also a way to frame evolution and literally all of biology.
paganel 4y ago
> It requires some association between various stylistic elements and the concept of sarcasm. That sounds very mechanic-ist, including Dennett's theory (I haven't read Dennett, to be exact, I'm going by your explanation). Which I guess it's par for the course when talking about a scaled up Mechanical Turk concoction like this GPT thing is. It won't "create" any Radio Erevan jokes anytime soon, that's for sure, though.
PeterisP 4y ago
> I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm. I am quite certain that detecting sarcasm can't be done based on stylistic elements" and requires some (even if implicit) estimation of what the author is thinking that contrasts with what is being said. E.g. a relevant example I have actually seen in doing sentiment analysis on tweets to evaluate how customers perceive a company is "#CompanyName Got my order delivered in just under three hours. Thank you for great service! thumbsup-emoji" - now is this a positive review or sarcasm? And the thing is, you can't tell from the message by itself, you need an understanding of the customers' expectation (guess you might call it "a theory of mind") that for a pizza chain this is obviously sarcasm, but for a web store that sells some electronics the same thing would actually mean a fast delivery and great service. IMHO sarcasm detection is mostly about 'world knowledge' about what the implied expectations are, and not about stylistic elements at all.

meghan_rain 4y ago

Wait so they tested if it detects sarcasm, it failed, and then they were like "let's pretend this never happened" and wrote "We did not test for the ability to understand sarcasm"?

famouswaffles 4y ago
It didn't fail ?
salad-tycoon 4y ago
There was a conflict of opinions with D3 and human annotator. The quote above at least does not indicate who was right or wrong, merely noting that the machine had a more sensitive sarcasm detector. So I guess maybe it’s saying the human was wrong after all?
- ChatGTP 4y ago
  If you read the IMDB review, there is no sarcasm in it. Maybe more like appreciation with some humour thrown in? Are we all a bit confused and second guessing our own use and interpretation of language now ?
  - knaik94 4y ago
    I got confused about whether the review was positive or negative too. The part of the review where it says "he must have never in his life seen a flick about any small towns" could be classified as sarcastic. The review isn't very clear in terms of prose and conclusion. The reviewer clearly says "you should watch the movie", but after re-reading it a few times, I would consider that as sarcasm too. I can't say for sure whether the reviewer enjoyed it themselves.
    - ChatGTP 4y ago
      Sorry I wasn't clear, I mean appreciation that he does not living in a horrible small town with monsters etc. Not exactly for the movie. "he must have never in his life seen a flick about any small towns" That's probably the least sarcastic sentence for me, because it's just a reinforcement of the opening statement: "I am so happy not to live in an American small town. Because whenever I'm shown some small town in the States it is populated with all kinds of monsters among whom flesh hungry zombies, evil aliens and sinister ghosts are most harmless." I don't think it's a "positive review" I think it's neither, it's fairly neutral, and the author kind of suggests the reader watches the movie. Out of interest I asked someone who has no idea about ChatGPT-4 and it's apparent sarcasm detection abilities and they didn't think it was sarcastic, albeit a bit 'weird' and poorly written. Confirmation bias? We could say more, however, there are more important things to do...

djmips 4y ago

In fact this is starting to feel more like an ARG than real life. https://en.wikipedia.org/wiki/Alternate_reality_game https://en.wikipedia.org/wiki/Alternate_reality_game