5 ms·

This is the issue, bots/AI can’t comprehend sarcasm, jokes, or otherwise human behaviors. Facebook doesn’t have human reviewers.

Solvency 3y ago
Not true. At all. ChatGPT could (and does already contain) training data on internet memes and you can prompt it to consider memes, sarcasm, inside jokes, etc. Literally ask it now with examples and it'll work. "It seems like those comments might be exaggerated or joking responses to the presence of a spider. Arson is not a reasonable solution for dealing with a spider in your house. Most likely, people are making light of the situation."
- mega_dean 3y ago
  > you can prompt it to consider memes, sarcasm, inside jokes, etc. I use Custom Instructions that specifically ask for "accurate and helpful answers": "Please call me "Dave" and talk in the style of Hal from 2000: A Space Odyssey. When I say "Hal", I am referring to ChatGPT. I would still like accurate and helpful answers, so don't be evil like Hal from the movie, just talk in the same style." I just started a conversation to test if it needed to be explicitly told to consider humor, or if it would realize that I was joking: You: Open the pod bay doors please, Hal. ChatGPT: I'm sorry, Dave. I'm afraid I can't do that.
  - reactordev 3y ago
    You may find that humorous but it's not humor. It's playing the role you said it should. According to the script, "I'm sorry, Dave. I'm afraid I can't do that." is said by HAL more than any other line HAL says.
- barbazoo 3y ago
  How about the "next" meme, one it hasn't been trained on?
  - esafak 3y ago
    It won't do worse than the humans that Facebook hires to review cases. Humans miss jokes too.
    - reactordev 3y ago
      this is a very poignant argument as well. As we strive for 100% accuracy, are we even that accurate? Can we just strive for more accurate than "Bob"?
- vidarh 3y ago
  I was disappointed that ChatGPT didn't catch the, presumably unintended, funny bit it introduced in its explanation, though: "people are making light of the situation" in an explanation about arson. I asked it more and more leading questions and I had to explicitly point to the word "light" to make it catch it.
  - reactordev 3y ago
    I think it’s interesting that you had to re-prompt and focus for it to weight the right weights. I do think that given more training GPT will nail this subtlety of human expression.
- reactordev 3y ago
  Very True. Completely. ChatGPT can detect and classify jokes it has already heard or "seen" but still fails to detect jokes it hasn't. Also, I was talking about Facebook Moderation AI and bots and not GPT. Last time I checked, Facebook isn't using ChatGPT to moderate content.
fragmede 3y ago
ChatGPT-4 isn't your father's bot. It is able to deduce that the comment made is an attempt at humor, and even helpfully explains the joke. This kills the joke, unfortunately, but it shows a modern AI wouldn't have moderated the comment away. https://chat.openai.com/share/7d883836-ca9c-4c04-83fd-356d4ab70c98 https://chat.openai.com/share/7d883836-ca9c-4c04-83fd-356d4a...
- tmoravec 3y ago
  Having ChatGPT-4 moderate Facebook would probably be even more expensive than having humans review everything.
  - esafak 3y ago
    They're Facebook; they have their own LLMs. This is definitely a great first line of review. Then they can manually scrutinize the edge cases.
  - ForkMeOnTinder 3y ago
    How? I thought we all agreed AI was cheaper than humans (accuracy notwithstanding), otherwise why would everyone be afraid AI is going to take their jobs?
  - dragonwriter 3y ago
    Using Llama Guard as a first pass screen and then passing on material needing more comprehensive review to a more capable model (or human reviewer, or a mix) seems more likely ti be useful and efficient than using a heavyweight model as the primary moderation tool.
  - fragmede 3y ago
    More expensive in what? The GPUs to run them on are certainly exorbitantly expensive in dollars, but ChatGPT-4 viewing CSAM and violent depraved videos doesn't get tired or need to go to therapy. It's not a human that's going to lose their shit because they watched a person hit a kitten with a hammer for fun in order to moderate it away, so in terms of human cost, it seems quite cheap!
School-Cotton 3y ago
Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ?
- barbazoo 3y ago
  > Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ? How do you know they have "not tried modern AI like GPT4"?
  - esafak 3y ago
    Because they would know GPT4 is capable of getting the joke.
    - reactordev 3y ago
      I was talking about FB moderation AI, not GPT4. There are a couple AI LLM's that can recall jokes and match sentiment, context, "joke" and come to the conclusion it's a joke. Facebook's moderation AI isn't that sophisticated (yet).
- soulofmischief 3y ago
  It's an epidemic, and when you suggest they try GPT-4, most flat-out refuse, having already made up their minds. It's like people have completely forgotten the concept of technological progression, which by the way is happening at a blistering pace.
  - reactordev 3y ago
    I disagree with this view. I think most people who are interested, have tried it. They tried it with a variety of prompts. They found it novel but not entirely accurate. So while I do think there are some people who refuse to use AI at all, I’d love to point out that they already are. GPT on the other hand, is next level. There’s a level after that even. The point I was articulating is that the current gen bot/AI moderation models are not GPT level. At least today from my own experience in dealing with moderation and content flagging trolls. I do believe FB/Meta is fervently working on this with their models they are publishing in competition to GPT. So before you go burning down the town - Accept that technological advancement is great, when it solves your problem. Otherwise it’s bells and whistles to these people.

Solvency 3y ago

Not true. At all. ChatGPT could (and does already contain) training data on internet memes and you can prompt it to consider memes, sarcasm, inside jokes, etc. Literally ask it now with examples and it'll work. "It seems like those comments might be exaggerated or joking responses to the presence of a spider. Arson is not a reasonable solution for dealing with a spider in your house. Most likely, people are making light of the situation."

mega_dean 3y ago
> you can prompt it to consider memes, sarcasm, inside jokes, etc. I use Custom Instructions that specifically ask for "accurate and helpful answers": "Please call me "Dave" and talk in the style of Hal from 2000: A Space Odyssey. When I say "Hal", I am referring to ChatGPT. I would still like accurate and helpful answers, so don't be evil like Hal from the movie, just talk in the same style." I just started a conversation to test if it needed to be explicitly told to consider humor, or if it would realize that I was joking: You: Open the pod bay doors please, Hal. ChatGPT: I'm sorry, Dave. I'm afraid I can't do that.
- reactordev 3y ago
  You may find that humorous but it's not humor. It's playing the role you said it should. According to the script, "I'm sorry, Dave. I'm afraid I can't do that." is said by HAL more than any other line HAL says.
barbazoo 3y ago
How about the "next" meme, one it hasn't been trained on?
- esafak 3y ago
  It won't do worse than the humans that Facebook hires to review cases. Humans miss jokes too.
  - reactordev 3y ago
    this is a very poignant argument as well. As we strive for 100% accuracy, are we even that accurate? Can we just strive for more accurate than "Bob"?
vidarh 3y ago
I was disappointed that ChatGPT didn't catch the, presumably unintended, funny bit it introduced in its explanation, though: "people are making light of the situation" in an explanation about arson. I asked it more and more leading questions and I had to explicitly point to the word "light" to make it catch it.
- reactordev 3y ago
  I think it’s interesting that you had to re-prompt and focus for it to weight the right weights. I do think that given more training GPT will nail this subtlety of human expression.
reactordev 3y ago
Very True. Completely. ChatGPT can detect and classify jokes it has already heard or "seen" but still fails to detect jokes it hasn't. Also, I was talking about Facebook Moderation AI and bots and not GPT. Last time I checked, Facebook isn't using ChatGPT to moderate content.

fragmede 3y ago

ChatGPT-4 isn't your father's bot. It is able to deduce that the comment made is an attempt at humor, and even helpfully explains the joke. This kills the joke, unfortunately, but it shows a modern AI wouldn't have moderated the comment away. https://chat.openai.com/share/7d883836-ca9c-4c04-83fd-356d4ab70c98 https://chat.openai.com/share/7d883836-ca9c-4c04-83fd-356d4a...

tmoravec 3y ago
Having ChatGPT-4 moderate Facebook would probably be even more expensive than having humans review everything.
- esafak 3y ago
  They're Facebook; they have their own LLMs. This is definitely a great first line of review. Then they can manually scrutinize the edge cases.
- ForkMeOnTinder 3y ago
  How? I thought we all agreed AI was cheaper than humans (accuracy notwithstanding), otherwise why would everyone be afraid AI is going to take their jobs?
- dragonwriter 3y ago
  Using Llama Guard as a first pass screen and then passing on material needing more comprehensive review to a more capable model (or human reviewer, or a mix) seems more likely ti be useful and efficient than using a heavyweight model as the primary moderation tool.
- fragmede 3y ago
  More expensive in what? The GPUs to run them on are certainly exorbitantly expensive in dollars, but ChatGPT-4 viewing CSAM and violent depraved videos doesn't get tired or need to go to therapy. It's not a human that's going to lose their shit because they watched a person hit a kitten with a hammer for fun in order to moderate it away, so in terms of human cost, it seems quite cheap!

School-Cotton 3y ago

Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ?

barbazoo 3y ago
> Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ? How do you know they have "not tried modern AI like GPT4"?
- esafak 3y ago
  Because they would know GPT4 is capable of getting the joke.
  - reactordev 3y ago
    I was talking about FB moderation AI, not GPT4. There are a couple AI LLM's that can recall jokes and match sentiment, context, "joke" and come to the conclusion it's a joke. Facebook's moderation AI isn't that sophisticated (yet).
soulofmischief 3y ago
It's an epidemic, and when you suggest they try GPT-4, most flat-out refuse, having already made up their minds. It's like people have completely forgotten the concept of technological progression, which by the way is happening at a blistering pace.
- reactordev 3y ago
  I disagree with this view. I think most people who are interested, have tried it. They tried it with a variety of prompts. They found it novel but not entirely accurate. So while I do think there are some people who refuse to use AI at all, I’d love to point out that they already are. GPT on the other hand, is next level. There’s a level after that even. The point I was articulating is that the current gen bot/AI moderation models are not GPT level. At least today from my own experience in dealing with moderation and content flagging trolls. I do believe FB/Meta is fervently working on this with their models they are publishing in competition to GPT. So before you go burning down the town - Accept that technological advancement is great, when it solves your problem. Otherwise it’s bells and whistles to these people.