4 ms·

If you were to take a look at the list of signatories on safe.ai, that's basically everyone who is everyone that works on building AI, what could Emily B Bender

If you were to take a look at the list of signatories on safe.ai, that's basically everyone who is everyone that works on building AI, what could Emily B Bender a professor of computer linguistics possibly add to the conversation and how would she be able to talk more about the "real AI safety" than any of those people? Edit: Sorry if it sounds arrogant, I don't mean Emily wouldn't have anything to add, but not sure how the parent can just write off basically that whole list and claim someone who isn't a leader in the field would be the "real voice"?

sebzim4500 3y ago
I think we need to be realistic and accept that people are going to pick the expert that agrees with them, even if on paper they are far less qualified.
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]
Planktonne 3y ago
She's a professor of computer linguistics; it's literally her field that's being discussed. The list of signatories includes people with far less relevant qualifications, and significantly greater profit motive. She's an informed party who doesn't stand to profit; we should listen to her a lot more readily than others.
- brookst 3y ago
  How are fame, speaking engagements, and book deals not a form of profit? She's intelligent and worth listening to, but she has just as much personal bias and motivation as anyone else.
  - Planktonne 3y ago
    The (very small) amount of fame she's collected has come through her work in the field, and it's a field she's been in for a while; she's hardly chasing glory.
    - brookst 3y ago
      People don’t have to be chasing fame to be warped by it. She has cultivated a following of like-minded people who provide ever more positive feedback for her ever more ideological writing. I mean she is literally dismissing people who disagree with her based on their skin color. Can we stop for a minute to wonder about the incentives that encourage that? (and I generally like her writing and think she has interesting things to say… but I do see a reward cycle going on)
- qt31415926 3y ago
  Her field has also taken the largest hit from the success of LLMs and her research topics and her department are probably no longer prioritized by research grants. Given how many articles she's written that have criticized LLMs it's not surprising she has incentives.
  - Planktonne 3y ago
    LLMs are in her field; they are one of her research topics and they're definitely getting funding. We absolutely should not be ignoring research that doesn't support popular narratives; dismissing her work because it is critical of LLMs is not reasonable.
    - qt31415926 3y ago
      In her field doesn't mean that's what she researches, LLMs are loosely in her field but the methods are completely different. Computational linguistics != deep learning. Deep learning does not directly use concepts from linguistics, semantics, grammars or grammar engineering, which is what Emily was researcing for the past decades. It's the same thing as saying a number theorist and a set theorist are in the same field cause they both work in the Math field.
      - Planktonne 3y ago
        They are what she researches though. She has published research on them. LLMs don't directly use concepts from linguistics but they do produce and model language/grammar; it's entirely valid to use techniques from those fields to evaluate them, which is what she does. In the same vein, though a self-driving car doesn't work the same way as a human driver does, we can measure their performance on similar tasks.
        qt31415926 3y ago
        Hmm I looked into it, and looked at papers/pdfs in google scholar's advanced search with her as an author that mentioned LLMs or GPT in the past 3 years. Every single one was a criticism about how they couldn't actually understand anything (e.g. "they're only trained on form" and "at best they can only understand things in a limited well scoped fashion") and that linguistic fundamentals for NLP was more important. Good to know my hunch was correct
AlanYx 3y ago
She's the first author of the stochastic parrots paper, and she's fairly representative of the group of "AI safety" researchers who view the field from a statistical perspective linked to social justice issues. That's distinct from the group of "AI safety" researchers who focus on the "might destroy humanity" perspective. There are other groups too obviously -- the field seems to cluster into ideological perspectives.
- qt31415926 3y ago
  Current topic aside, I feel like that stochastic parrots paper aged really poorly in its criticisms of LLMs, and reading it felt like political propaganda with its exaggerated rhetoric and its anemic amount of scientific substance e.g. > Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. I'm surprised its cited so much given how many of its claims fell flat 1.5 years later
  - Der_Einzige 3y ago
    It's extremely easy to publish in NLP right now. 20-30% acceptance rates at even the top end conferences and plenty of tricks to increase your chances. Just because someone is first author on a highly cited paper doesn't imply that they're "right"
- adamsmith143 3y ago
  >She's the first author of the stochastic parrots paper, That alone is enough to disqualify any of her opinions on AI.
h___ 3y ago
She’s contributed to many academic papers on large language models and has a better technical understanding of how they work and their limitations than most signatories of this statement, or the previous widely hyped “AI pause” letter, which referenced one of her own papers. Read her statement about that letter (https://www.dair-institute.org/blog/letter-statement-March2023 https://www.dair-institute.org/blog/letter-statement-March20...) or listen to some of the many podcasts she’s appeared on talking about this. I find her and Timnit Gebru’s arguments highly persuasive. In a nutshell, the capabilities of “AI” are hugely overhyped and concern about Sci-Fi doom scenarios is disingenuously being used to frame the issue in ways that benefits players like OpenAI and diverts attention away from much more real, already occurring present-day harms such as the internet being filled with increasing amounts of synthetic text spam.
- calf 3y ago
  Thanks for the link, I read it with interest but philosophically it is a flawed argument (IMHO). It's a nonsequitur or something, for the following two reasons. First, I'm inclined to think that longtermism is an invalid and harmful ideology, but also acknowledge that AGI / existential risk is something that needs looking at seriously. The external factors, such as corporate interests and 1% wealthy interests/prejudices, are not a good reason to dismiss AGI concerns. I'd like to imagine there's a reasonable way to address short-term issues as well as long-term issues. It's not an either-or debate. Second, even from just a reading comprehension level: one side says AGI is a problem, then the other side cannot just say, "No, AGI is a false problem and here are the real problems". The reasonable argument is to say, AGI is a false problem because <of such and such reasons>. Bender et. al are just sidestepping the moot point, and rhetorically this is not an okay move. I think honest experts could simply say, ultimately we don't really know what will happen. But that would be boring to say because it would require acknowledging multiple issues being valid. (There's a well known chart, the hierarchy of disagreements. The most sophisticated intellectual disagreements point out what's wrong with the argument. Less sophisticated disagreements do things like point out alternatives, without pointing out what the critical mistake is. The critical mistake in this case hinges on whether the premise of AGI is true or not. That's the crux of disagreement. Substituting that with short-term issues, which are valid in themselves, are the example of a lower-level of argumentation. And of course even lower levels of argumentation are bad-faith readings and so forth, I forget but the chart had several levels. It's funny that professional academics nevertheless don't practice this and so get into endless, intellectually unsatisfactory debates.) So I think this is actually an example of different factional experts constantly talking past each other. It's funny that famous intellectuals/experts constantly do this, let their egos get the better of themselves and having a real intellectual conversation rather than make basic debate mistakes like nonsequiturs that any college student should be able to point at.

sebzim4500 3y ago

I think we need to be realistic and accept that people are going to pick the expert that agrees with them, even if on paper they are far less qualified.

deleted 3y ago

[deleted]

deleted 3y ago
[deleted]

Planktonne 3y ago

She's a professor of computer linguistics; it's literally her field that's being discussed. The list of signatories includes people with far less relevant qualifications, and significantly greater profit motive. She's an informed party who doesn't stand to profit; we should listen to her a lot more readily than others.

brookst 3y ago
How are fame, speaking engagements, and book deals not a form of profit? She's intelligent and worth listening to, but she has just as much personal bias and motivation as anyone else.
- Planktonne 3y ago
  The (very small) amount of fame she's collected has come through her work in the field, and it's a field she's been in for a while; she's hardly chasing glory.
  - brookst 3y ago
    People don’t have to be chasing fame to be warped by it. She has cultivated a following of like-minded people who provide ever more positive feedback for her ever more ideological writing. I mean she is literally dismissing people who disagree with her based on their skin color. Can we stop for a minute to wonder about the incentives that encourage that? (and I generally like her writing and think she has interesting things to say… but I do see a reward cycle going on)
qt31415926 3y ago
Her field has also taken the largest hit from the success of LLMs and her research topics and her department are probably no longer prioritized by research grants. Given how many articles she's written that have criticized LLMs it's not surprising she has incentives.
- Planktonne 3y ago
  LLMs are in her field; they are one of her research topics and they're definitely getting funding. We absolutely should not be ignoring research that doesn't support popular narratives; dismissing her work because it is critical of LLMs is not reasonable.
  - qt31415926 3y ago
    In her field doesn't mean that's what she researches, LLMs are loosely in her field but the methods are completely different. Computational linguistics != deep learning. Deep learning does not directly use concepts from linguistics, semantics, grammars or grammar engineering, which is what Emily was researcing for the past decades. It's the same thing as saying a number theorist and a set theorist are in the same field cause they both work in the Math field.
    - Planktonne 3y ago
      They are what she researches though. She has published research on them. LLMs don't directly use concepts from linguistics but they do produce and model language/grammar; it's entirely valid to use techniques from those fields to evaluate them, which is what she does. In the same vein, though a self-driving car doesn't work the same way as a human driver does, we can measure their performance on similar tasks.
      - qt31415926 3y ago
        Hmm I looked into it, and looked at papers/pdfs in google scholar's advanced search with her as an author that mentioned LLMs or GPT in the past 3 years. Every single one was a criticism about how they couldn't actually understand anything (e.g. "they're only trained on form" and "at best they can only understand things in a limited well scoped fashion") and that linguistic fundamentals for NLP was more important. Good to know my hunch was correct

AlanYx 3y ago

She's the first author of the stochastic parrots paper, and she's fairly representative of the group of "AI safety" researchers who view the field from a statistical perspective linked to social justice issues. That's distinct from the group of "AI safety" researchers who focus on the "might destroy humanity" perspective. There are other groups too obviously -- the field seems to cluster into ideological perspectives.

qt31415926 3y ago
Current topic aside, I feel like that stochastic parrots paper aged really poorly in its criticisms of LLMs, and reading it felt like political propaganda with its exaggerated rhetoric and its anemic amount of scientific substance e.g. > Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. It can’t have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. I'm surprised its cited so much given how many of its claims fell flat 1.5 years later
- Der_Einzige 3y ago
  It's extremely easy to publish in NLP right now. 20-30% acceptance rates at even the top end conferences and plenty of tricks to increase your chances. Just because someone is first author on a highly cited paper doesn't imply that they're "right"
adamsmith143 3y ago
>She's the first author of the stochastic parrots paper, That alone is enough to disqualify any of her opinions on AI.

h___ 3y ago

She’s contributed to many academic papers on large language models and has a better technical understanding of how they work and their limitations than most signatories of this statement, or the previous widely hyped “AI pause” letter, which referenced one of her own papers. Read her statement about that letter (https://www.dair-institute.org/blog/letter-statement-March2023 https://www.dair-institute.org/blog/letter-statement-March20...) or listen to some of the many podcasts she’s appeared on talking about this. I find her and Timnit Gebru’s arguments highly persuasive. In a nutshell, the capabilities of “AI” are hugely overhyped and concern about Sci-Fi doom scenarios is disingenuously being used to frame the issue in ways that benefits players like OpenAI and diverts attention away from much more real, already occurring present-day harms such as the internet being filled with increasing amounts of synthetic text spam.

calf 3y ago
Thanks for the link, I read it with interest but philosophically it is a flawed argument (IMHO). It's a nonsequitur or something, for the following two reasons. First, I'm inclined to think that longtermism is an invalid and harmful ideology, but also acknowledge that AGI / existential risk is something that needs looking at seriously. The external factors, such as corporate interests and 1% wealthy interests/prejudices, are not a good reason to dismiss AGI concerns. I'd like to imagine there's a reasonable way to address short-term issues as well as long-term issues. It's not an either-or debate. Second, even from just a reading comprehension level: one side says AGI is a problem, then the other side cannot just say, "No, AGI is a false problem and here are the real problems". The reasonable argument is to say, AGI is a false problem because <of such and such reasons>. Bender et. al are just sidestepping the moot point, and rhetorically this is not an okay move. I think honest experts could simply say, ultimately we don't really know what will happen. But that would be boring to say because it would require acknowledging multiple issues being valid. (There's a well known chart, the hierarchy of disagreements. The most sophisticated intellectual disagreements point out what's wrong with the argument. Less sophisticated disagreements do things like point out alternatives, without pointing out what the critical mistake is. The critical mistake in this case hinges on whether the premise of AGI is true or not. That's the crux of disagreement. Substituting that with short-term issues, which are valid in themselves, are the example of a lower-level of argumentation. And of course even lower levels of argumentation are bad-faith readings and so forth, I forget but the chart had several levels. It's funny that professional academics nevertheless don't practice this and so get into endless, intellectually unsatisfactory debates.) So I think this is actually an example of different factional experts constantly talking past each other. It's funny that famous intellectuals/experts constantly do this, let their egos get the better of themselves and having a real intellectual conversation rather than make basic debate mistakes like nonsequiturs that any college student should be able to point at.