6 ms·

I admit I’m completely out of my depth when it comes to this field — I don’t even typically care about AI — but Eliezer’s response looks really bad to anyone in

by dvzk 3y ago

I admit I’m completely out of my depth when it comes to this field — I don’t even typically care about AI — but Eliezer’s response looks really bad to anyone in research. Asking for a citable and thorough written argument is as basic a requirement as it gets. To repudiate that request with “everyone has a different objection” is nearly unthinkable. And to David Chalmers no less! I think podcast hosts, tweet authors, bloggers and live streamers sometimes forget that progress in academic fields comes from real contributions, and that public conversations (especially oral) don’t really do anything besides spread common awareness.

Cardinal7167 3y ago
This is what social media’s real damage is. It turned academic progress into arguable points. Your research degree and countless lab hours is equally valid to my two-second no-research hot take, and if you’re not on board with it then you’re swarmed.
- wnevets 3y ago
  > Your research degree and countless lab hours is equally valid to my two-second no-research hot take, But you don't understand, I did my own research
- Swizec 3y ago
  > my two-second no-research hot take This is more like watching an argument between Feynman and Oppenheimer. They look like hot takes on twitter, but these guys have spent plenty of time thinking about it. Yudkowsky is an AI ethics and safety researcher, also founder of LessWrong, an HN favorite – https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.wikipedia.org/wiki/Eliezer_Yudkowsky And Chalmers is a philosopher focusing on consciousness and cognitive science – https://en.wikipedia.org/wiki/David_Chalmers https://en.wikipedia.org/wiki/David_Chalmers I wouldn’t put their replies in exactly the same category as your typical social media reply guys.
  - Cardinal7167 3y ago
    I didn’t mean my comment about this tweet exchange as an example, I was only trying to speak to the comment I replied to about a general trend. This particular exchange is not the problem; it’s the inability for the public to gauge when it’s not like this, though, if that makes sense.
  - ThrowawayR2 3y ago
    Yudkowsky and LessWrong was regarded as kooky for ages and, after the Roko's Basilisk affair (https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk and read Yudkowsky's hilariously deranged all-caps response), a laughingstock as well. Suggesting that he's on the same plane as serious researchers is inappropriate.
- p-e-w 3y ago
  > This is what social media’s real damage is. It turned academic progress into arguable points. If that is "damage", the problem must surely lie with academia, not with social media. Academia's only raison d'être is to facilitate formal discourse. Wherever it doesn't do that, it loses its legitimacy. "I'm a researcher and you are not" is not an argument. In fact, I'd go so far as to claim that anyone who makes such an "argument" should be excluded from any future discourse.
  - danaris 3y ago
    I think you may be misunderstanding the GP (though it's possible I am); my reading of that was not that it actually changed what "academic progress" is, but rather that it changed the public perception of it, as it gets discussed publicly on social media, where (particularly on Twitter) it's much more important who can draw the most views/shares/etc than who actually has the better science.
rnk 3y ago
What's the academically supported description/ argument or whatever for why nuclear weapons are dangerous for the world if they exist? Obviously they are dangerous for the world because we might kill everyone if someone makes a mistake or gets mad and shoots one first. The danger of nukes is more obvious than why AI is dangerous.
- ThrowawayR2 3y ago
  > "What's the academically supported description/ argument or whatever for why nuclear weapons are dangerous for the world if they exist?" The Wikipedia article on mutual assured destruction (https://en.wikipedia.org/wiki/Mutual_assured_destruction https://en.wikipedia.org/wiki/Mutual_assured_destruction) traces the origin and evolution of exactly that analysis, including even pre-atom bomb precursors based on earlier weapons that would hypothetically make war "too terrible to ever happen again".
- ifdefdebug 3y ago
  Well THAT support starts with the experimental records of Manhattan Project and fills a whole library. Support for current ML techniques being dangerous in that sense are handwavy descriptions of what might well be science fiction for another 500 years.
  - AstralStorm 3y ago
    Well, support for ML techniques being stupidly wrong is much bigger. It's the default state. Then there's some less support for them being misleadingly wrong, much more rarely. Science fiction Terminators it is not. Paperclip maximizers, however, yes. In fact it can be argued that a typical capitalist company is a variant of a paperclip maximizer and is a simple form of superhuman intelligence. Nick Bostrom invented that thought experiment. Start from there and go digging through AI safety research of which there are a few journals. Real life samples include 2010 flash crash, social media automatically optimizing for engagement at any cost, a few computer vision issues causing medical misdiagnosis, Tesla's "self-driving", video game AI exploiting engine bugs rather than playing, suddenly racist face recognition... And those are just mistakes, a malicious actor can do more with these. DARPA and IARPA have a whole damn wing on handling trojan attacks against ML systems. A superhuman, even weakly superhuman paperclip maximizer is an extremely dangerous thing. (Analyze climate change in this light...)
mitthrowaway2 3y ago
This seems like it's just an issue with the way conversations look on Twitter. If this were two people walking down the street and talking, I don't think that would be taken as a repudiation, just a request to clarify a vague question. Presumably, Yudkowksy could point to any of several of his own publications, like this one: https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf
- bordercases 3y ago
  He doesn't want to open himself to criticism from people that might actually do a good job of it.
  - mitthrowaway2 3y ago
    Did he tell you this? That seems like a mean-spirited accusation.
    - bordercases 3y ago
      You would have to deal with the community's pseudointellectualism long enough to identify it as a pattern. Taken in isolation it looks like nothing, but the arc of the rationalist community has been insularity combined with CYA until a crisis comes along which makes them both more guarded with others while appearing to "update" in public discussion, using such a limited notion of belief because they view their core texts and blog posts as a canonical view of philosophy, and covering up degenerate behavior like sexual abuse on a regular basis. The real rationalists just move on from this crazy making. I've had to decondition myself like an ex-cultist which is why I'm reacting this way and I hope others can see how he behaves problematically too.
adfgadfhionio 3y ago
This field has created a lot of writings but, as far as I can tell, no knowledge. The subject matter of this field does not exist. I don't think you need to know much of anything to match the supposed experts. Watching Terminator is probably good enough.
bordercases 3y ago
As someone who has outgrown that particular community, Yudkowsky has a talent for acting in bad faith argumentatively while not appearing to do so. He relies heavily on convservational implicature and passing along the burden of proof (as he did here) with no intention of being helpful to those that can generally improve his case through criticism or a distinctive aproach, as it would detract from sustaining his perceived position a focal or pioneering contributor. He should be applauded as a popularizer and point of inspiration, but his duplitousness in his argumentative style, once I identfied it, always left a bad taste in my mouth.

by dvzk 3y ago

Cardinal7167 3y ago

This is what social media’s real damage is. It turned academic progress into arguable points. Your research degree and countless lab hours is equally valid to my two-second no-research hot take, and if you’re not on board with it then you’re swarmed.

wnevets 3y ago
> Your research degree and countless lab hours is equally valid to my two-second no-research hot take, But you don't understand, I did my own research
Swizec 3y ago
> my two-second no-research hot take This is more like watching an argument between Feynman and Oppenheimer. They look like hot takes on twitter, but these guys have spent plenty of time thinking about it. Yudkowsky is an AI ethics and safety researcher, also founder of LessWrong, an HN favorite – https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.wikipedia.org/wiki/Eliezer_Yudkowsky And Chalmers is a philosopher focusing on consciousness and cognitive science – https://en.wikipedia.org/wiki/David_Chalmers https://en.wikipedia.org/wiki/David_Chalmers I wouldn’t put their replies in exactly the same category as your typical social media reply guys.
- Cardinal7167 3y ago
  I didn’t mean my comment about this tweet exchange as an example, I was only trying to speak to the comment I replied to about a general trend. This particular exchange is not the problem; it’s the inability for the public to gauge when it’s not like this, though, if that makes sense.
- ThrowawayR2 3y ago
  Yudkowsky and LessWrong was regarded as kooky for ages and, after the Roko's Basilisk affair (https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk and read Yudkowsky's hilariously deranged all-caps response), a laughingstock as well. Suggesting that he's on the same plane as serious researchers is inappropriate.
p-e-w 3y ago
> This is what social media’s real damage is. It turned academic progress into arguable points. If that is "damage", the problem must surely lie with academia, not with social media. Academia's only raison d'être is to facilitate formal discourse. Wherever it doesn't do that, it loses its legitimacy. "I'm a researcher and you are not" is not an argument. In fact, I'd go so far as to claim that anyone who makes such an "argument" should be excluded from any future discourse.
- danaris 3y ago
  I think you may be misunderstanding the GP (though it's possible I am); my reading of that was not that it actually changed what "academic progress" is, but rather that it changed the public perception of it, as it gets discussed publicly on social media, where (particularly on Twitter) it's much more important who can draw the most views/shares/etc than who actually has the better science.

rnk 3y ago

What's the academically supported description/ argument or whatever for why nuclear weapons are dangerous for the world if they exist? Obviously they are dangerous for the world because we might kill everyone if someone makes a mistake or gets mad and shoots one first. The danger of nukes is more obvious than why AI is dangerous.

ThrowawayR2 3y ago
> "What's the academically supported description/ argument or whatever for why nuclear weapons are dangerous for the world if they exist?" The Wikipedia article on mutual assured destruction (https://en.wikipedia.org/wiki/Mutual_assured_destruction https://en.wikipedia.org/wiki/Mutual_assured_destruction) traces the origin and evolution of exactly that analysis, including even pre-atom bomb precursors based on earlier weapons that would hypothetically make war "too terrible to ever happen again".
ifdefdebug 3y ago
Well THAT support starts with the experimental records of Manhattan Project and fills a whole library. Support for current ML techniques being dangerous in that sense are handwavy descriptions of what might well be science fiction for another 500 years.
- AstralStorm 3y ago
  Well, support for ML techniques being stupidly wrong is much bigger. It's the default state. Then there's some less support for them being misleadingly wrong, much more rarely. Science fiction Terminators it is not. Paperclip maximizers, however, yes. In fact it can be argued that a typical capitalist company is a variant of a paperclip maximizer and is a simple form of superhuman intelligence. Nick Bostrom invented that thought experiment. Start from there and go digging through AI safety research of which there are a few journals. Real life samples include 2010 flash crash, social media automatically optimizing for engagement at any cost, a few computer vision issues causing medical misdiagnosis, Tesla's "self-driving", video game AI exploiting engine bugs rather than playing, suddenly racist face recognition... And those are just mistakes, a malicious actor can do more with these. DARPA and IARPA have a whole damn wing on handling trojan attacks against ML systems. A superhuman, even weakly superhuman paperclip maximizer is an extremely dangerous thing. (Analyze climate change in this light...)

mitthrowaway2 3y ago

This seems like it's just an issue with the way conversations look on Twitter. If this were two people walking down the street and talking, I don't think that would be taken as a repudiation, just a request to clarify a vague question. Presumably, Yudkowksy could point to any of several of his own publications, like this one: https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf

bordercases 3y ago
He doesn't want to open himself to criticism from people that might actually do a good job of it.
- mitthrowaway2 3y ago
  Did he tell you this? That seems like a mean-spirited accusation.
  - bordercases 3y ago
    You would have to deal with the community's pseudointellectualism long enough to identify it as a pattern. Taken in isolation it looks like nothing, but the arc of the rationalist community has been insularity combined with CYA until a crisis comes along which makes them both more guarded with others while appearing to "update" in public discussion, using such a limited notion of belief because they view their core texts and blog posts as a canonical view of philosophy, and covering up degenerate behavior like sexual abuse on a regular basis. The real rationalists just move on from this crazy making. I've had to decondition myself like an ex-cultist which is why I'm reacting this way and I hope others can see how he behaves problematically too.

adfgadfhionio 3y ago

This field has created a lot of writings but, as far as I can tell, no knowledge. The subject matter of this field does not exist. I don't think you need to know much of anything to match the supposed experts. Watching Terminator is probably good enough.

bordercases 3y ago

As someone who has outgrown that particular community, Yudkowsky has a talent for acting in bad faith argumentatively while not appearing to do so. He relies heavily on convservational implicature and passing along the burden of proof (as he did here) with no intention of being helpful to those that can generally improve his case through criticism or a distinctive aproach, as it would detract from sustaining his perceived position a focal or pioneering contributor. He should be applauded as a popularizer and point of inspiration, but his duplitousness in his argumentative style, once I identfied it, always left a bad taste in my mouth.