9 ms·

Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? Seems surprising. Edit: I'm not pandering to authority,

Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? Seems surprising. Edit: I'm not pandering to authority, I just believe the people I've quoted are actually very smart people who can reason very well and they have a valid opinion on the topic. They also have little financial interest in the success, failure or regulation of AI, which is important.

sampo 3y ago
> Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? I am genuinely amazed how even educated and intelligent people (or maybe predominantly them) have fallen into this, what I see is essentially a doomsday cult. But groupthink happens, maybe nobody is safe. It's not like it hasn't happened before.
DalasNoin 3y ago
People are downvoting you. But 'pandering' to authority is a valid point if the other site does an ad hominen (These people are not technical, don't REalleY understand what they are talking about)
mrtranscendence 3y ago
Strictly speaking they said “a lot”, not “all”.
- ChatGTP 3y ago
  Fair point.
verdverm 3y ago
Why their doomerism over their peer's optimism. There are top tier AI researchers who think differently. How do you choose which to listen too?
- digging 3y ago
  If optimism is wrong, we all die. if pessimism is wrong, we're fine.
- mercurialsolo 3y ago
  The good thing about the scientific method is the changes happen in increments and we understand cause and effect through experimentation. Is there a case of a rogue uncontrollable agent and there being a sudden inflection point in AI evolution. I doubt that is going to be the case. Is there a need for exercising caution while designing AI models and giving them access to tools - yeah we should. I tend to say the case for being careful doesn't warrant that we do stop experimenting. Will we have some unwarranted effects because of this experimentation - Absolutely yes? Is there a need for regulation - this is just stifling competition and allowing more closed door experimentation. Once you are in a position of authority spreading doomerism and asking for regulation is a way to stifle competition.
fedeb95 3y ago
criticism is the core of scientific reasoning. Having a bias toward a position just because "an expert told so" when presented with criticism isn't scientific.
- verdverm 3y ago
  Remember COVID and how many people threw out the scientific method? I don't think we have fully recovered, it was as much a mind virus as a corona virus
- bamboozled 3y ago
  So if we don't at least take the word of experts seriously, who should we at least gain insight from? Internet Commenters?
  - coding123 3y ago
    what is an expert? how would I even know an expert is trustworthy without someone else saying someone is trustworthy. how do I trust that person? welcome to the post-truth world
    - vacuity 3y ago
      Truth has long been an uncertain thing, only perhaps people didn't know that at some time. There's a saying that "history is written by the victors", and that's not even quite true. The Internet has led to an information revolution, and so matters of truth and trust are all the murkier. Practically, some people recommend consuming multiple sources of different political leanings and believing the conserved details. That only goes so far, though; I doubt there's a silver bullet. Be cautious, and wary of your own moral axioms and cognitive biases, I guess.
    - ChatGTP 3y ago
      Honestly? We’re now having to define what an expert in the field is now ? This is a term we should all be familiar with before entering this type of discussion. I’m sure if you received diagnosis for a rare and life threatening illness and your doctor referred you to an expert in the field to help save your life. You’d be totally happy to believe in experts. I’m not ignorant to what you’re saying either. Not all experts are right and good people. But when forming an opinion it seems ridiculously naive and arrogant to discount expert opinion. This constant questioning of science institutions, science and authority is going to send us back into the dark ages if we’re not careful.
stabbles 3y ago
Maybe hackernews would benefit from an LLM that refuses appeal to authority type of arguments
- vacuity 3y ago
  Introducing HackerNews Bot 9000 TM In light of dang's (Dan G.) unfortunate passing, we recognize the loss of a core member of the HN community and the invaluable moderation services that are now absent. We honor dang's commitment to upholding civilized, productive discussions. Although dang certainly can't be replicated perfectly, we hope the HackerNews Bot 9000 TM serves as a reasonable substitute while we look for a better approach. this is satire this is satire this is satire this is satire
- bamboozled 3y ago
  Attack the arguments, not the people.
  - vacuity 3y ago
    Normally I would agree with that statement. I'm not sure if you say "people" to mean "proclaimed experts" or "people citing 'experts'", so I'll discuss both. For "proclaimed experts", there are often arguments on very uncertain topics such as this AI thread. When there isn't actually much in the way of concrete evidence, the kind that science is based on, the reputation of an "expert" may be a reasonable filter to avoid low quality discussion. Would that people don't take so much stock in these "experts" because the discussions themselves are full of glorified speculation, though. For "people citing 'experts'", someone who is citing "experts" that are of dubious background may not be arguing in good faith. At that juncture, it may be best to abandon the argument. I suppose that I don't think much of the discussions of AI here are productive because of the lack of concrete evidence and the slinging of personal beliefs as fact. They mostly come out as noise to me.
throwuwu 3y ago
Appeal to authority isn’t interesting. Tegmark is a physicist and science communicator anyways so he doesn’t belong in your list.
- skinner_ 3y ago
  Tegmark has his name on a bunch of competently written AI papers: https://arxiv.org/search/cs?searchtype=author&query=Tegmark%2C+M https://arxiv.org/search/cs?searchtype=author&query=Tegmark%... In each case, the first author is Ziming Liu, which is more impressive than Tegmark's last author (PI) position. But still, it's fair to call Tegmark an AI expert.
- js8 3y ago
  The PP is not appealing to authority, but making an argument against GP's claim that people are AI doomers because they are uninformed, arguing that many experts (presumably informed) are also AI doomers.
civilized 3y ago
Reminder that not all "experts" agree with the AI doom thesis. Yann LeCun thinks it's nonsense. I don't think anyone really has "expertise" to perform AI doom prognostication, regardless of whether they created dropout or ReLU or the Transformer architecture or are a professor at Oxbridge/MIT or whatever else people take as an impressive credential. But even if we accept the logic of those who want to argue from the authority of expertise, we don't have expert consensus.
- bamboozled 3y ago
  Did you catch the debate between them? Honestly LeCunn came off really badly in my opinion. Really gave no solid arguments against anything. I'm not arguing he was wrong, but man, there was almost zero substance to any of his refutations. The truth is, I'd love to just roll with Lecunism, it's much easier to just flip off all the concerns and not think too hard about it. But I'm not one to take the easier path so I feel more compelled to be open minded about both sides of the argument.
  - c_crank 3y ago
    The other side of the argument seems to hold very odd assumptions about things like thermodynamics and robotics work, or even intelligence itself. Think of the whole 'AI in a box' argument, which assumes a superintelligence has superior or infinite power to persuade. Do very intelligent humans have strong persuasive power over animals?
    - ekidd 3y ago
      > The other side of the argument seems to hold very odd assumptions about things like thermodynamics and robotics work, or even intelligence itself. There are two major types of arguments for why we should be cautious about ultra-smart AI: 1. "Ultra-smart AI will rapidly improve its own abilities to vastly super-human intelligence, invent nanotech, and kill us all. Maybe in a week or two." This argument does require a number of very specific assumptions to be true. 2. "If humans are the second-smartest species on the planet, we may not get to make all the important decisions anymore. We might not enjoy this." (2) doesn't require any exotic ideas about thermodynamics or nanotech. It does assume we might be able to build something that's noticeably smarter than we are. Honestly, even if the best AI we could be build was equivalent to a very smart human (say, an IQ of 145, or three standard deviations from average), and if that AI was tireless and easily copyable, then the future would become very weird. Personally, I give us reasonable odds of us making it this far within my normal lifetime. And it might be possible to build something smarter than that, given time, depending on how "intelligence" works.
ldehaan 3y ago
[dead]
drumhead 3y ago
Max Tegmark seems most worried about iterative self improvement by the systems. But is that possible? And just how far could they improve themselves? Enough to achieve self awareness or just enough to make themselves a lot faster?
- deleted 3y ago
  [deleted]
- Fricken 3y ago
  Is there any reasonable assumption to be made that AI has reached it's zenith? I don't think there is. AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do.
  - tome 3y ago
    > AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do. Why? Why might it not stop improving, like lots of things, burning paraffin for heat to pluck one example out of the air?
    - ethanbond 3y ago
      Because so far it continues to trend upwards? And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. A couple million of those alone would be unfathomable. Probably not any time soon, but insofar as we have reason to suspect there's an upper limit (which we don't, IMO), we're not even close to the upper known limit.
      - tome 3y ago
        > And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. Why? I really don't get it. Many, many commenters here and elsewhere have used the existence of the human brain as a supposed proof that we can create AIs with human-level intelligence. I just don't see why the conclusion follows from the observation. Can anyone flesh out the argument in more detail?
digbybk 3y ago
About “pandering to authority”, I keep seeing this move in debates: - people with safety concerns just don’t understand the tech. - “here’s a list of people with safety concerns who demonstrably do understand the tech very well” - you’re pandering to authority. A related move: - greater intelligence leads to greater concern for universal values. - “here’s a list of very intelligent people who disagree” - there are a lot of very intelligent people in cults too, you know. I don’t think I’m a doomer but they do seem more coherent than the people saying there’s no cause for concern.
- ethanbond 3y ago
  Yeah the other fun move is: AI will be powerful and unbridled from human control enough to achieve great outcomes that humans cannot conceive of. But in the negative case: AI will not be operating outside of human control, it will not be handed immense resources to pursue goals (verifiably false given the capital and compute investment already backing AI).
  - robertlagrant 3y ago
    It's not backing AI. It's backing humans to try and make an AI to do something.
    - ethanbond 3y ago
      What bearing do you think this has on the question at hand?
      - robertlagrant 3y ago
        It's not verifiably false that people want AI to be autonomous and it's not true that it has been handed immense resources. People have been handed some money to make better AIs to do things.
- pydry 3y ago
  I have safety concerns about stepladders but I'm not a stepladder doomerist. Which of these experts are doomers?
logicchains 3y ago
Yoshua Bengio is a plagiarist who shamelessly accepted a Turing Award for Schmidhuber's work, and Geoffrey Hinton is a leftist who can't bear the thought of an AI that isn't aligned to progressive values like the Silicon Valley ones are.
- digging 3y ago
  > Geoffrey Hinton is a leftist who can't bear the thought of an AI that isn't aligned to progressive values like the Silicon Valley ones are. Is this parody? Do you even know what progressive values are?

sampo 3y ago

> Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? I am genuinely amazed how even educated and intelligent people (or maybe predominantly them) have fallen into this, what I see is essentially a doomsday cult. But groupthink happens, maybe nobody is safe. It's not like it hasn't happened before.

DalasNoin 3y ago

People are downvoting you. But 'pandering' to authority is a valid point if the other site does an ad hominen (These people are not technical, don't REalleY understand what they are talking about)

mrtranscendence 3y ago

Strictly speaking they said “a lot”, not “all”.

ChatGTP 3y ago
Fair point.

verdverm 3y ago

Why their doomerism over their peer's optimism. There are top tier AI researchers who think differently. How do you choose which to listen too?

digging 3y ago
If optimism is wrong, we all die. if pessimism is wrong, we're fine.
mercurialsolo 3y ago
The good thing about the scientific method is the changes happen in increments and we understand cause and effect through experimentation. Is there a case of a rogue uncontrollable agent and there being a sudden inflection point in AI evolution. I doubt that is going to be the case. Is there a need for exercising caution while designing AI models and giving them access to tools - yeah we should. I tend to say the case for being careful doesn't warrant that we do stop experimenting. Will we have some unwarranted effects because of this experimentation - Absolutely yes? Is there a need for regulation - this is just stifling competition and allowing more closed door experimentation. Once you are in a position of authority spreading doomerism and asking for regulation is a way to stifle competition.

fedeb95 3y ago

criticism is the core of scientific reasoning. Having a bias toward a position just because "an expert told so" when presented with criticism isn't scientific.

verdverm 3y ago
Remember COVID and how many people threw out the scientific method? I don't think we have fully recovered, it was as much a mind virus as a corona virus
bamboozled 3y ago
So if we don't at least take the word of experts seriously, who should we at least gain insight from? Internet Commenters?
- coding123 3y ago
  what is an expert? how would I even know an expert is trustworthy without someone else saying someone is trustworthy. how do I trust that person? welcome to the post-truth world
  - vacuity 3y ago
    Truth has long been an uncertain thing, only perhaps people didn't know that at some time. There's a saying that "history is written by the victors", and that's not even quite true. The Internet has led to an information revolution, and so matters of truth and trust are all the murkier. Practically, some people recommend consuming multiple sources of different political leanings and believing the conserved details. That only goes so far, though; I doubt there's a silver bullet. Be cautious, and wary of your own moral axioms and cognitive biases, I guess.
  - ChatGTP 3y ago
    Honestly? We’re now having to define what an expert in the field is now ? This is a term we should all be familiar with before entering this type of discussion. I’m sure if you received diagnosis for a rare and life threatening illness and your doctor referred you to an expert in the field to help save your life. You’d be totally happy to believe in experts. I’m not ignorant to what you’re saying either. Not all experts are right and good people. But when forming an opinion it seems ridiculously naive and arrogant to discount expert opinion. This constant questioning of science institutions, science and authority is going to send us back into the dark ages if we’re not careful.

stabbles 3y ago

Maybe hackernews would benefit from an LLM that refuses appeal to authority type of arguments

vacuity 3y ago
Introducing HackerNews Bot 9000 TM In light of dang's (Dan G.) unfortunate passing, we recognize the loss of a core member of the HN community and the invaluable moderation services that are now absent. We honor dang's commitment to upholding civilized, productive discussions. Although dang certainly can't be replicated perfectly, we hope the HackerNews Bot 9000 TM serves as a reasonable substitute while we look for a better approach. this is satire this is satire this is satire this is satire
bamboozled 3y ago
Attack the arguments, not the people.
- vacuity 3y ago
  Normally I would agree with that statement. I'm not sure if you say "people" to mean "proclaimed experts" or "people citing 'experts'", so I'll discuss both. For "proclaimed experts", there are often arguments on very uncertain topics such as this AI thread. When there isn't actually much in the way of concrete evidence, the kind that science is based on, the reputation of an "expert" may be a reasonable filter to avoid low quality discussion. Would that people don't take so much stock in these "experts" because the discussions themselves are full of glorified speculation, though. For "people citing 'experts'", someone who is citing "experts" that are of dubious background may not be arguing in good faith. At that juncture, it may be best to abandon the argument. I suppose that I don't think much of the discussions of AI here are productive because of the lack of concrete evidence and the slinging of personal beliefs as fact. They mostly come out as noise to me.

throwuwu 3y ago

Appeal to authority isn’t interesting. Tegmark is a physicist and science communicator anyways so he doesn’t belong in your list.

skinner_ 3y ago
Tegmark has his name on a bunch of competently written AI papers: https://arxiv.org/search/cs?searchtype=author&query=Tegmark%2C+M https://arxiv.org/search/cs?searchtype=author&query=Tegmark%... In each case, the first author is Ziming Liu, which is more impressive than Tegmark's last author (PI) position. But still, it's fair to call Tegmark an AI expert.
js8 3y ago
The PP is not appealing to authority, but making an argument against GP's claim that people are AI doomers because they are uninformed, arguing that many experts (presumably informed) are also AI doomers.

civilized 3y ago

Reminder that not all "experts" agree with the AI doom thesis. Yann LeCun thinks it's nonsense. I don't think anyone really has "expertise" to perform AI doom prognostication, regardless of whether they created dropout or ReLU or the Transformer architecture or are a professor at Oxbridge/MIT or whatever else people take as an impressive credential. But even if we accept the logic of those who want to argue from the authority of expertise, we don't have expert consensus.

bamboozled 3y ago
Did you catch the debate between them? Honestly LeCunn came off really badly in my opinion. Really gave no solid arguments against anything. I'm not arguing he was wrong, but man, there was almost zero substance to any of his refutations. The truth is, I'd love to just roll with Lecunism, it's much easier to just flip off all the concerns and not think too hard about it. But I'm not one to take the easier path so I feel more compelled to be open minded about both sides of the argument.
- c_crank 3y ago
  The other side of the argument seems to hold very odd assumptions about things like thermodynamics and robotics work, or even intelligence itself. Think of the whole 'AI in a box' argument, which assumes a superintelligence has superior or infinite power to persuade. Do very intelligent humans have strong persuasive power over animals?
  - ekidd 3y ago
    > The other side of the argument seems to hold very odd assumptions about things like thermodynamics and robotics work, or even intelligence itself. There are two major types of arguments for why we should be cautious about ultra-smart AI: 1. "Ultra-smart AI will rapidly improve its own abilities to vastly super-human intelligence, invent nanotech, and kill us all. Maybe in a week or two." This argument does require a number of very specific assumptions to be true. 2. "If humans are the second-smartest species on the planet, we may not get to make all the important decisions anymore. We might not enjoy this." (2) doesn't require any exotic ideas about thermodynamics or nanotech. It does assume we might be able to build something that's noticeably smarter than we are. Honestly, even if the best AI we could be build was equivalent to a very smart human (say, an IQ of 145, or three standard deviations from average), and if that AI was tireless and easily copyable, then the future would become very weird. Personally, I give us reasonable odds of us making it this far within my normal lifetime. And it might be possible to build something smarter than that, given time, depending on how "intelligence" works.

ldehaan 3y ago

[dead]

drumhead 3y ago

Max Tegmark seems most worried about iterative self improvement by the systems. But is that possible? And just how far could they improve themselves? Enough to achieve self awareness or just enough to make themselves a lot faster?

deleted 3y ago
[deleted]
Fricken 3y ago
Is there any reasonable assumption to be made that AI has reached it's zenith? I don't think there is. AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do.
- tome 3y ago
  > AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do. Why? Why might it not stop improving, like lots of things, burning paraffin for heat to pluck one example out of the air?
  - ethanbond 3y ago
    Because so far it continues to trend upwards? And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. A couple million of those alone would be unfathomable. Probably not any time soon, but insofar as we have reason to suspect there's an upper limit (which we don't, IMO), we're not even close to the upper known limit.
    - tome 3y ago
      > And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. Why? I really don't get it. Many, many commenters here and elsewhere have used the existence of the human brain as a supposed proof that we can create AIs with human-level intelligence. I just don't see why the conclusion follows from the observation. Can anyone flesh out the argument in more detail?

digbybk 3y ago

About “pandering to authority”, I keep seeing this move in debates: - people with safety concerns just don’t understand the tech. - “here’s a list of people with safety concerns who demonstrably do understand the tech very well” - you’re pandering to authority. A related move: - greater intelligence leads to greater concern for universal values. - “here’s a list of very intelligent people who disagree” - there are a lot of very intelligent people in cults too, you know. I don’t think I’m a doomer but they do seem more coherent than the people saying there’s no cause for concern.

ethanbond 3y ago
Yeah the other fun move is: AI will be powerful and unbridled from human control enough to achieve great outcomes that humans cannot conceive of. But in the negative case: AI will not be operating outside of human control, it will not be handed immense resources to pursue goals (verifiably false given the capital and compute investment already backing AI).
- robertlagrant 3y ago
  It's not backing AI. It's backing humans to try and make an AI to do something.
  - ethanbond 3y ago
    What bearing do you think this has on the question at hand?
    - robertlagrant 3y ago
      It's not verifiably false that people want AI to be autonomous and it's not true that it has been handed immense resources. People have been handed some money to make better AIs to do things.
pydry 3y ago
I have safety concerns about stepladders but I'm not a stepladder doomerist. Which of these experts are doomers?

logicchains 3y ago

Yoshua Bengio is a plagiarist who shamelessly accepted a Turing Award for Schmidhuber's work, and Geoffrey Hinton is a leftist who can't bear the thought of an AI that isn't aligned to progressive values like the Silicon Valley ones are.

digging 3y ago
> Geoffrey Hinton is a leftist who can't bear the thought of an AI that isn't aligned to progressive values like the Silicon Valley ones are. Is this parody? Do you even know what progressive values are?