4 ms·

1) Obviously intelligence plays a role in how fast science develops. The quality of experiments and the quality of inference from those experiments are (very ob

1) Obviously intelligence plays a role in how fast science develops. The quality of experiments and the quality of inference from those experiments are (very obviously) modulated by intelligence. No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Lots of reasons to believe fundamental understanding of the universe is more accessible to intelligent beings than to unintelligent ones. 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power. 3) Sure some do, but it’s not a necessary assumption to take AGI risk seriously. 4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad” 5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem? 6) So we agree charisma is at best not necessary to manipulate the world

coldtea 4y ago
>Obviously intelligence plays a role in how fast science develops Sort of. Most science is based on tinkering and experiment, with serendipity thrown in, including most major inventions, and in many cases the actual invention history is changed to make it more like a story of rational thinking process and regular steps than it was. >No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Note that "hairless apes" is a weasel term, meant to psychologically-pressure the other to feel as if we're talking about some inherently inferior thing ("sounds right, mostly hairless apes can't be the pinnacle of intelligence"). Whereas if you had put it as "the best achievable result after hundreds of millions of years of evolution" it doesn't sound so bad, does it? > 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power. Potatoh, potato. Different hardware is still hardware - and will also have limitations of its own. >4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad” Which is an appropriate scalable multi-case response to the similarly multi-level: "AI will control the world through physical access!!!!! SINGULARITY IS COMING! WE'RE ALL DOOMED" which then in practice turns to mean "My smart fridge is connected to Amazon". >5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem? It's related to the conversation about AGI, since its what one of the most visible proponent of it's dangers wrote as his feared very possible scenario. Also, if one believes people warning about some subject are delluded, pointing to delluded things they've said about it looks pretty clearly pertinent. >6) So we agree charisma is at best not necessary to manipulate the world Sure, in the degree that we agree that intelligence alone is not sufficient.
- ethanbond 4y ago
  Sure, that’s true. And intelligence very clearly seems to matter. Not many 80 IQ folks are pushing the frontiers of understanding. The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come. There’s a difference between being the current known hyper local maximum and being at the universal limit. Talk about weaseling! :D No reason to believe we are the absolute pinnacle. There’s immense variation even within our own species, not to mention our near neighbors on evolution’s path. Edit: No one is worried about your fridge. Anyway I’ve gotta start my day, have a good rest of your week!
  - coldtea 4y ago
    >The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come. Still, this formulation heavily beats the "hairless apes" one doesn't it? Also, I'm not sure about the relevance of pointing out that it's the "best known". Or the number of stars. Are we aware of any "not known"? Or have seen any signs in any of those stars? Because decades of trying (with SETI, etc.) and so on, has mostly shown the opposite (consider e.g. the Fermi paradox). So, I think we better stick with the knowns (of which it's the best results among millions of competing organisms anyway), than handwave about what possible magic intelligence remote aliens we haven't seen and don't even know exist have... Sure, we can do it for fun and speculation, but we shouldn't use it as an argument for limitless intelligence. I mean, "Intelligence is limitless, because if E.T. exists, he could have 1000x our IQ" is not exactly an argument. >Edit: No one is worried about your fridge. No, but they're worried about the imminent danger of AGI conquering the world and killing us all when the practical examples attaibable are not far from the level of my fridge. Have a good week yourself. See you around!

coldtea 4y ago

>Obviously intelligence plays a role in how fast science develops Sort of. Most science is based on tinkering and experiment, with serendipity thrown in, including most major inventions, and in many cases the actual invention history is changed to make it more like a story of rational thinking process and regular steps than it was. >No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Note that "hairless apes" is a weasel term, meant to psychologically-pressure the other to feel as if we're talking about some inherently inferior thing ("sounds right, mostly hairless apes can't be the pinnacle of intelligence"). Whereas if you had put it as "the best achievable result after hundreds of millions of years of evolution" it doesn't sound so bad, does it? > 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power. Potatoh, potato. Different hardware is still hardware - and will also have limitations of its own. >4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad” Which is an appropriate scalable multi-case response to the similarly multi-level: "AI will control the world through physical access!!!!! SINGULARITY IS COMING! WE'RE ALL DOOMED" which then in practice turns to mean "My smart fridge is connected to Amazon". >5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem? It's related to the conversation about AGI, since its what one of the most visible proponent of it's dangers wrote as his feared very possible scenario. Also, if one believes people warning about some subject are delluded, pointing to delluded things they've said about it looks pretty clearly pertinent. >6) So we agree charisma is at best not necessary to manipulate the world Sure, in the degree that we agree that intelligence alone is not sufficient.

ethanbond 4y ago
Sure, that’s true. And intelligence very clearly seems to matter. Not many 80 IQ folks are pushing the frontiers of understanding. The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come. There’s a difference between being the current known hyper local maximum and being at the universal limit. Talk about weaseling! :D No reason to believe we are the absolute pinnacle. There’s immense variation even within our own species, not to mention our near neighbors on evolution’s path. Edit: No one is worried about your fridge. Anyway I’ve gotta start my day, have a good rest of your week!
- coldtea 4y ago
  >The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come. Still, this formulation heavily beats the "hairless apes" one doesn't it? Also, I'm not sure about the relevance of pointing out that it's the "best known". Or the number of stars. Are we aware of any "not known"? Or have seen any signs in any of those stars? Because decades of trying (with SETI, etc.) and so on, has mostly shown the opposite (consider e.g. the Fermi paradox). So, I think we better stick with the knowns (of which it's the best results among millions of competing organisms anyway), than handwave about what possible magic intelligence remote aliens we haven't seen and don't even know exist have... Sure, we can do it for fun and speculation, but we shouldn't use it as an argument for limitless intelligence. I mean, "Intelligence is limitless, because if E.T. exists, he could have 1000x our IQ" is not exactly an argument. >Edit: No one is worried about your fridge. No, but they're worried about the imminent danger of AGI conquering the world and killing us all when the practical examples attaibable are not far from the level of my fridge. Have a good week yourself. See you around!