3 ms·
>1) Why do we have a reason to believe there’s a cap to intelligence? There's a cap to all things. Diminishing returns is a thing. Both physical substrates hav
by coldtea 4y ago
>1) Why do we have a reason to believe there’s a cap to intelligence?
There's a cap to all things. Diminishing returns is a thing. Both physical substrates have limits, and there's a utility limit to thinking or having more data. Limitless intelligence is fairy-tale magical thinking - especially in the way it's portrayed as some magical ability to build and control everything. Like, invent working cold-fusion - as if you don't need the experiments, iterative process, random breakthroughs, and such, but just "smarter" thinking.
> 2) No, you don’t need to assume it happens via adding hardware
Yes, you do, else it's just the same capped-capability hardware, thinking itself into being smarter.
> 3) No, you don’t need to assume it can expand by itself
Well, I, personally, don't need to assume anything. But people talking about AGI, do assume it.
> 4) We’ve already seen people connecting AI systems to open internet and open internet to command and control of physical systems (cars, utilities, soon weapons systems) - turns out there was no convincing necessary
That's neither here nor there, though, because I didn't say that connecting AGI to the open internet is impossible or can't happen. Just that it doesn't mean much. We still control the kind of control we give an AI, and any physical systems any AGI would be connected will still have limitations. It's not some magical portal into conquering the world.
>5) Nanobots also not a necessary assumption
Didn't say it's necessary, just pointed out that it has been made - along with many other non necessary, and far-fetched assumptions and hand waving...
>Also, incentives, not charisma, are what steer human behavior.
Both do. With charisma you can do it without incentives - and even instill incentives into people. Otherwise you're stuck with innate incentives like people wanting to be rich - which you don't need to be a AGI to exploit, just rich or a con-man.
- ethanbond 4y ago1) Obviously intelligence plays a role in how fast science develops. The quality of experiments and the quality of inference from those experiments are (very obviously) modulated by intelligence. No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Lots of reasons to believe fundamental understanding of the universe is more accessible to intelligent beings than to unintelligent ones. 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power. 3) Sure some do, but it’s not a necessary assumption to take AGI risk seriously. 4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad” 5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem? 6) So we agree charisma is at best not necessary to manipulate the world
- coldtea 4y ago>Obviously intelligence plays a role in how fast science develops Sort of. Most science is based on tinkering and experiment, with serendipity thrown in, including most major inventions, and in many cases the actual invention history is changed to make it more like a story of rational thinking process and regular steps than it was. >No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Note that "hairless apes" is a weasel term, meant to psychologically-pressure the other to feel as if we're talking about some inherently inferior thing ("sounds right, mostly hairless apes can't be the pinnacle of intelligence"). Whereas if you had put it as "the best achievable result after hundreds of millions of years of evolution" it doesn't sound so bad, does it? > 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power. Potatoh, potato. Different hardware is still hardware - and will also have limitations of its own. >4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad” Which is an appropriate scalable multi-case response to the similarly multi-level: "AI will control the world through physical access!!!!! SINGULARITY IS COMING! WE'RE ALL DOOMED" which then in practice turns to mean "My smart fridge is connected to Amazon". >5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem? It's related to the conversation about AGI, since its what one of the most visible proponent of it's dangers wrote as his feared very possible scenario. Also, if one believes people warning about some subject are delluded, pointing to delluded things they've said about it looks pretty clearly pertinent. >6) So we agree charisma is at best not necessary to manipulate the world Sure, in the degree that we agree that intelligence alone is not sufficient.
- ethanbond 4y agoSure, that’s true. And intelligence very clearly seems to matter. Not many 80 IQ folks are pushing the frontiers of understanding. The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come. There’s a difference between being the current known hyper local maximum and being at the universal limit. Talk about weaseling! :D No reason to believe we are the absolute pinnacle. There’s immense variation even within our own species, not to mention our near neighbors on evolution’s path. Edit: No one is worried about your fridge. Anyway I’ve gotta start my day, have a good rest of your week!