3 ms·
> as critical thinking is a key component of intelligence. When I evaluate this statement, my brain raises a type error. Intelligence is a lot of things -- co
by ctoth 3y ago
> as critical thinking is a key component of intelligence.
When I evaluate this statement, my brain raises a type error.
Intelligence is a lot of things -- compression among them, and yes possibly an RL-based AI would use an actor-critic approach for evaluating its actions, but I doubt that at all maps onto the human activity we call "critical thinking."
To me, critical thinking involves stuff like questioning assumptions, logical reasoning, weighing whatever I'm thinking about against my experience with similar situations previously, yada yada, all stuff that are symptoms of intelligence but I am not at all sure are the actual embodiment there of.
I really don't see that critical thinking is at all required for a raw optimization process.
The problem they are trying to solve is what happens when that optimization process isn't aligned with human flourishing?
Think about it another way. Covid was a dumb optimization process, only evolutionarily-guided, and it still hit us pretty hard!
Edit: Another interesting way I just thought about this that might support your idea more is, of course critical thinking is the sort of thing that a "better" brain would do automatically, it would just be thinking.
Of course, we can't know that it's thinking "good" things--we can't even know if other humans are! So it's probably a good idea to figure out how to influence that sort of thing before making something with regular thinking which is equivalent to or superior to our critical thinking.
- logicchains 3y ago>I really don't see that critical thinking is at all required for a raw optimization process. The problem they are trying to solve is what happens when that optimization process isn't aligned with human flourishing. I agree it's possible to have a dangerous AI that lacks human "critical thinking", but I don't think it's reasonable to refer to an AI as much more intelligent than humans if there's any class of intellectual tasks humans can do but the AI cannot.
- unlikelymordant 3y agoIf a 'superintelligence' achieves the same outcomes as humans without engaging in the same class of intellectual tasks that humans do, wouldnt it still be a superitelligence? Deep blue was beating everybody at chess without engaging in the same process as humans. If chess is a metaphor for life, it seems some algorithm might do better at all the things a human does while not arriving at its decisions in a remotely similar way.
- ball_of_lint 3y ago(argued at various places better than I can. For example https://www.lesswrong.com/posts/7dkH5i7T8a78Da3ty/why-will-an-agi-be-rational https://www.lesswrong.com/posts/7dkH5i7T8a78Da3ty/why-will-a... ) Rationality is an attractor for high intelligence architectures. You may be able to get good results another way (and even good results that surpass a human) but at the limit rationality is the way to have high intelligence.