It totally could be meaningless. I think there is this conflation of two thing. I don't think "General Intelligence" is a thing. We don't have it, AIs probably

It totally could be meaningless. I think there is this conflation of two thing. I don't think "General Intelligence" is a thing. We don't have it, AIs probably won't. We have a collection of specialty neurological adaptations that are useful but certainly not adapted for "any" use. Even if they can be composed to be more powerful, there are a lot of limits to it. This doesn't mean that AI won't get smarter than us at everything we can do, but based on the fact we are not getting much smarter we have to ask: is that just a limit of biology? Or are there scaling limits to Intelligence? Organizations and societies are a great example of the scaling limits of "Intelligence", they bypass a lot of human limits to make gesalt intelligent agents more reliable and less prone to certain biases, but generally not meaningfully "smarter" than any individual, just more patient and consistent. There are clear limits on the scalability Intelligence, just throwing more computational power at the problem isn't enough for runaway-agi