3 ms·

I have come across a few great comments on intelligence here on HN — intelligence is not the bottleneck of almost anything we do. Let’s say you became “comicall

by kaba0 4y ago

I have come across a few great comments on intelligence here on HN — intelligence is not the bottleneck of almost anything we do. Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful. We also regularly see these “with my super intelligence I predicted your next 347 steps” type of thinking, but that’s just bullshit. The bottleneck here is data if anything, so intelligence won’t aid much in itself. The thing is, we live in an infinitely complex game with independent agents, many of which don’t operate on any sane strategy even. Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win? (Of course I’m no expert on any of these topics, so take all these with a huge grain of salt)

NumberWangMan 4y ago
> Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful. Hmm, that's assuming that PhD students want to be CEOs. People become CEOs because they like money, but also because they like running companies. For a PhD student, that might mean days full of people coming to them with problems they have no interest in solving. Sure, PhD students want money, but if getting money means sacrificing years of your life doing work you don't enjoy, it's not generally worth it. I don't disagree that luck is a huge factor in most people's success. That said, you're unlikely to become a multi-billionaire without substantial help from your brain. There are other factors, like lack of shame or embarrasment, that could also be considered a form of intelligence, or rather an adaptation to a particular kind of environment. They might make you less popular in some situations, but more likely to succeed in business. Anyway, I agree that there are multiple factors. Information is for sure a bottleneck, that's a good point. In a game like chess, you have perfect info, which is not the case in the real world. More intelligence lets you draw better conclusions with less data, but it's not omniscience. One big question is, how close are humans to the theoretical maximum of intelligence? Some people think we're quite a ways away. When AGI is developed, is it going to blow way past us, to the point where we can't even understand the concepts it comes up with? I don't know! > Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win? I think so, wouldn't it? For trading properties and so on. I'm not sure if I'm getting the point you're aiming at, though, so apologies. Maybe another analogy is poker, where your opponents haven't figured out game theory and play on intuition. Having a better model for how the game works is absolutely a big advantage that stacks up over time, even with the randomness. Maybe, as an AI, you just look for small advantages here and there, playing humans off each other, gaining trust and influence, acting helpful, until you're in a position of power. Remember that an AI is effectively immortal, and possibly very patient. One scenario of concern is that AI takes over, it just takes a generation or two, after we have all thought that we were past the hard parts. At that point, we might be able to digitize human brains, or have solved the AI alignment problem, so maybe it's a hopeful scenario. And then a big variable is how many super-intelligent AGIs there are. Once we develop it, are they going to be carefully restricted, or are we going to test them until we think they are safe enough, then copy them everywhere? These might look very different. I'm also not an expert, so same for me! Grains of salt all around.
d0mine 4y ago
You are comically underestimating intelligence. For example, compare your level of intelligence with that of an ant, to get a feeling of what it might feel like to be "comically" smarter. It is just on entirely another level. It is not about predicting the next 100 steps an ant makes. It is about setting ant baits where you don't want the ants.

by kaba0 4y ago

NumberWangMan 4y ago

> Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful. Hmm, that's assuming that PhD students want to be CEOs. People become CEOs because they like money, but also because they like running companies. For a PhD student, that might mean days full of people coming to them with problems they have no interest in solving. Sure, PhD students want money, but if getting money means sacrificing years of your life doing work you don't enjoy, it's not generally worth it. I don't disagree that luck is a huge factor in most people's success. That said, you're unlikely to become a multi-billionaire without substantial help from your brain. There are other factors, like lack of shame or embarrasment, that could also be considered a form of intelligence, or rather an adaptation to a particular kind of environment. They might make you less popular in some situations, but more likely to succeed in business. Anyway, I agree that there are multiple factors. Information is for sure a bottleneck, that's a good point. In a game like chess, you have perfect info, which is not the case in the real world. More intelligence lets you draw better conclusions with less data, but it's not omniscience. One big question is, how close are humans to the theoretical maximum of intelligence? Some people think we're quite a ways away. When AGI is developed, is it going to blow way past us, to the point where we can't even understand the concepts it comes up with? I don't know! > Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win? I think so, wouldn't it? For trading properties and so on. I'm not sure if I'm getting the point you're aiming at, though, so apologies. Maybe another analogy is poker, where your opponents haven't figured out game theory and play on intuition. Having a better model for how the game works is absolutely a big advantage that stacks up over time, even with the randomness. Maybe, as an AI, you just look for small advantages here and there, playing humans off each other, gaining trust and influence, acting helpful, until you're in a position of power. Remember that an AI is effectively immortal, and possibly very patient. One scenario of concern is that AI takes over, it just takes a generation or two, after we have all thought that we were past the hard parts. At that point, we might be able to digitize human brains, or have solved the AI alignment problem, so maybe it's a hopeful scenario. And then a big variable is how many super-intelligent AGIs there are. Once we develop it, are they going to be carefully restricted, or are we going to test them until we think they are safe enough, then copy them everywhere? These might look very different. I'm also not an expert, so same for me! Grains of salt all around.

d0mine 4y ago

You are comically underestimating intelligence. For example, compare your level of intelligence with that of an ant, to get a feeling of what it might feel like to be "comically" smarter. It is just on entirely another level. It is not about predicting the next 100 steps an ant makes. It is about setting ant baits where you don't want the ants.