4 ms·

It seems to me that the bitter lesson is a counter-argument of sorts to intelligence explosion: the idea that stacking more layers is more effective than trying

It seems to me that the bitter lesson is a counter-argument of sorts to intelligence explosion: the idea that stacking more layers is more effective than trying to be really clever about it may well persist through all levels of intelligence. To put it in another way, if the bottleneck in the progress of intelligence, or of technology in general, is not intelligence, then greater intelligence will be of limited utility in accelerating it.

loandbehold 3y ago
The bitter lesson just says that general purpose machine learning algorithms with large amount of compute work better than domain-specific algorithms. It doesn't say that general purpose algorithms cannot be optimized. General purpose machine learning system that's able to optimize itself will lead to intelligence explosion.
- breuleux 3y ago
  I do think that there is also a "bitter lesson" in optimization: simple optimizations are usually more effective, robust and powerful than complicated ones. Also, optimization becomes progressively harder as the system trends to optimal performance, which is likely to cancel out the advantage of higher intelligence in finding them.
  - comex 3y ago
    Also, even to the extent model architecture can be optimized, in the short term, thanks to the bitter lesson, the bottleneck seems to be less “human intelligence coming up with new ideas for optimizations” and more “compute time to test ideas on sufficiently large models”. An AI that followed the same basic thought process as humans, even if its ideas were a little better than humans’ on average, probably wouldn’t help that much. On the other hand, an AI that in some sense ‘understands’ its own structure, which is to us almost entirely a black box, might be able to do much better. Consider OpenAI’s attempt to make GPT-4 explain the purpose of neurons in GPT-2; it basically didn’t work at all, but it would have opened a huge new frontier of research if it had. But that understanding ability is hard to train, given the lack of any meaningful training data. If it does show up, it could act as an accelerant. But it could just as easily not show up, at least not until AI is already comfortably superhuman in the “thinking like a human” department – in which case it won’t help us reach that point in the first place.
    - breuleux 3y ago
      > On the other hand, an AI that in some sense ‘understands’ its own structure, which is to us almost entirely a black box, might be able to do much better. True... but is this a plausible capability? I don't mean in practice. I mean theoretically. I feel we often forget about the free lunch theorem when it comes to intelligence. To us it is this magical, omnipotent thing, such that a sufficiently advanced intelligence can understand anything. But mathematically, there is no such thing as an algorithm that is better than another at everything: whatever intelligence is, it has to be intrinsically bad at certain things. Which brings me to this thought: I think it is highly plausible that understanding one's own internal structure intimately enough to reorganize and optimize it is in fact one of these things Intelligence (with a big I: all levels of intelligence) is intrinsically bad at, both directly and indirectly. What makes me think it is plausible is that intelligence is mainly about manipulating abstractions, but reality is far too messy to be abstracted... unless you simplify it. And so it seems to me that what makes humanity incredibly powerful is the one-two punch of morseling reality into simple pieces that match our abstractions, and building with these pieces. In other words, intelligence is only effective over an ablated version of reality, and we happen to be very good at ablating... reality itself. But if the thing you are trying to understand is truly complex, you can't really ablate it to a point where it is simple enough to be understood, and basically every single level of intelligence will fail at understanding it. There has to be some kind of tradeoff to intelligence's uncanny ability to manipulate and understand systems, and I think this is where it is. I very well could be wrong, of course.
      - loandbehold 3y ago
        AI risk argument is that human brain "hardware" is suboptimal as it relies on relatively slow electrochemical communication between neurons. Even with same algorithms, performance of digital intelligence will be orders of magnitude higher simply because of speed and memory capacity. Evolution rarely achieves optimal performance (optimal within constrains of physics). Evolution is iterative and tends to find local maxima. Think how we can build construction equipment thousands times stronger than any animal's muscle. And for AGI to improve itself — it's not necessary to be able to model itself with 100% accuracy. It only needs to understand itself at a high level, kinda like the way human understands its own body. Software engineers work on systems they don't completely understand all the time. E.g. AGI may notice inefficient memory use in its own implementation. It will try to fix it and run copy of itself. If it finds it indeed improves performance it will "upgrade" itself to new version, maybe just keeping one instance of previous version as a backup just in case. Now new version is smarter and is able to come up with even better optimizations. Eventually it will exhaust all possible software optimizations and will need to build new hardware to keep improving. How it will do it? Use your creativity. There are many ways. With ability to build custom hardware the only limit is now physics. It will keep improving and getting smarter until it reaches physical limits of computation. At this point it's billions time smarter than human.

loandbehold 3y ago

The bitter lesson just says that general purpose machine learning algorithms with large amount of compute work better than domain-specific algorithms. It doesn't say that general purpose algorithms cannot be optimized. General purpose machine learning system that's able to optimize itself will lead to intelligence explosion.

breuleux 3y ago
I do think that there is also a "bitter lesson" in optimization: simple optimizations are usually more effective, robust and powerful than complicated ones. Also, optimization becomes progressively harder as the system trends to optimal performance, which is likely to cancel out the advantage of higher intelligence in finding them.
- comex 3y ago
  Also, even to the extent model architecture can be optimized, in the short term, thanks to the bitter lesson, the bottleneck seems to be less “human intelligence coming up with new ideas for optimizations” and more “compute time to test ideas on sufficiently large models”. An AI that followed the same basic thought process as humans, even if its ideas were a little better than humans’ on average, probably wouldn’t help that much. On the other hand, an AI that in some sense ‘understands’ its own structure, which is to us almost entirely a black box, might be able to do much better. Consider OpenAI’s attempt to make GPT-4 explain the purpose of neurons in GPT-2; it basically didn’t work at all, but it would have opened a huge new frontier of research if it had. But that understanding ability is hard to train, given the lack of any meaningful training data. If it does show up, it could act as an accelerant. But it could just as easily not show up, at least not until AI is already comfortably superhuman in the “thinking like a human” department – in which case it won’t help us reach that point in the first place.
  - breuleux 3y ago
    > On the other hand, an AI that in some sense ‘understands’ its own structure, which is to us almost entirely a black box, might be able to do much better. True... but is this a plausible capability? I don't mean in practice. I mean theoretically. I feel we often forget about the free lunch theorem when it comes to intelligence. To us it is this magical, omnipotent thing, such that a sufficiently advanced intelligence can understand anything. But mathematically, there is no such thing as an algorithm that is better than another at everything: whatever intelligence is, it has to be intrinsically bad at certain things. Which brings me to this thought: I think it is highly plausible that understanding one's own internal structure intimately enough to reorganize and optimize it is in fact one of these things Intelligence (with a big I: all levels of intelligence) is intrinsically bad at, both directly and indirectly. What makes me think it is plausible is that intelligence is mainly about manipulating abstractions, but reality is far too messy to be abstracted... unless you simplify it. And so it seems to me that what makes humanity incredibly powerful is the one-two punch of morseling reality into simple pieces that match our abstractions, and building with these pieces. In other words, intelligence is only effective over an ablated version of reality, and we happen to be very good at ablating... reality itself. But if the thing you are trying to understand is truly complex, you can't really ablate it to a point where it is simple enough to be understood, and basically every single level of intelligence will fail at understanding it. There has to be some kind of tradeoff to intelligence's uncanny ability to manipulate and understand systems, and I think this is where it is. I very well could be wrong, of course.
    - loandbehold 3y ago
      AI risk argument is that human brain "hardware" is suboptimal as it relies on relatively slow electrochemical communication between neurons. Even with same algorithms, performance of digital intelligence will be orders of magnitude higher simply because of speed and memory capacity. Evolution rarely achieves optimal performance (optimal within constrains of physics). Evolution is iterative and tends to find local maxima. Think how we can build construction equipment thousands times stronger than any animal's muscle. And for AGI to improve itself — it's not necessary to be able to model itself with 100% accuracy. It only needs to understand itself at a high level, kinda like the way human understands its own body. Software engineers work on systems they don't completely understand all the time. E.g. AGI may notice inefficient memory use in its own implementation. It will try to fix it and run copy of itself. If it finds it indeed improves performance it will "upgrade" itself to new version, maybe just keeping one instance of previous version as a backup just in case. Now new version is smarter and is able to come up with even better optimizations. Eventually it will exhaust all possible software optimizations and will need to build new hardware to keep improving. How it will do it? Use your creativity. There are many ways. With ability to build custom hardware the only limit is now physics. It will keep improving and getting smarter until it reaches physical limits of computation. At this point it's billions time smarter than human.