4 ms·
AI might not recursively self improve (part 2)
- techpineapple 1y agoSo, nothing about my mental model of the world thinks that AI recursively self-improving (limitlessly) is possible but wouldn't a basic sort of math-y version of this be that most improvements AI can find in itself would be linear, but the marginal cost of improvements would be exponential? I get that none of this stuff is like solid, but recursive self-improvement feels similar to a perpetual motion machine. Base assumptions about the fundamental nature of the world have changed in the past (I like some of the examples in `sapiens`, even arguing that the idea of being able to improve our environment as a species was not believed for most of history) so I could be wrong, but fast takeoff feels like a fantasy. I also think about cookie clicker math. A new technological paradigm may give us exponential improvement, but we eventually hit the next exponential. i.e. when you buy a big upgrade in cookie clicker, it can start making your current exponential (say if you're at millions of cookies) really fast, the wheel spins really fast until you hit about 1 billion in which case the wheel slows down to a crawl.
- ffwd 1y agoYeah interesting I have to think about this.
- darad 1y agotechnically its not possible to have AI limitlessly improve just because nothing can hold infinity for to hold infinity you yourself have to be infinite(storage and data)