The idea is that the things doing the designing are also hard at work redesigning themselves, presumably with a goal to get better at that, too. At least until

The idea is that the things doing the designing are also hard at work redesigning themselves, presumably with a goal to get better at that, too. At least until intelligence reaches some sort of plateau, which would almost certainly be hit at some level way above human ability, it makes sense to assume that some sort of locally exponential or even super-exponential intelligence growth would occur. That is, if humans are able to start the chain reaction and come up with something that's able to rewrite its own brain so that it's smarter than it started...a lump of plutonium that's not at critical mass is just going to sit there uselessly spewing out a slow trickle of neutrons, which is more or less what's happening with AI today.