The only bit of the book Superintelligence that I liked was the historical bit at the start, where it described the problem AI researchers have: as soon as they

The only bit of the book Superintelligence that I liked was the historical bit at the start, where it described the problem AI researchers have: as soon as they achieve a milestone in AI, the goalposts shift and that milestone is no longer considered an AI marker. The simplest example was "can make a human believe they're talking to another human", which was achieved with ELIZA. Before then, what ELIZA could do was "AI". After, it was "just this thing, you know"...