Here's an extensive reply to Chollet's essay, also by the author of "There's No Fire Alarm": https://intelligence.org/2017/12/06/chollet/ https://intelligence.

Here's an extensive reply to Chollet's essay, also by the author of "There's No Fire Alarm": https://intelligence.org/2017/12/06/chollet/ https://intelligence.org/2017/12/06/chollet/ > ...some systems function very well in a broad variety of structured low-entropy environments. E.g. the human brain functions much better than other primate brains in an extremely broad set of environments, including many that natural selection did not explicitly optimize for. We remain functional on the Moon, because the Moon has enough in common with the Earth on a sufficiently deep meta-level that, for example, induction on past experience goes on functioning there. >> The intelligence of an octopus is specialized in the problem of being an octopus. The intelligence of a human is specialized in the problem of being human. > The problem that a human solves is much more general than the problem an octopus solves, which is why we can walk on the Moon and the octopus can’t. >> Recursively self-improving systems, because of contingent bottlenecks, diminishing returns, and counter-reactions arising from the broader context in which they exist, cannot achieve exponential progress in practice. Empirically, they tend to display linear or sigmoidal improvement. > Falsified by a graph of world GDP on almost any timescale.