the GP stated: > It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress. So I point out a recent example

the GP stated: > It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress. So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory". Why would you compare this with performing manual backpropagation? >Everything is easy in hindsight Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation... The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.