IMO the one thing we really need is some reliable way to do online learning. If we can get NNs to the point where you can just say, somewhere in code, "make thi

IMO the one thing we really need is some reliable way to do online learning. If we can get NNs to the point where you can just say, somewhere in code, "make this decision NNly, based on this set of features" and get reasonably good decisionmaking if the decision is in some sense approximable as a function of the features, I think GOFAI suddenly becomes feasible again. We would then enter an area of risk where we don't really understand NNs and we don't really understand the algorithms we can build with them either, but these algorithms could still have a lot of power. The aphorism used to be "hard things are easy and easy things are hard". Ie. arithmetic with large numbers is trivial, but opening a door or recognizing a label is the most difficult problem in the world. We're starting to close in on reliable, possibly prepackaged solutions for the latter; on the other hand, understanding the logic of intelligence turns out to have some teeth after all. Reality would do us a great favor if building genuine intelligence required actual understanding and knowledge of how to build reliable cognition; unfortunately our precedent suggests it's mostly done by kludging a half-baked barely-debugged motivation system onto a big heap of instinct-driven neural networks. I'm worried some spark will get the bright idea to combine some approximating variant of AIXI and a neural network for theory selection...