> But that's not a question of fairness, rather it's a question of feasibility. I initially took the challenge's intention to be about highlighting modern mach

> But that's not a question of fairness, rather it's a question of feasibility. I initially took the challenge's intention to be about highlighting modern machine learning's weaknesses compared to biological intelligence, and so barring certain already-existing generalization techniques seemed an arbitrary and asymmetrical restriction. If it's more meant as "Classifiers can already achieve this particular goal, but I have a theory that human-determined 'predicates' will scale up better in the long run, I challenge you to progress my idea", then I currently disagree but understand. > If it took us many thousands of years to learn our background knowledge from the real world over many human generations, it's difficult to see how we can reproduce this result with the comparatively poor computational resources and data in our disposal. My belief would be that we can surpass this result with a combination of using our existing intuition alongside techniques that outperform evolution's hypo-glacial pace and inefficient data utilization. Given the same narrow problem, some human insight for the search space and a couple of hours of gradient descent on a GPU can match what would take evolution many generations. That doesn't prove we'll get such a speedup on achieving broader intelligence, but at least natural selection hasn't appeared to be a speed limit so far. > or, we can find a way to transfer the background knowledge bestowed upon us by thousand years of evolution to guide the training of our learning systems towards the goals we want them to achieve, whatever those are. > To clarify, I'm not saying we should go back to feature engineering > Clearly not explicitly coding expert knowledge in production rules as in expert systems. Much of our knowledge is maybe impossible to articulate explicitly. So we must find a way to encode implicit knowledge, also. I'm all for finding ways to use human domain knowledge to guide the network in the right direction, essentially making use of a gigantic dataset from life's history. The trend seems to be to do this at an increasingly high level: weights are found by gradient descent, and hyperparameters by NAS or similar, but humans still designing various layers and blocks. "Predicates" being a probably-small set of feature detectors which can describe all 2D images makes me think of something like eigenfaces, which it felt backwards to have humans determine. Maybe intended to be broader than that? > It's basically a trade-off. If you have good background knowledge, you don't need a lot of data. Good background knowledge helps you build robustly generalisable concepts. And if you can reuse the learned concepts as background knowledge, then the sky is the limit. I'd claim that this is in effect also what pretraining is. Pretraining and instilling a model with human background knowledge both allow faster low-data generalization to new tasks by utilizing large amounts of prior data. Difference is in whether the base data is organic or digital. Using both to find useful predicates seems most promising so far.