For one, unlike DL models, we approximate things, and I do not mean that in the universal approximator sense. What I mean is that most of the things we do are e

For one, unlike DL models, we approximate things, and I do not mean that in the universal approximator sense. What I mean is that most of the things we do are essentially pulling from memory similar situations and stitching everything together. Unless you are John Von Neumann, you are either approximately correct or very slow in solving novel tasks. On the other hand, we have the capacity to generalize much easier than our DL models. I think that that relates to our memory. That is, we compress information in a way that allows us to search for the structure, map to the target domain, and then stitch it all together. Hofstadter argues about it a lot in his books, that is, that we use analogies to learn stuff. In the same vein and in a very handwave-y manner, we can think of analogies as functors across categories. That is, as mappings between one domain to another where the relationship (morphism) between the objects is preserved. For whatever reason we are capable of identifying those mappings quickly and applying them everywhere. This relates to symmetries where we have little to no trouble identifying objects that underwent certain transformations but our DL have trouble with. A user in HN suggested that with respect to the visual transformations, we can do that because our optic nerve travels a long distance, from our eyes to our visual cortex, and that allows our brains to perform all kinds of data cleaning and signal processing. The ideas about compression that I mentioned here is essentially what we saw in deepmind's GATO. More precisely, creating multimodal embeddings in that manner seems to enable good multi-task performance. Sorry if this comes across as rambling.