I think they're on to something. Our constraint on abstraction layers that they must be legible and grokkable by human minds, that's just something we impose be

I think they're on to something. Our constraint on abstraction layers that they must be legible and grokkable by human minds, that's just something we impose because we feel that we should understand. But that constraint is overburden. I feel that neural nets are starting to prod at this. Our biology works and does incredibly complex computation (including social computation), and it's creation process didn't give a fuck whether a human could understand it. We're still playing catch-up with that sort of low-power computation that biology does. A sandwich powers a wildly powerful neural network This also reminds me of how alien and organic ML-assisted design is -- when a design task is constrained only by its environmental constraints, and human aesthetic and comforts are discarded: http://jcsitnet.com/journals/jcsit/Vol_7_No_2_December_2019/7.pdf http://jcsitnet.com/journals/jcsit/Vol_7_No_2_December_2019/... https://medium.com/intuitionmachine/the-alien-look-of-deep-learning-generative-design-5c5f871f7d10 https://medium.com/intuitionmachine/the-alien-look-of-deep-l... Straight lines and clean edges and nice angles are just abstractions that we humans happen to have preference for due to our very particular evolutionary history. As in, we happen to like clean straight lines and certain logics in physical and visual domain, but have no such constraints on taste and smell preferences, and in fact love complexity in those sensory domains, because that served us in our past ecological niches. But discarding these visual/logical sensibilities and allowing abstraction interfaces to float free of our preferences, that could open up a ton of possibilities for improvement. Of course, it would become as magical and messy as biology, and perhaps quite challenging to our reasoning. There's good sci-fi about this. About our machines having more comfort with these complexities, and moving into realms of work where we just can't keep up. And we either trust-fall into being ok with what they're building, or we cripple their abilities to match ours. And maybe the society that cripples its AI less, ends up running circles around the ones who do...? EDIT: to be clear, I believe our minds are already doing this: our brain is using "hidden physics" of matter to bring about our conscious experience. We keep finding that ever more niche effects are potential important to the hardware that runs our minds. Because "matter" didn't give a fuck about being legible to its future inhabitant :) We're still teasing apart the interactions, and likely will succeed in regrowing consciousness (with additional legibility) before we actually fully understand how our own is working. Discloser: Am trained as biochemist, so might have a bent on this that doesn't resonate with more hard CS folks.