> We have absolutely no idea what consciousness is or how it works. We know what it is, we just don't know how it works. Self-awareness comes in stages. We ge

> We have absolutely no idea what consciousness is or how it works. We know what it is, we just don't know how it works. Self-awareness comes in stages. We generally move through those stages throughout our daily lives. Animals display wide variations in qualities of self-awareness as well. So we can define consciousness as the sum of all of these qualities. What I think is going to happen is that we'll start separating out lots of aspects of consciousness and explore them in software, adding more and more of them in as the state of the art of hardware gets better. The consciousness algorithms will slowly, over time, shed complexity. Eventually, well before we get to even human-level self-awareness, we'll run into hard physical limits and realize that biology is way more effective at making the sort of compact, yet incredibly complex evolved system than any design process could be. Biology has an advantage we don't have, it does not have to understand what it is doing, and it works ceaselessly. It can simply try over and over again over millions of years. I predict that getting machines to become truly self-aware will be more trouble than it's worth, and that then you'll be choosing levels of comparatively-lower self-awareness for each individual component of a software system as part of architecting it. In fact, we have that tradeoff now. Do I really need to pull out ML to write a shell script?