Systems of logic should not be hard coded into AIs (or AGIs) imo. The basic problems with logic is that it doesn't take uncertainty into account, and doesn't su

Systems of logic should not be hard coded into AIs (or AGIs) imo. The basic problems with logic is that it doesn't take uncertainty into account, and doesn't support the fact that almost every statement about the real world is only approximately true, or a useful model; it also usually can't handle inconsistencies very well. It would be unable to prove or in fact have any useful facts about the real world. For example, consider the statement "The sky is blue.". It is not categorically true -- the sky becomes red near sunset, (depending on chrominance vs luminance interpretation) it gets black at night, may be other colors in other planets, etc. So the system would be stuck trying to convey all necessary conditions, might not be able to formulate simple thoughts or communicate well. It would almost certainly lack abstraction capabilities. There are some logical systems that make slight modifications to logic to accommodate probabilities and uncertainty, but those still might be insufficient or incomplete. The fundamental issue is that, while logic works and is useful for us, it is just a tool. The end goal of any real world agent is not logical correctness, it is achieving tasks, understanding/predicting its environment, communicating, etc. Because of the specificity of those environments and tasks, there isn't going to be a general method that is efficient in them. You need a evolutionary-like, or leaning system that can adapt to the problem with minimal underlying assumptions. The assumptions of logic that everything should be binned according to falsehoods, probabilities, etc, is ultimately not necessarily a good way to (for example) control a robot in a complex environment, make it assemble a product, play videogames, or even do mathematics. Perhaps they can be augmented with side models (again tools) allowing them to formulate subtasks as logical problems (although its not clear those approaches would be better than just training neural networks). Nothing can be set in stone, everything must be learned and modifiable. So again the main problem with AGI is formulating good frameworks that 1) Have good, general goals aligned with what we expect from this AI; 2) Is able to adapt in almost every way to achieve this goal, without degrading its ability to recognize its rewards/goals. This is such a hard problem that it occupies a significant chunk of even humans' lifetimes. We spend lots of time thinking about long term goals, which in part are dictated by society, in part dictated by biology, and which we reflect upon to conclude what we should do -- from small daily goals like eating or getting to work, to long term career advancing and relationships. Then we set to achieve those goals by learning and acting on the world around us. It does happen often that our goals degenerate, ultimately because of our ability to modify ourselves and set our own goals: the reward achieved from a drug is a degeneracy that overrides other long term goals; various addictions exploit loopholes in our reward system; we suffer from despair, depression or lack of motivation that can even override the basic biological rewards. If we could not override basic biological rewards, setting our own goals, then we would fall to addictions and inability to live in complex environments which require complex unnatural goals related to careers, learning, and other abstract concepts. And too much of this power can leave us in desperation, depression, meaninglessness. So there is a constant balance and search for genuine rewards, and constant refinement of them. That is central to human existence, and to conscious experience in general.