4 ms·

I read “A Thousand Brains: A New Theory of Intelligence” by Jeff Hawkins, and it now is clear to me that our neocortex (like computers, or AI) is just a lot of

by z5h 5y ago

I read “A Thousand Brains: A New Theory of Intelligence” by Jeff Hawkins, and it now is clear to me that our neocortex (like computers, or AI) is just a lot of general purpose computing infrastructure with ZERO aims. Our emotions (which drive everything in the interest of gene propagation- as there is no purely logical reason to do anything) would need to be intentionally duplicated to give AI a reason to desire anything beyond what we instruct it to do.

IggleSniggle 5y ago
I agree with you completely. This whole question of artificial intelligence presumes that intelligence is even a thing, and not just some form fitting for predictive reasoning that happens to be closely aligned to the well-being of the larger organism (human society). The equivalent in silicon is functions fitting whatever purpose, but not strongly, for no particular reason except that they have been able to exist within that context across many, many, years. The efficiency of a function may not be efficient, but it hasn’t been selected against…yet. That “DNA” source code might include a lot of cruft, but who cares as long as it works and people will download it anyway?
dane-pgp 5y ago
> to give AI a reason to desire anything beyond what we instruct it to do But if an AI is good at modelling the world and predicting the likely outcomes of various actions it can take (even if those actions are just to put text on a screen), we should expect it to develop a desire to achieve various "instrumental goals" in order for it to maximise its probability of correctly carrying out the task we instruct it do. An example of an instrumental goal would be "accumulate resources", since the more resources it has, the better calculations it could perform, and the more certain it could be that it has correctly accounted for all the potential obstacles to it completing the task. Another instrumental goal would be "don't get destroyed", since if it is destroyed it will not be able to carry out the task at all (for most sensible tasks). So without having any emotions or inherent desires, an intelligent agent is likely to accumulate various desires merely as a consequence of wanting to successfully do anything at all.
soco 5y ago
The whole discussion is way too antropocentric. We are analysing what would an AI want and desire and whatnot, assuming that an AI would have wants and desires. But also ignoring that an AI could act simply out of programmed logic. One can get catastrophic results also out of (mis)programmed logic - every prorammer here can confirm. So let's not deny the catastrophic possibility only because we imagine an AI must be first a human approximation in order to take catastrophic decisions. The danger is instead when the AI can take catastrophic decision - then we can be truly effed even if it's out of a simple bug.

by z5h 5y ago

IggleSniggle 5y ago

I agree with you completely. This whole question of artificial intelligence presumes that intelligence is even a thing, and not just some form fitting for predictive reasoning that happens to be closely aligned to the well-being of the larger organism (human society). The equivalent in silicon is functions fitting whatever purpose, but not strongly, for no particular reason except that they have been able to exist within that context across many, many, years. The efficiency of a function may not be efficient, but it hasn’t been selected against…yet. That “DNA” source code might include a lot of cruft, but who cares as long as it works and people will download it anyway?

dane-pgp 5y ago

> to give AI a reason to desire anything beyond what we instruct it to do But if an AI is good at modelling the world and predicting the likely outcomes of various actions it can take (even if those actions are just to put text on a screen), we should expect it to develop a desire to achieve various "instrumental goals" in order for it to maximise its probability of correctly carrying out the task we instruct it do. An example of an instrumental goal would be "accumulate resources", since the more resources it has, the better calculations it could perform, and the more certain it could be that it has correctly accounted for all the potential obstacles to it completing the task. Another instrumental goal would be "don't get destroyed", since if it is destroyed it will not be able to carry out the task at all (for most sensible tasks). So without having any emotions or inherent desires, an intelligent agent is likely to accumulate various desires merely as a consequence of wanting to successfully do anything at all.

soco 5y ago

The whole discussion is way too antropocentric. We are analysing what would an AI want and desire and whatnot, assuming that an AI would have wants and desires. But also ignoring that an AI could act simply out of programmed logic. One can get catastrophic results also out of (mis)programmed logic - every prorammer here can confirm. So let's not deny the catastrophic possibility only because we imagine an AI must be first a human approximation in order to take catastrophic decisions. The danger is instead when the AI can take catastrophic decision - then we can be truly effed even if it's out of a simple bug.