6 ms·

The main objective function in nature is very simple though: maximize the number of copies of your genes. Such an objective enforced in a resource-constrained m

The main objective function in nature is very simple though: maximize the number of copies of your genes. Such an objective enforced in a resource-constrained multi-agent world (as suggested in the slides) could really lead to quite complex sub-objectives which may as well lead to general intelligence. For example, if each agent can process information and perform work, it follows that individuals that have better cooperative abilities also have a larger reproductive success. Cooperation is, however, extremely complex: It requires communication, identification of other individuals, establishment of trust, early detection of betrayal etc. The necessity for modeling the actions of other agents alone provides plenty of correction signals toward general intelligence because modelling other agents is such a difficult task.

CuriouslyC 9y ago
I don't know that "maximize descendants of self" is necessarily right though. It seems like a better statement might be "maximize the development of complexity in the universe". Just as an example, lots of people choose to forgo having children to focus on contributing to the universe in other ways (myself included). This isn't just a self-centered drive for fame/wealth/etc either, as many people pursue their quests in poverty/obscurity, and some even choose anonymity.
- deleted 9y ago
  [deleted]
- mirimir 9y ago
  Well, for biological organisms, it's all about reproductive success. I mean, what exits today reflects what managed to reproduce, and how well. Overall, that has created lots of complexity. But that's just because there are so many niches and ways to be successful in them. What you say about people reflects cooperative behavior that drives reproductive success for shared gene complexes.
  - TheOtherHobbes 9y ago
    It took nature four billion years to invent humans, who are actually - if we're honest - pretty terrible as an example of workable AGI. In fact what nature invented was a persistent colony organism with external memory. Wild solo humans are only a little smarter than wolves individually, but being able to share and externalise invention and learning created a massive advantage. Humans are successful because although only a tiny minority of individuals are any good at invention, the fact that information persists and is shared means the entire population benefits. The problem for AI is modelling the learning and invention process. Classifiers and recognisers are getting better, but they're not really learning in the human sense, which is a combination of abstraction, mimicry, and occasional invention. IMO there's no chance of AGI developing until there's a persistent, transferable, abstracted model generated as an output from classifier systems and other learning machines which is a symbolic - not just a statistical - summary of the learning.
    - red75prime 9y ago
      > until there's a persistent, transferable, abstracted model [...] It can be compressed to "until AIs can talk".
      - mirimir 9y ago
        ... to other AIs.
        red75prime 9y ago
        or "... to itself"
    - visarga 9y ago
      Transfer learning is a thing (one NN learns from another or multiple NNs), also, large ontologies representing billions of facts.
  - visarga 9y ago
    > for biological organisms, it's all about reproductive success Another way of putting it - the source of meaning is life, or death (prolonging life, avoiding death as much as possible). Reproduction is just the start of life. From this game of life and death come reward signals that teach us how to act in the world (our values).
- mannykannot 9y ago
  There is no guarantee that pursuing complexity as a goal in itself will lead to intelligence. The one thing going for mannigfaltig's proposal is that it has been known to work, though very inefficiently, and we don't have enough examples to estimate the yield. One might suggest that having the right definition of complexity would produce the desired result, but coming up with that definition takes us right back to CuriouslyC's point.
  - visarga 9y ago
    I give you a simple definition: maximize diversity (differentiation) and integration. They are opposites, to a degree - so there is a tradeoff and maximum at the middle in both diversity and integration. This idea comes from the Integrated Information Theory of Consciousness (Giulio Tononi, Christoph Koch). In a neural net we do just that - maximize diversity by splitting the signal over many neurons, each with different weights, computing different things. Integration is maximized by the mixing together of signals from other neurons and training them together with a common loss function. Even the internet as a medium requires diversity and integration to be successful. For example, net neutrality is related to diversity. Integration is related to national firewalls, copyright barriers, filter bubbling effect (where one sees only content from parts of the internet they agree with), walled gardens (like the app stores), and other things that cut the connection between people. You can apply diversity and integration to other fields as well, for example, in politics/governance. We can compare a federal system (more diversity) with a centrally planned system (less diversity) and see the effects. With integration - we can compare free trade with regulated trade. The same principles apply to free speech - where diversity and integration are basically promised by the constitution.
    - mannykannot 9y ago
      Well, maybe, but it looks very speculative to me. I think anything deserving the label 'definition' would have to be much more definite than that.
mirimir 9y ago
I don't think that we'd want that sort of AI, as a competitor. But if we could become one, that would be cool.
simonh 9y ago
There's no guarantee this will favour the development of intelligence though. To take examples from nature, flighted birds are so optimised for weight that they would never develop large, heavy enough craniums to support sentient brains. Many selective criteria may turn out to favour optimisation patterns that could actualy exclude general intelligence, or at least might effectively close off evolutionary paths that could lead to it.
- kpil 9y ago
  Except that ravens and other corvids seems to be self aware and are highly intelligent, it's correct that evolving things does not guarantee high intelligence. Insects and especially social insects like ants are good examples of very successful survivors with very little general intelligence.
  - BooglyWoo 9y ago
    As another user TheOtherHobbes noted, humans share this characteristic with insects to some extent: > Wild solo humans are only a little smarter than wolves individually, but being able to share and externalise invention and learning created a massive advantage. This seems to resonate with a strain of Philosophy of Mind https://plato.stanford.edu/entries/content-externalism/ https://plato.stanford.edu/entries/content-externalism/ which deals with our mental content being distributed not only around the brain and body, but on paper, computers and relations with other people.
- mannigfaltig 9y ago
  On the other hand, one can possibly steer the evolution towards a direction that enables general intelligence to evolve. In nature there are many local minima due to biological constraints (body weight, cranium size, birth channel size, predators, payoff between energy investment into large neural networks vs large muscles, ecological niches). In a simulation one can probably avoid many of these local minima by changing the rules that govern the simulation.

CuriouslyC 9y ago

I don't know that "maximize descendants of self" is necessarily right though. It seems like a better statement might be "maximize the development of complexity in the universe". Just as an example, lots of people choose to forgo having children to focus on contributing to the universe in other ways (myself included). This isn't just a self-centered drive for fame/wealth/etc either, as many people pursue their quests in poverty/obscurity, and some even choose anonymity.

deleted 9y ago
[deleted]
mirimir 9y ago
Well, for biological organisms, it's all about reproductive success. I mean, what exits today reflects what managed to reproduce, and how well. Overall, that has created lots of complexity. But that's just because there are so many niches and ways to be successful in them. What you say about people reflects cooperative behavior that drives reproductive success for shared gene complexes.
- TheOtherHobbes 9y ago
  It took nature four billion years to invent humans, who are actually - if we're honest - pretty terrible as an example of workable AGI. In fact what nature invented was a persistent colony organism with external memory. Wild solo humans are only a little smarter than wolves individually, but being able to share and externalise invention and learning created a massive advantage. Humans are successful because although only a tiny minority of individuals are any good at invention, the fact that information persists and is shared means the entire population benefits. The problem for AI is modelling the learning and invention process. Classifiers and recognisers are getting better, but they're not really learning in the human sense, which is a combination of abstraction, mimicry, and occasional invention. IMO there's no chance of AGI developing until there's a persistent, transferable, abstracted model generated as an output from classifier systems and other learning machines which is a symbolic - not just a statistical - summary of the learning.
  - red75prime 9y ago
    > until there's a persistent, transferable, abstracted model [...] It can be compressed to "until AIs can talk".
    - mirimir 9y ago
      ... to other AIs.
      - red75prime 9y ago
        or "... to itself"
  - visarga 9y ago
    Transfer learning is a thing (one NN learns from another or multiple NNs), also, large ontologies representing billions of facts.
- visarga 9y ago
  > for biological organisms, it's all about reproductive success Another way of putting it - the source of meaning is life, or death (prolonging life, avoiding death as much as possible). Reproduction is just the start of life. From this game of life and death come reward signals that teach us how to act in the world (our values).
mannykannot 9y ago
There is no guarantee that pursuing complexity as a goal in itself will lead to intelligence. The one thing going for mannigfaltig's proposal is that it has been known to work, though very inefficiently, and we don't have enough examples to estimate the yield. One might suggest that having the right definition of complexity would produce the desired result, but coming up with that definition takes us right back to CuriouslyC's point.
- visarga 9y ago
  I give you a simple definition: maximize diversity (differentiation) and integration. They are opposites, to a degree - so there is a tradeoff and maximum at the middle in both diversity and integration. This idea comes from the Integrated Information Theory of Consciousness (Giulio Tononi, Christoph Koch). In a neural net we do just that - maximize diversity by splitting the signal over many neurons, each with different weights, computing different things. Integration is maximized by the mixing together of signals from other neurons and training them together with a common loss function. Even the internet as a medium requires diversity and integration to be successful. For example, net neutrality is related to diversity. Integration is related to national firewalls, copyright barriers, filter bubbling effect (where one sees only content from parts of the internet they agree with), walled gardens (like the app stores), and other things that cut the connection between people. You can apply diversity and integration to other fields as well, for example, in politics/governance. We can compare a federal system (more diversity) with a centrally planned system (less diversity) and see the effects. With integration - we can compare free trade with regulated trade. The same principles apply to free speech - where diversity and integration are basically promised by the constitution.
  - mannykannot 9y ago
    Well, maybe, but it looks very speculative to me. I think anything deserving the label 'definition' would have to be much more definite than that.

mirimir 9y ago

I don't think that we'd want that sort of AI, as a competitor. But if we could become one, that would be cool.

simonh 9y ago

There's no guarantee this will favour the development of intelligence though. To take examples from nature, flighted birds are so optimised for weight that they would never develop large, heavy enough craniums to support sentient brains. Many selective criteria may turn out to favour optimisation patterns that could actualy exclude general intelligence, or at least might effectively close off evolutionary paths that could lead to it.

kpil 9y ago
Except that ravens and other corvids seems to be self aware and are highly intelligent, it's correct that evolving things does not guarantee high intelligence. Insects and especially social insects like ants are good examples of very successful survivors with very little general intelligence.
- BooglyWoo 9y ago
  As another user TheOtherHobbes noted, humans share this characteristic with insects to some extent: > Wild solo humans are only a little smarter than wolves individually, but being able to share and externalise invention and learning created a massive advantage. This seems to resonate with a strain of Philosophy of Mind https://plato.stanford.edu/entries/content-externalism/ https://plato.stanford.edu/entries/content-externalism/ which deals with our mental content being distributed not only around the brain and body, but on paper, computers and relations with other people.
mannigfaltig 9y ago
On the other hand, one can possibly steer the evolution towards a direction that enables general intelligence to evolve. In nature there are many local minima due to biological constraints (body weight, cranium size, birth channel size, predators, payoff between energy investment into large neural networks vs large muscles, ecological niches). In a simulation one can probably avoid many of these local minima by changing the rules that govern the simulation.