4 ms·

The brain is the result of maximizing biological objective functions. Since that led to something that fabricates a lot of things very often, saying the object

The brain is the result of maximizing biological objective functions. Since that led to something that fabricates a lot of things very often, saying the objective function of an LLM is "wrong" because it also fabricates is nonsensical.

jacquesm 3y ago
> The brain is the result of maximizing biological objective functions. That's not how evolution works at all.
- famouswaffles 3y ago
  a mutation happens and if that mutation succeeds in ensuring survival, it stays and then spreads. Reproduce is a function evolution maximizes for. Not intentionally sure but that's irrelevant. The whole point of artificial neural networks is that they teach themselves. They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. There's no intentionality in the shifting numbers either.
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
  - jacquesm 3y ago
    Evolution is not a mechanism that maximizes it is a set of interrelated elements that operate at entirely different levels (molecular, cellular, individual and species) to pass on those traits from one generation to another that result in (possible) change of future individuals within a population, which may affect the survival of those individuals increasing or decreasing the chances of passing those traits on to their (hopefully viable!) offspring. It does nothing to ensure survival of any particular individual, at best it may help a trait that has a function in survival to be passed on. Mutations don't 'succeed' they get passed on or they don't, usually without an immediate effect on the individual, especially not in the individual where the mutation first takes place. But over a longer period mutations may result in a statistical advantage against some environmental factor (including predators) resulting in an increased chance of that particular mutation becoming more widespread. The result is possibly that that mutation ends up being carried by all individuals of the species but that takes a relatively long time and rarely results in 100% coverage in the first generations if at all. ANN's do not necessarily 'teach themselves', there are supervised ways of using ANNs and there are unsupervised ways of using them. > They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. No, they optimize for the statistical likelihood of getting all of the answers right across the whole training set, and then we check if it worked by verification with held back data. So it is very well possible that a change in the weights negatively affects some outputs but positively affects others, if the change is a 'net positive' we keep it. To make matters more interesting every now and then we throw away all but a fraction of all the weights. In the case of supervised learning we (humans) label the data and are the 'teachers' and in the case of unsupervised learning the ANN does its own labeling of clusters of related data (or, more accurately, clusters of data that seem to be related). There are also intermediary versions where humans do some of the labeling to set up the initial clusters and then the mechanical labeling takes over to rapidly increase the size of the training set taking into account the possibility of getting some of the training data wrong but still coming out ahead across the board. So yes, in ANNs there is a global maximization around a goal, and we set it up that way. Evolution doesn't have a particular goal, it is a result of a number of interrelated factors not an input and not something that was put together with a goal in mind.
    - famouswaffles 3y ago
      Whether the brain is "designed" or not is really irrelevant to the point here. It has nothing to do with whether an objective function is "wrong" or not.
      - jacquesm 3y ago
        But it does have to do with whether there is an objective function or not. And there isn't. Brains are the way they are because they evolved that way, because circumstances at some point favored primates with larger brains. Maybe because it allowed us to cooperate, maybe because it enabled skills such as language or higher order thinking and modeling whatever trait you want to substitute for 'the' advantage that allowed our brains to become so large we are in trouble just from being born. Or maybe it wasn't any of that and it was a whole series of small things too tiny to notice individually but with a large enough cumulative effect. None of it had a goal, none of it fit any particular objective function, it's just random chance and local advantages. If natural selection was the whole story that would make the study of evolution of 30 minute thing. But it is far more complex than that. You'd have to explain co-adaptation between two or more species, horizontal gene transfer, the incredibly wasteful ways in which evolution sometimes works. And none of that fits with the notion of an objective function. It's a Platonic ideal. But evolution is incredibly messy and random, with an almost equal chance of going backwards as it has of going forwards on whatever trait you wish to observe. And with the bulk of the mutations resulting in no effect, a negative effect or sterility or death. The whole notion of 'objective functions' doesn't enter into it, let alone whether or not they are right or wrong. You can retroactively infer one and say that's what evolution is optimizing for but that's confusing cause and effect. It's about as sophisticated a view of evolution as a billiard ball simulation of the universe. It just doesn't work that way, it's way too simple a representation to have a chance of modeling the observed complexity. You can try to collapse it into such a simplistic model if you want to explain evolution to a small child. Like a placeholder for something better when they're more equipped to deal with the added complexity. Like when we say electrons move from plus to minus and how the simple semiconductor model allows you to design functional circuitry. But the underlying physics is a lot more complex than that.

jacquesm 3y ago

> The brain is the result of maximizing biological objective functions. That's not how evolution works at all.

famouswaffles 3y ago
a mutation happens and if that mutation succeeds in ensuring survival, it stays and then spreads. Reproduce is a function evolution maximizes for. Not intentionally sure but that's irrelevant. The whole point of artificial neural networks is that they teach themselves. They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. There's no intentionality in the shifting numbers either.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- jacquesm 3y ago
  Evolution is not a mechanism that maximizes it is a set of interrelated elements that operate at entirely different levels (molecular, cellular, individual and species) to pass on those traits from one generation to another that result in (possible) change of future individuals within a population, which may affect the survival of those individuals increasing or decreasing the chances of passing those traits on to their (hopefully viable!) offspring. It does nothing to ensure survival of any particular individual, at best it may help a trait that has a function in survival to be passed on. Mutations don't 'succeed' they get passed on or they don't, usually without an immediate effect on the individual, especially not in the individual where the mutation first takes place. But over a longer period mutations may result in a statistical advantage against some environmental factor (including predators) resulting in an increased chance of that particular mutation becoming more widespread. The result is possibly that that mutation ends up being carried by all individuals of the species but that takes a relatively long time and rarely results in 100% coverage in the first generations if at all. ANN's do not necessarily 'teach themselves', there are supervised ways of using ANNs and there are unsupervised ways of using them. > They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. No, they optimize for the statistical likelihood of getting all of the answers right across the whole training set, and then we check if it worked by verification with held back data. So it is very well possible that a change in the weights negatively affects some outputs but positively affects others, if the change is a 'net positive' we keep it. To make matters more interesting every now and then we throw away all but a fraction of all the weights. In the case of supervised learning we (humans) label the data and are the 'teachers' and in the case of unsupervised learning the ANN does its own labeling of clusters of related data (or, more accurately, clusters of data that seem to be related). There are also intermediary versions where humans do some of the labeling to set up the initial clusters and then the mechanical labeling takes over to rapidly increase the size of the training set taking into account the possibility of getting some of the training data wrong but still coming out ahead across the board. So yes, in ANNs there is a global maximization around a goal, and we set it up that way. Evolution doesn't have a particular goal, it is a result of a number of interrelated factors not an input and not something that was put together with a goal in mind.
  - famouswaffles 3y ago
    Whether the brain is "designed" or not is really irrelevant to the point here. It has nothing to do with whether an objective function is "wrong" or not.
    - jacquesm 3y ago
      But it does have to do with whether there is an objective function or not. And there isn't. Brains are the way they are because they evolved that way, because circumstances at some point favored primates with larger brains. Maybe because it allowed us to cooperate, maybe because it enabled skills such as language or higher order thinking and modeling whatever trait you want to substitute for 'the' advantage that allowed our brains to become so large we are in trouble just from being born. Or maybe it wasn't any of that and it was a whole series of small things too tiny to notice individually but with a large enough cumulative effect. None of it had a goal, none of it fit any particular objective function, it's just random chance and local advantages. If natural selection was the whole story that would make the study of evolution of 30 minute thing. But it is far more complex than that. You'd have to explain co-adaptation between two or more species, horizontal gene transfer, the incredibly wasteful ways in which evolution sometimes works. And none of that fits with the notion of an objective function. It's a Platonic ideal. But evolution is incredibly messy and random, with an almost equal chance of going backwards as it has of going forwards on whatever trait you wish to observe. And with the bulk of the mutations resulting in no effect, a negative effect or sterility or death. The whole notion of 'objective functions' doesn't enter into it, let alone whether or not they are right or wrong. You can retroactively infer one and say that's what evolution is optimizing for but that's confusing cause and effect. It's about as sophisticated a view of evolution as a billiard ball simulation of the universe. It just doesn't work that way, it's way too simple a representation to have a chance of modeling the observed complexity. You can try to collapse it into such a simplistic model if you want to explain evolution to a small child. Like a placeholder for something better when they're more equipped to deal with the added complexity. Like when we say electrons move from plus to minus and how the simple semiconductor model allows you to design functional circuitry. But the underlying physics is a lot more complex than that.