4 ms·

Go back and look at the history of AI, including current papers from the most advanced research teams. Nearly every component is based on humans - neural net

Go back and look at the history of AI, including current papers from the most advanced research teams. Nearly every component is based on humans - neural net - long/short term memory - attention - reasoning - activation function - learning - hallucination - evolutionary algorithm If you're just consuming an AI to build a React app then you don't have to care. If you are building an artificial intelligence then in practice everyone who's anyone is very deliberately modeling it on humans.

root_axis 1y ago
You're anthropomorphizing terms of art.
orbital-decay 1y ago
How far back do I have to look, and what definition do you use? Because I can start with theorem provers and chess engines of the 1950s. Nothing in that list is based on humans, even remotely. Only neural networks were a vague form of biomimicry early on and currently have academic biomimicry approaches, of which all suck because they poorly map to available semiconductor manufacturing processes. Attention is misleadingly called that, reasoning is ill-defined, etc. LLMs are trained on human-produced data, and ML in general shares many fundamentals and emergent phenomena with biological learning (a lot more than some people talking about "token predictors" realize). That's it. Producing artificial humans or imitating real ones was never the goal nor the point. We can split hairs all day long, but the point of AI as a field since 1950s is to produce systems that do something that is considered only doable by humans.
- ants_everywhere 1y ago
  > How far back do I have to look The earliest reference I know off the top of my head is Aristotle, which would be the 4th century BCE > I can start with theorem provers If you're going to talk about theorem provers, you may want to include the medieval theory of obligations and their game-semantic-like nature. Or the Socratic notion of a dialogue in which arguments are arrived at via a back and forth. Or you may want to consider that "logos" from which we get logic means "word". And if you contemplate these things for a minute or two you'll realize that logic since ancient times has been a model of speech and often specifically of speaking with another human. It's a way of having words (and later written symbols) constrain thought to increase the signal to noise ratio. Chess is another kind of game played between two people. In this case it's a war game, but that seems not so essential. The essential thing is that chess is a game and games are relatively constrained forms of reasoning. They're modeling a human activity. By 1950, Alan Turing had already written about the imitation game (or Turing test) that evaluated whether a computer could be said to be thinking based on its ability to hold a natural language conversation with humans. He also built an early chess system and was explicitly thinking about artificial intelligence as a model of what humans could do. > Attention is misleadingly called that, reasoning is ill-defined, None of this dismissiveness bears on the point. If you want to argue that humans are not the benchmark and model of intelligence (which frankly I think is a completely indefensible position, but that's up to you) then you have to argue that these things were not named or modeled after human activities. It's not sufficient that you think their names are poorly chosen. > Producing artificial humans or imitating real ones was never the goal nor the point. Artificial humans is exactly the concept of androids or humanoid robots. You are claiming that nobody has ever wanted to make humanoid robots? I'm sure you can't believe that but I'm at a loss for what point you're trying to make. > 1950s is to produce systems that do something that is considered only doable by humans. Unless this is a typo and you meant to write that this was NOT the goal, then you're conceding my point that humans are the benchmark and model for AI systems. They are, after all, the most intelligent beings we know to exist at present. And so to reiterate my original point, talking about AI with the constraint that you can't compare them to humans is totally insane.
  - portaouflop 1y ago
    You can compare them to humans but it’s kind of boring. Maybe more interesting if you are an “ai” researcher
janalsncm 1y ago
Those terms sound similar to biological concepts but they’re very different. Neural networks are not like brains. They don’t grow new neurons. A “neuron” in an artificial neural net is represented with a single floating point number. Sometimes even quantized down to a 4 bit int. Their degrees of freedom are highly limited compared to a brain. Most importantly, the brain does not do back propagation like an ANN does. LSTMs have about as much to do with brain memory as RAM does. Attention is a specific mathematical operation applied to matrices. Activation functions are interesting because originally they were more biologically inspired and people used sigmoid. Now people tend to use simpler ones like ReLU or its leaky cousin. Turns out what’s important is creating nonlinearities. Hallucinations in LLMs have to do with the fact that they’re statistical models not grounded in reality. Evolutionary algorithms, I will give you that one although they’re way less common than backprop.
- akoboldfrying 1y ago
  Neural networks are a lot like brains. That they don't generally grow new neurons is something that (a) could be changed with a few lines of code and (b) seems like an insignificant detail anyway. > the brain does not do back propagation Do we know this? Ruling this out is tantamount to claiming that we know how brains do learn. My suspicion is that we don't currently know, and that it will turn out that, e.g., sleep does something that is a coarse approximation of backprop.
  - wizzwizz4 1y ago
    No, we're pretty sure brains don't do backprop. See e.g. https://doi.org/10.1038/s41598-018-35221-w https://doi.org/10.1038/s41598-018-35221-w
    - akoboldfrying 1y ago
      Do we know that backprop is disjoint from variational free energy minimisation? Or could it be that one is an approximation to or special case of the other? I Ctrl-F'd "backprop" and found nothing, so I think they aren't compared in the paper, but maybe this is common knowledge in the field.
Sharlin 1y ago
What your examples show is that humans like to repurpose existing words to refer to new things based on generalizations or vague analogies. Not much more than that.
littlestymaar 1y ago
Just because something is named after the name of a biological concept doesn't mean it has anything to do with the original thing the name was taken from.
- oblio 1y ago
  Whoa, hold it right there! Next you'll tell me that Windows Hibernate and Bear® Hibernate™ have nothing in common?
- ants_everywhere 1y ago
  Name collisions are possible, but in these cases the terms are explicitly modeled on the biological concepts.
  - littlestymaar 1y ago
    It's not name “collision”, they took a biological name that somehow felt apt for what they where doing. To continue oblios's analogy, when you use the “hibernation mode” of your OS, it only has superficial similarity with how manals hibernate during winter…
    - ants_everywhere 1y ago
      [flagged]
      - littlestymaar 1y ago
        Well, your statements show that you are much less informed than you believe.
        deleted 1y ago
        [deleted]
        ants_everywhere 1y ago
        As I've said, go look at the literature. If you find in the literature any evidence for your theory that the people who invented these concepts believed they had no relationship to the biological concepts then please contribute it. If you're unwilling to do the bare minimum required to take the opposite of my position then you aren't really defending a point of view you're just gainsaying reflexively.

root_axis 1y ago

You're anthropomorphizing terms of art.

orbital-decay 1y ago

How far back do I have to look, and what definition do you use? Because I can start with theorem provers and chess engines of the 1950s. Nothing in that list is based on humans, even remotely. Only neural networks were a vague form of biomimicry early on and currently have academic biomimicry approaches, of which all suck because they poorly map to available semiconductor manufacturing processes. Attention is misleadingly called that, reasoning is ill-defined, etc. LLMs are trained on human-produced data, and ML in general shares many fundamentals and emergent phenomena with biological learning (a lot more than some people talking about "token predictors" realize). That's it. Producing artificial humans or imitating real ones was never the goal nor the point. We can split hairs all day long, but the point of AI as a field since 1950s is to produce systems that do something that is considered only doable by humans.

ants_everywhere 1y ago
> How far back do I have to look The earliest reference I know off the top of my head is Aristotle, which would be the 4th century BCE > I can start with theorem provers If you're going to talk about theorem provers, you may want to include the medieval theory of obligations and their game-semantic-like nature. Or the Socratic notion of a dialogue in which arguments are arrived at via a back and forth. Or you may want to consider that "logos" from which we get logic means "word". And if you contemplate these things for a minute or two you'll realize that logic since ancient times has been a model of speech and often specifically of speaking with another human. It's a way of having words (and later written symbols) constrain thought to increase the signal to noise ratio. Chess is another kind of game played between two people. In this case it's a war game, but that seems not so essential. The essential thing is that chess is a game and games are relatively constrained forms of reasoning. They're modeling a human activity. By 1950, Alan Turing had already written about the imitation game (or Turing test) that evaluated whether a computer could be said to be thinking based on its ability to hold a natural language conversation with humans. He also built an early chess system and was explicitly thinking about artificial intelligence as a model of what humans could do. > Attention is misleadingly called that, reasoning is ill-defined, None of this dismissiveness bears on the point. If you want to argue that humans are not the benchmark and model of intelligence (which frankly I think is a completely indefensible position, but that's up to you) then you have to argue that these things were not named or modeled after human activities. It's not sufficient that you think their names are poorly chosen. > Producing artificial humans or imitating real ones was never the goal nor the point. Artificial humans is exactly the concept of androids or humanoid robots. You are claiming that nobody has ever wanted to make humanoid robots? I'm sure you can't believe that but I'm at a loss for what point you're trying to make. > 1950s is to produce systems that do something that is considered only doable by humans. Unless this is a typo and you meant to write that this was NOT the goal, then you're conceding my point that humans are the benchmark and model for AI systems. They are, after all, the most intelligent beings we know to exist at present. And so to reiterate my original point, talking about AI with the constraint that you can't compare them to humans is totally insane.
- portaouflop 1y ago
  You can compare them to humans but it’s kind of boring. Maybe more interesting if you are an “ai” researcher

janalsncm 1y ago

Those terms sound similar to biological concepts but they’re very different. Neural networks are not like brains. They don’t grow new neurons. A “neuron” in an artificial neural net is represented with a single floating point number. Sometimes even quantized down to a 4 bit int. Their degrees of freedom are highly limited compared to a brain. Most importantly, the brain does not do back propagation like an ANN does. LSTMs have about as much to do with brain memory as RAM does. Attention is a specific mathematical operation applied to matrices. Activation functions are interesting because originally they were more biologically inspired and people used sigmoid. Now people tend to use simpler ones like ReLU or its leaky cousin. Turns out what’s important is creating nonlinearities. Hallucinations in LLMs have to do with the fact that they’re statistical models not grounded in reality. Evolutionary algorithms, I will give you that one although they’re way less common than backprop.

akoboldfrying 1y ago
Neural networks are a lot like brains. That they don't generally grow new neurons is something that (a) could be changed with a few lines of code and (b) seems like an insignificant detail anyway. > the brain does not do back propagation Do we know this? Ruling this out is tantamount to claiming that we know how brains do learn. My suspicion is that we don't currently know, and that it will turn out that, e.g., sleep does something that is a coarse approximation of backprop.
- wizzwizz4 1y ago
  No, we're pretty sure brains don't do backprop. See e.g. https://doi.org/10.1038/s41598-018-35221-w https://doi.org/10.1038/s41598-018-35221-w
  - akoboldfrying 1y ago
    Do we know that backprop is disjoint from variational free energy minimisation? Or could it be that one is an approximation to or special case of the other? I Ctrl-F'd "backprop" and found nothing, so I think they aren't compared in the paper, but maybe this is common knowledge in the field.

Sharlin 1y ago

What your examples show is that humans like to repurpose existing words to refer to new things based on generalizations or vague analogies. Not much more than that.

littlestymaar 1y ago

Just because something is named after the name of a biological concept doesn't mean it has anything to do with the original thing the name was taken from.

oblio 1y ago
Whoa, hold it right there! Next you'll tell me that Windows Hibernate and Bear® Hibernate™ have nothing in common?
ants_everywhere 1y ago
Name collisions are possible, but in these cases the terms are explicitly modeled on the biological concepts.
- littlestymaar 1y ago
  It's not name “collision”, they took a biological name that somehow felt apt for what they where doing. To continue oblios's analogy, when you use the “hibernation mode” of your OS, it only has superficial similarity with how manals hibernate during winter…
  - ants_everywhere 1y ago
    [flagged]
    - littlestymaar 1y ago
      Well, your statements show that you are much less informed than you believe.
      - deleted 1y ago
        [deleted]
      - ants_everywhere 1y ago
        As I've said, go look at the literature. If you find in the literature any evidence for your theory that the people who invented these concepts believed they had no relationship to the biological concepts then please contribute it. If you're unwilling to do the bare minimum required to take the opposite of my position then you aren't really defending a point of view you're just gainsaying reflexively.