3 ms·

This is a well-written and well-reasoned argument - BUT - I tend toward the materialist philosophy, so the argument doesn't really hold there. Yes, an ML model

by nlh 5y ago

This is a well-written and well-reasoned argument - BUT - I tend toward the materialist philosophy, so the argument doesn't really hold there. Yes, an ML model that infers B from A might not "understand" what A or B are....yet. But what is it to "understand" anyway? Just a more complex process in a different part of the machine. If the human brain is just a REALLY large, trained, NN, there's no reason that we won't be able to replicate it given enough computing power.

jaredklewis 5y ago
> If the human brain is just a REALLY large, trained, NN, there's no reason that we won't be able to replicate it given enough computing power. I think one clear sign that the human mind is more than just a big NN is how large neural networks are already. Take GPT-3, which is was trained on 45 terabytes of text and has 175 billion parameters. Contrast that with the human brain, which has around 86 billion neurons and is able to do much of what GPT-3 can do with only a tiny fraction of the training data. And it has to be said that while GPT-3 has more competency than an average human at some text generation related tasks, the average human brain is vastly more capable than GPT-3 at any non-text related task. So for neural networks to approach human level capability we would need a whole stack of GPT3-ish size networks for all the other non-text related things the human brain can do: speech, vision, motor control, social interactions, and so on. By that point the amount of training data and parameters is so astronomical, there can be no question that the functioning of human brains must be significantly different than that of contemporary computer neural networks. To be clear, I am also a materialist and subscribe to the computational theory of mind, but just based on the size of training data alone, it seems obvious that human brains work differently than neural networks.
- Simon321 5y ago
  A parameter in a neural network is more comparable to a synapse of which the brain has 100 trillion. And yes, we will get there too one day.
  - jaredklewis 5y ago
    That's a good point, but amount of training data these neural networks take doesn't seem compatible to me. If I read all day everyday from the moment I was born until now, I couldn't have read 45 terabytes of text.
    - exporectomy 5y ago
      Don't forget the training that you did before you were born through evolution. It wasn't text but was a bunch of transferrable skills that help us understand text.
      - jaredklewis 5y ago
        Yes, but what we understand about how the process of natural selection shaped the human brain looks even less like a neural network than what we understand about brains. I guess if we want to reduce the comparison to something vague like "the human brain and neural networks both developed over many iterations," I could agree with that, but that doesn't seem very interesting. If the two were actually comparable, it would look more like: 1. Use NN (or GP) to develop set of hyper parameters 2. Give said HP to neural nets that, thanks to the HP, can be trained on a very limited said of data. I am not aware of any successes with methods like these.

by nlh 5y ago

jaredklewis 5y ago

> If the human brain is just a REALLY large, trained, NN, there's no reason that we won't be able to replicate it given enough computing power. I think one clear sign that the human mind is more than just a big NN is how large neural networks are already. Take GPT-3, which is was trained on 45 terabytes of text and has 175 billion parameters. Contrast that with the human brain, which has around 86 billion neurons and is able to do much of what GPT-3 can do with only a tiny fraction of the training data. And it has to be said that while GPT-3 has more competency than an average human at some text generation related tasks, the average human brain is vastly more capable than GPT-3 at any non-text related task. So for neural networks to approach human level capability we would need a whole stack of GPT3-ish size networks for all the other non-text related things the human brain can do: speech, vision, motor control, social interactions, and so on. By that point the amount of training data and parameters is so astronomical, there can be no question that the functioning of human brains must be significantly different than that of contemporary computer neural networks. To be clear, I am also a materialist and subscribe to the computational theory of mind, but just based on the size of training data alone, it seems obvious that human brains work differently than neural networks.

Simon321 5y ago
A parameter in a neural network is more comparable to a synapse of which the brain has 100 trillion. And yes, we will get there too one day.
- jaredklewis 5y ago
  That's a good point, but amount of training data these neural networks take doesn't seem compatible to me. If I read all day everyday from the moment I was born until now, I couldn't have read 45 terabytes of text.
  - exporectomy 5y ago
    Don't forget the training that you did before you were born through evolution. It wasn't text but was a bunch of transferrable skills that help us understand text.
    - jaredklewis 5y ago
      Yes, but what we understand about how the process of natural selection shaped the human brain looks even less like a neural network than what we understand about brains. I guess if we want to reduce the comparison to something vague like "the human brain and neural networks both developed over many iterations," I could agree with that, but that doesn't seem very interesting. If the two were actually comparable, it would look more like: 1. Use NN (or GP) to develop set of hyper parameters 2. Give said HP to neural nets that, thanks to the HP, can be trained on a very limited said of data. I am not aware of any successes with methods like these.