4 ms·

A really good example of a smart person reasoning from completely made up assumptions to a punchy-sounding but almost-certainly wildly wrong conclusion. Just b

A really good example of a smart person reasoning from completely made up assumptions to a punchy-sounding but almost-certainly wildly wrong conclusion. Just because our DNA can be efficiently encoded doesn't mean that our brain is a tiny proportion of that encoding. Your DNA doesn't change much from when you're born to when you die (random degredation aside) and yet your cognative abilities change beyond all recognition. Why is that? Well maybe there's more to what's in the brain than just what's encoded in your DNA. Secondly, how does he get to the 40Mb number? I don't think we know anywhere near enough to know how much information it would take to encode a brain, but 40Mb seems just made up. For starters, consider the amount of random stuff you can remember from your entire life. Are you saying that all can be encoded in just 40Mb? Seems very unlikely.

vacuumcl 4y ago
He is saying that the "base code" / DNA which makes up the brain might be only 40MB (for example). Not that what eventually emerges from that (such as our memories and learned abilities) can be captured with just 40MB. It's similar to how the Game of Life can be implemented in just a few lines, but very complex behavior can emerge from that. The key is to find a sufficiently simple but general model from which intelligence equal to our own can emerge given sufficient training.
- seanhunter 4y ago
  I understand that, but that is an extremely banal observation if you think about it, because the fact that there is this incredible emergent behavior from a simple starting system is the heart of the mystery here. One of the things that everyone is sort of skipping over is the "sufficient training" part. There is no bootstrap reinforcement learning possible for AGI. You can't alphago this sucker and have it play simulations against itself because the whole idea of generality is that there isn't a simple rule framework in which you could run such a simulation. So training any kind of AGI is a really hard problem.
  - anthonypasq 4y ago
    hes specifically answering the question of why he thinks he has any chance of success doing this independently when there are giant organizations funding this.
  - vacuumcl 4y ago
    There are ways that LLM's can self-improve, such as in this paper: https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610 I would speculate that there are more ways to train on logical consistency of the output, and improve the models further.
- yifanl 4y ago
  That seems... just deeply wrong? How much knowledge is gained from observation after birth as opposed to just being innate in your brain?
  - sn9 4y ago
    He's talking about sentience. He admits that the equivalent of years of "training" would still be needed to take an toddler-level consciousness to something approaching an adult human.
m3kw9 4y ago
The brains structure is derived from dna but it contains learning capabilities that comes after the initial creation. Pretty much like how 5 lines of code can get you image recognition, of course it uses lots of libs, but I can still do that myself with 5 lines. The training is the hard part
- seanhunter 4y ago
  Yes exactly. The only training we know of currently that works is for a biological human being to live a decent proportion of their life. For me his statement is one of those things that is sort of not even wrong, like when people say humans only have two eyes and can drive a car therefore an automated driving system must be possible with only cameras. On the face of it, this seems like it could be true but of course it handwaves away the hardest and most mysterious part of intelligence and focusses instead on the easy bit that we already know reasonably well. It's exactly like if someone said they had a secret formula to be an NBA superstar which is: 1. Be really tall 2. Be really agile 3. Be really good at basketball Like yeah of course but all the hard parts are left out.
- sn9 4y ago
  Is the training the hard part? Is that not what humanity has developed from centuries/millennia of experience with all the approaches to child raising and education? Engineering something trainable is clearly the difficult part given the entire conversation is about whether or not it's even possible.

vacuumcl 4y ago

He is saying that the "base code" / DNA which makes up the brain might be only 40MB (for example). Not that what eventually emerges from that (such as our memories and learned abilities) can be captured with just 40MB. It's similar to how the Game of Life can be implemented in just a few lines, but very complex behavior can emerge from that. The key is to find a sufficiently simple but general model from which intelligence equal to our own can emerge given sufficient training.

seanhunter 4y ago
I understand that, but that is an extremely banal observation if you think about it, because the fact that there is this incredible emergent behavior from a simple starting system is the heart of the mystery here. One of the things that everyone is sort of skipping over is the "sufficient training" part. There is no bootstrap reinforcement learning possible for AGI. You can't alphago this sucker and have it play simulations against itself because the whole idea of generality is that there isn't a simple rule framework in which you could run such a simulation. So training any kind of AGI is a really hard problem.
- anthonypasq 4y ago
  hes specifically answering the question of why he thinks he has any chance of success doing this independently when there are giant organizations funding this.
- vacuumcl 4y ago
  There are ways that LLM's can self-improve, such as in this paper: https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610 I would speculate that there are more ways to train on logical consistency of the output, and improve the models further.
yifanl 4y ago
That seems... just deeply wrong? How much knowledge is gained from observation after birth as opposed to just being innate in your brain?
- sn9 4y ago
  He's talking about sentience. He admits that the equivalent of years of "training" would still be needed to take an toddler-level consciousness to something approaching an adult human.

m3kw9 4y ago

The brains structure is derived from dna but it contains learning capabilities that comes after the initial creation. Pretty much like how 5 lines of code can get you image recognition, of course it uses lots of libs, but I can still do that myself with 5 lines. The training is the hard part

seanhunter 4y ago
Yes exactly. The only training we know of currently that works is for a biological human being to live a decent proportion of their life. For me his statement is one of those things that is sort of not even wrong, like when people say humans only have two eyes and can drive a car therefore an automated driving system must be possible with only cameras. On the face of it, this seems like it could be true but of course it handwaves away the hardest and most mysterious part of intelligence and focusses instead on the easy bit that we already know reasonably well. It's exactly like if someone said they had a secret formula to be an NBA superstar which is: 1. Be really tall 2. Be really agile 3. Be really good at basketball Like yeah of course but all the hard parts are left out.
sn9 4y ago
Is the training the hard part? Is that not what humanity has developed from centuries/millennia of experience with all the approaches to child raising and education? Engineering something trainable is clearly the difficult part given the entire conversation is about whether or not it's even possible.