5 ms·

IMHO, there could be a tipping point of scaling "primate" brain. For a given task, when size of brain can handle so much knowledge that unknowns are fewer than

by est 4y ago

IMHO, there could be a tipping point of scaling "primate" brain. For a given task, when size of brain can handle so much knowledge that unknowns are fewer than knowns, the advantage grows exponentially. Because you can deduce outcomes correctly by eliminating the wrong answers without actually "understands" the problem itself.

extasia 4y ago
Do you have any literature on this?
- est 4y ago
  Just my own thoughts. Add `IMHO` prefix.
ivxvm 4y ago
Idk. I think the real tipping point is being able to succesfully use external memory so that you don't have to keep everything in your working memory when doing reasoning or solving problems.
- YetAnotherNick 4y ago
  Yes, humans have been around for 300,000 years and only in the last 10,000 years human crossed that tipping point to be able to think higher order things.
  - kaba0 4y ago
    Do you have any evidence for this claim? Just because people didn’t start agriculture doesn’t mean they weren’t as smart as people today. Hell, 300000 years is nothing from an evolutionary perspective, they were for all practical purposes genetically identical. It was just most probably a nurture thing — there was not as much accumulated knowledge to “bootstrap” a more advanced civilization (or perhaps, no will to do so. Why change when your system works just fine?). It’s not like Einstein could have achieved much back then without “standing on the shoulder of giants” before him.
- imtringued 4y ago
  That is basically what continuous learning is and so far everything I have seen is that it is discounted as not important or something you could easily work around. There is a difference between waiting for the next minor release which updates the weights based on whatever OpenAI gathered from your chat history and having the model update its weights as you query it.
blackbear_ 4y ago
This is exactly what happened with large language models: they are quite useless until a certain number of parameters, after which their abilities quickly become impressive (read more here: http://arxiv.org/abs/2001.08361 http://arxiv.org/abs/2001.08361)
- kaba0 4y ago
  Except they are fundamentally limited and can’t algorithmically think no matter the scale?
  - blackbear_ 4y ago
    I was merely commenting on the apparent similarity between the phase transition in ability observed in the brain of primates and artificial neural networks, without necessarily equating the capabilities of the two. On that note, maybe in the future we will discover some neural architectures that are able to "algorithmically think", who knows.

by est 4y ago

extasia 4y ago

Do you have any literature on this?

est 4y ago
Just my own thoughts. Add `IMHO` prefix.

ivxvm 4y ago

Idk. I think the real tipping point is being able to succesfully use external memory so that you don't have to keep everything in your working memory when doing reasoning or solving problems.

YetAnotherNick 4y ago
Yes, humans have been around for 300,000 years and only in the last 10,000 years human crossed that tipping point to be able to think higher order things.
- kaba0 4y ago
  Do you have any evidence for this claim? Just because people didn’t start agriculture doesn’t mean they weren’t as smart as people today. Hell, 300000 years is nothing from an evolutionary perspective, they were for all practical purposes genetically identical. It was just most probably a nurture thing — there was not as much accumulated knowledge to “bootstrap” a more advanced civilization (or perhaps, no will to do so. Why change when your system works just fine?). It’s not like Einstein could have achieved much back then without “standing on the shoulder of giants” before him.
imtringued 4y ago
That is basically what continuous learning is and so far everything I have seen is that it is discounted as not important or something you could easily work around. There is a difference between waiting for the next minor release which updates the weights based on whatever OpenAI gathered from your chat history and having the model update its weights as you query it.

blackbear_ 4y ago

This is exactly what happened with large language models: they are quite useless until a certain number of parameters, after which their abilities quickly become impressive (read more here: http://arxiv.org/abs/2001.08361 http://arxiv.org/abs/2001.08361)

kaba0 4y ago
Except they are fundamentally limited and can’t algorithmically think no matter the scale?
- blackbear_ 4y ago
  I was merely commenting on the apparent similarity between the phase transition in ability observed in the brain of primates and artificial neural networks, without necessarily equating the capabilities of the two. On that note, maybe in the future we will discover some neural architectures that are able to "algorithmically think", who knows.