4 ms·

>>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA ba

>>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.) We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data. I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data. A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.

MauranKilom 6y ago
Throw some novel text prompt/task at it and see what happens. If it was just "glorified lookup tables" then the result should be consistently garbage. Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
- abernard1 6y ago
  > Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training. The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters! Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.
  - MauranKilom 6y ago
    > The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters! You know what else is trained by the experiences of thousands of individual (and billions of collective) human lifetimes of data? And several trillions of non-human ones? > Humans aren't born trained with data. That's either very wrong or about to evolve into a no true scotsman regarding what counts as data.
Joeri 6y ago
Humans are exposed to much more input than just the things they read. Think of every thing you've ever seen and how much data that represents. All of that is your training data. Is it more or less than GPT-3?
virtue3 6y ago
https://en.wikipedia.org/wiki/Fixed_action_pattern#:~:text=A%20fixed%20action%20pattern%20is,a%20sign%20stimulus%20or%20releaser https://en.wikipedia.org/wiki/Fixed_action_pattern#:~:text=A.... AKA "why is it so hard to swat a fly?" because they literally have a direct linkage betweeen sensing incoming air pressure and jumping. Thats why fly swatters don't make a lot of air pressure. Why do you yank your hand back when you get burned? It's not a controlled reaction. Where did you learn it? You didn't. If you think the brain is much more than a chemical computer you are sadly mistaken. I would encourage you (not really but it's funny to say) to go experiment with psychedelics and steroids and you will quickly realize that these substances can take over your own perceived intelligence. The most fascinating of all of this is articles/documentaries about trans people that have started taking hormones and how their perception of the world -drastically- changed. From "feeling" a fast car all of a sudden, to being able to visualize flavors. It's absolutely amazing.

MauranKilom 6y ago

Throw some novel text prompt/task at it and see what happens. If it was just "glorified lookup tables" then the result should be consistently garbage. Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.

abernard1 6y ago
> Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training. The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters! Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.
- MauranKilom 6y ago
  > The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters! You know what else is trained by the experiences of thousands of individual (and billions of collective) human lifetimes of data? And several trillions of non-human ones? > Humans aren't born trained with data. That's either very wrong or about to evolve into a no true scotsman regarding what counts as data.

Joeri 6y ago

Humans are exposed to much more input than just the things they read. Think of every thing you've ever seen and how much data that represents. All of that is your training data. Is it more or less than GPT-3?

virtue3 6y ago

https://en.wikipedia.org/wiki/Fixed_action_pattern#:~:text=A%20fixed%20action%20pattern%20is,a%20sign%20stimulus%20or%20releaser https://en.wikipedia.org/wiki/Fixed_action_pattern#:~:text=A.... AKA "why is it so hard to swat a fly?" because they literally have a direct linkage betweeen sensing incoming air pressure and jumping. Thats why fly swatters don't make a lot of air pressure. Why do you yank your hand back when you get burned? It's not a controlled reaction. Where did you learn it? You didn't. If you think the brain is much more than a chemical computer you are sadly mistaken. I would encourage you (not really but it's funny to say) to go experiment with psychedelics and steroids and you will quickly realize that these substances can take over your own perceived intelligence. The most fascinating of all of this is articles/documentaries about trans people that have started taking hormones and how their perception of the world -drastically- changed. From "feeling" a fast car all of a sudden, to being able to visualize flavors. It's absolutely amazing.