4 ms·

> The line defining “code” and “data” is thus heavily blurred and any argument about how humans are far superior because of the “reduced number of training exam

> The line defining “code” and “data” is thus heavily blurred and any argument about how humans are far superior because of the “reduced number of training examples” is definitely missing the millennia of evolution that created us in the first place. I disagree. The line is quite clear. Our factual memories do not persist from one generation to another. Yet this is what modern ML does. The "data" encoded in DNA is not about knowledge or facts, it is knowledge about our architecture. Modern ML is a like a factory that outputs widgets based upon knowledge of lots of pre-existing widgets. DNA is a like a factory that outputs widgets based upon lots of previous pre-existing factories. The factory is the "code" or "verb." The widgets are the "data" or "nouns." Completely separable and objectively so.

imoverclocked 2y ago
If you are talking about facts like "cos(0) = 1" then yes, of course I agree; Those kinds of facts do not persist simply by giving birth. However, that's a very narrow view of "data" or "knowledge" when talking about biological systems. Humans use culture and collaboration to pass that kind of knowledge on. Spiders don't have culture and collaboration in the same sense. We are wired/evolved with the ability to form communities which is a different kind of knowledge altogether. It seems like you are simultaneously arguing that humans (who have a far more complex network, or set of networks than current LLMs) can recognize a class of something given a single or few specific examples of the thing while also arguing that the structure has nothing to do with the success of that. The structure was created over many generations going all the way back to the first single-celled organisms over 3.7 billion years ago. The more successful networks for what eventually became humans were able to survive based on the traits that we largely have today. Those traits were useful back then for understanding that one cat might act like another cat without needing to know all cats. There are things our brains can just barely do (eg: higher level mathematics) that a slightly different structure might enable... and may have existed in the past only to be wiped out by someone who could think a little faster. Also, check out epigenetics. DNA is expressed differently based on environmental factors of previous and current generations. The "factories" you speak of aren't so mechanical as you would make them seem. All of this is to say, Human biology is wonderfully complicated. Comparing LLMs to humanity is going to be fraught with issues because they are two very different things. Human intelligence is a combination of our form, our resources and our passed on knowledge. So far, LLMs are simply another representation of our passed on knowledge.

imoverclocked 2y ago

If you are talking about facts like "cos(0) = 1" then yes, of course I agree; Those kinds of facts do not persist simply by giving birth. However, that's a very narrow view of "data" or "knowledge" when talking about biological systems. Humans use culture and collaboration to pass that kind of knowledge on. Spiders don't have culture and collaboration in the same sense. We are wired/evolved with the ability to form communities which is a different kind of knowledge altogether. It seems like you are simultaneously arguing that humans (who have a far more complex network, or set of networks than current LLMs) can recognize a class of something given a single or few specific examples of the thing while also arguing that the structure has nothing to do with the success of that. The structure was created over many generations going all the way back to the first single-celled organisms over 3.7 billion years ago. The more successful networks for what eventually became humans were able to survive based on the traits that we largely have today. Those traits were useful back then for understanding that one cat might act like another cat without needing to know all cats. There are things our brains can just barely do (eg: higher level mathematics) that a slightly different structure might enable... and may have existed in the past only to be wiped out by someone who could think a little faster. Also, check out epigenetics. DNA is expressed differently based on environmental factors of previous and current generations. The "factories" you speak of aren't so mechanical as you would make them seem. All of this is to say, Human biology is wonderfully complicated. Comparing LLMs to humanity is going to be fraught with issues because they are two very different things. Human intelligence is a combination of our form, our resources and our passed on knowledge. So far, LLMs are simply another representation of our passed on knowledge.