4 ms·

What absolute piffle. We know nothing of the sort. Imagine the human brain as an engine for compressing the world around it - this is, mathematically true, look

What absolute piffle. We know nothing of the sort. Imagine the human brain as an engine for compressing the world around it - this is, mathematically true, look it up - AIXI. An organism needs the ability to compress the world around it and use that compression to make choices about what to do. That is the sum total of human intelligence. In what significant way is GPT-3 different from this model. At the very most you can argue that GPT-3 is the compression model without the prediction model. You have a GPT inside your brain - just start saying words - the first thing that comes into your head. These are the words of your statistical model of the universe, your internal GPT-3. read what you have written back - it will make sense, because it is not just a parrot, it is your subconscious.

tsimionescu 5y ago
If you take abstraction to the extreme, sure - our brain is the same as GPT-3; but only in the same sense in which our brain is equivalent to any function with discrete output - that is, our brain maps something (the space of all sensory inputs) to something else (the space of mental states). In this same sense, our brain is just like the whole universe, which is just a function from the entire world at time t to the world at time t+x. If we look at anything more specific than 'mathematical function', our brain is nothing like GPT-3. GPT-3 is not trained on sensory data about the world, it is trained on letters of text which have nothing directly to do with the world. The brain has plenty of structure and knowledge that is not learned (except in a very roundabout way, through evolution)*. GPT-3 is lacking any sort of survivalist motivation, which the brain obviously has. The implementation substrate is obviously not even slightly similar. The brain has numerous specialized functions, most of which have nothing to do with language, while GPT-3 has a single kind of functionality and is entirely concerned with language. And even if I start writing down random words, what I'm doing is not in any way similar to GPT-3, and my output won't be either. It will probably be quasi-random non-language (words strung together without grammar) vaguely related to various desires of my subconscious. What it will NOT be is definitely not a plausible sounding block of text that I determine to resemble as closely as possible some sequence of tokens that I observed before, which is what GPT-3 outputs. I do not have an inner GPT-3. The way I use language is by converting some inner thought structure that i have decided to communicate into a langauge I know, via some 1:1 mapping of internal concepts to language structures (words, phrases). In particular, even the basics here are different: letters, the things GPT-3 is trained on, are completely irrelevant to human language use outside of writing. People express themselves in words and phrases, and learn language at that level. Decomposing words into sounds/letters is an artificial, approximate model that we have chosen to use for various reasons, but it is not intrinsic to language, and it doesn't come naturally to any language users (you have to learn the canonical spelling for any word, and even the canonical pronunciation; and there is significant semantic info not captured directly in the letters/sounds of one word, through inflection, or tonality and accent; or in sign languages, there is often no decomposition of words equivalent to letters). * if you don't believe that the brain comes with much knowledge built-in, you'll have to explain how most mammals learn by example how to walk, run, and jump within minutes to hours of birth - what is the data set they are using to learn these extremely complex motor skills, including the perception skills necessary to do so.
- plutonorm 5y ago
  Imagine intelligence like a thermodynamic process. An engine for compressing the world into a smaller representation. Its not about any particular configuration or any particular set of data. Just as the complex structure of the cell arises from the guiding principles of evolution, so intelligence arises from the, as yet, not quite understood processes of thermodynamics within open systems. See England's work in this area. We are constructing systems that play out this universal property of systems to compress. The structures that arise from this driving force of Megawatts of power flowing through graphics cards are only the product of this kind of flow of information. Just as the structure of the human brain is derived from eons of sunlight pouring down upon the photosynthetic cells of plants. There are not two processes at work here. Its one continuous one that rolls on up from the prebiotic soup to hyper advanced space aliens. You are probably stuck on the idea of emergence. You imagine there must be some dividing line between intelligent and non intelligent. Therefore the point at which it emerges must be some spectacular miracle of engineering. When in fact there is no dividing line, just a continuum of consciousness from the small to the large. Read up on panspermia.
  - plutonorm 5y ago
    sorry I mean panpsychism! finger slip

tsimionescu 5y ago

If you take abstraction to the extreme, sure - our brain is the same as GPT-3; but only in the same sense in which our brain is equivalent to any function with discrete output - that is, our brain maps something (the space of all sensory inputs) to something else (the space of mental states). In this same sense, our brain is just like the whole universe, which is just a function from the entire world at time t to the world at time t+x. If we look at anything more specific than 'mathematical function', our brain is nothing like GPT-3. GPT-3 is not trained on sensory data about the world, it is trained on letters of text which have nothing directly to do with the world. The brain has plenty of structure and knowledge that is not learned (except in a very roundabout way, through evolution)*. GPT-3 is lacking any sort of survivalist motivation, which the brain obviously has. The implementation substrate is obviously not even slightly similar. The brain has numerous specialized functions, most of which have nothing to do with language, while GPT-3 has a single kind of functionality and is entirely concerned with language. And even if I start writing down random words, what I'm doing is not in any way similar to GPT-3, and my output won't be either. It will probably be quasi-random non-language (words strung together without grammar) vaguely related to various desires of my subconscious. What it will NOT be is definitely not a plausible sounding block of text that I determine to resemble as closely as possible some sequence of tokens that I observed before, which is what GPT-3 outputs. I do not have an inner GPT-3. The way I use language is by converting some inner thought structure that i have decided to communicate into a langauge I know, via some 1:1 mapping of internal concepts to language structures (words, phrases). In particular, even the basics here are different: letters, the things GPT-3 is trained on, are completely irrelevant to human language use outside of writing. People express themselves in words and phrases, and learn language at that level. Decomposing words into sounds/letters is an artificial, approximate model that we have chosen to use for various reasons, but it is not intrinsic to language, and it doesn't come naturally to any language users (you have to learn the canonical spelling for any word, and even the canonical pronunciation; and there is significant semantic info not captured directly in the letters/sounds of one word, through inflection, or tonality and accent; or in sign languages, there is often no decomposition of words equivalent to letters). * if you don't believe that the brain comes with much knowledge built-in, you'll have to explain how most mammals learn by example how to walk, run, and jump within minutes to hours of birth - what is the data set they are using to learn these extremely complex motor skills, including the perception skills necessary to do so.

plutonorm 5y ago
Imagine intelligence like a thermodynamic process. An engine for compressing the world into a smaller representation. Its not about any particular configuration or any particular set of data. Just as the complex structure of the cell arises from the guiding principles of evolution, so intelligence arises from the, as yet, not quite understood processes of thermodynamics within open systems. See England's work in this area. We are constructing systems that play out this universal property of systems to compress. The structures that arise from this driving force of Megawatts of power flowing through graphics cards are only the product of this kind of flow of information. Just as the structure of the human brain is derived from eons of sunlight pouring down upon the photosynthetic cells of plants. There are not two processes at work here. Its one continuous one that rolls on up from the prebiotic soup to hyper advanced space aliens. You are probably stuck on the idea of emergence. You imagine there must be some dividing line between intelligent and non intelligent. Therefore the point at which it emerges must be some spectacular miracle of engineering. When in fact there is no dividing line, just a continuum of consciousness from the small to the large. Read up on panspermia.
- plutonorm 5y ago
  sorry I mean panpsychism! finger slip