4 ms·

What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive st

What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive steps. If GPT-4 is simply trained on a large corpus of text, how is it able to do this? Does this imply that there is some emergent deductive ability that you get simply by learning "language?" Or am I missing something? Obviously, I'm assuming that GPT-4 wasn't trained on the exams that it was tested against.

machiaweliczny 4y ago
See hutter prize. Best way to compress data is by understanding it. I am not exactly sure how it manifests in transformer architecture.
- jacquesm 4y ago
  The future: You don't compress the movie frames, you supply a script and a list of actors and scenery and garb descriptions.
  - baq 4y ago
    The Kolmogorov complexity, applied to entertainment. Yes, looks like we’re going there.
  - agnosticmantis 4y ago
    Looks eerily like the past, when cameras didn’t exist and people wrote plays to be acted in theaters…
dannyz 4y ago
It would be interesting to see some example questions and answers. Since the test is multiple choice is it possible that the model has gotten very good at estimating how likely a possible answer is?
Analemma_ 4y ago
It's totally possible: Daniel Dennett's theory of sentient consciousness-- specifically, what we have that animals do not-- is that it is "ignited" by language acquisition. It's within the realm of possibility that LLMs provide empirical proof or disproof of this hypothesis.
- smith7018 4y ago
  I always find it outrageously pious to say we have "sentient consciousness" whereas animals don't. Animals have emotions; memories; wants; needs; the ability to use tools; personalities; an understanding of grief; an understanding of cause and effect; and much more. Just because they lack a formal language (that we can understand) doesn't mean they're any less "sentient" or "conscious."
  - fnovd 4y ago
    Sentient consciousness, you mean that weird meatbag thinking style? AI consciousness will be so, so much more.
- goatlover 4y ago
  LLM's don't have any sensory modalities. All of our conscious experiences are built up on either perceptual, proprioceptual or emotional sensations. An LLM generating text of a sunset isn't seeing the colors.
- goatlover 4y ago
  LLM's don't have any sensory modalities. All of our conscious experiences are built up on either perceptual, proprioceptual or emotional sensations. An LLM generating text of a sunset isn't seeing the colors. Dennett thinks consciousness, in the sense of the hard problem/subjectivity, is some kind of trick of the brain. So he proposes a linguistic trick. Language fools us into thinking there is something more than a functional stream of information.
- grantcas 4y ago
  [dead]
macrolocal 4y ago
They do leverage emergent abstractions. For example, in [1] a transformer model learns the coset structure of a group to better grok its multiplication table. [1] https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper.pdf https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper....
zamadatix 4y ago
I recently finished a 100 level informatics course so threw in one of the assignment questions about optimizing shirt sales under constraints. A "you can make these colors but you only have 8 hours to make them with a limited amount of the different dyes" kind of thing but nothing crazy like mixing dyes just a straight "do you know how to apply linear programming" thing. GPT-4 knew to use linear programming and acknowledged the constraints, even without me formatting the tabular data so the labels were with the values and properly separated! It also ran all of the 2-3 digit integer multiplications/divisions/subtractions/additions correctly. It still failed to "put it all together" in the final step and forgot some constraints. I prompted it "won't I run out of time?" and it acknowledged it then redid it forgetting a different constraint. I wasn't able to get it to come to the right conclusion. It feels like it has learned a pattern for solving these types of questions but hasn't really gained any actual reasoning about whether it's applying the pattern in a way that makes sense. It confidently announces that it followed all of the constraints when the pattern it chose to follow didn't involve one of the constraints. It then acknowledges it was wrong but doesn't apply reason as much as knows to apply a different pattern that fixes that specific issue. Another example is I asked it to configure some network interfaces on a Cisco switch in a certain way. I gave it 3 VLANs to configure the interface with knowing 1 was incorrect (in the 5000s, VLANs are only 12 bits long). It created the answer with tagging VLAN 5031. I asked what problems I'd run into running the generated commands and it gave some hypothetical risks, one of which being that VLANs must be in a certain range, but didn't reason that the commands included an invalid VLAN. I told it "isn't VLAN 5031 invalid?" and it apologize and corrected it. I then told it "isn't VLAN 1000 invalid?" and it apologized for it not being a valid VLAN and corrected it all the same even though it was valid. All that testing the limits said... it may not have emergent deductive ability but I think this learned pattern matching approach based on training situations extends far past where most people would think it would. I think GPT-5 or GPT-6 may well avoid the above problems without necessarily gaining emergent logical reasoning for them as much as just having a larger depth in the patterns. Large number operations are still interesting though and I'm not sure how they fit in. 646864613385/41348.5 returns "approximately" 15652.172205 which has the right first 3 digits but is off by a factor of 1000 and the rest of the digits are made up. I'm not sure if this is similarly explained by applying a pattern without reasoning about it but it feels like it could be. All that said I really don't know much about how the system is constructed, I just use it :).
jaqalopes 4y ago
From the blog post: "A minority of the problems in the exams were seen by the model during training, but we believe the results to be representative—see our technical report for details." They have a chart where they broke out results for the model with versus without "vision" i.e. having trained on the exam questions before.
00F_ 4y ago
emergent deductive ability. lol. what do you call its writing, emergent writing ability? this is an algorithm where all the numbers are random, literally everything about it is emergent except the structure of the nodes. people have this stupid idea that GTP cant do this or cant do that. if GTP were just a set of nodes on paper, without any weights, in the year 2010, everyone in the world would say there is no way in hell that a structure of nodes such as that could write lucidly like a human being and perform as well as a human on various tests. they would say "you mean to tell me that if you just applied random numbers to those nodes, it would acquire some kind of emergent writing ability, some kind of emergent reading ability? it could have anything. it could have anything that is in the set of all algorithms that fit in those nodes. that could be AGI, it could be anything. there is zero doubt to anyone with any sense that it is finding in those random numbers some basic primitives or elements of conscious thought. while not demonstrating sentience or human logic, it clearly forms abstractions that are similar to ones used in animal minds and it clearly possess ways of reasoning about or connecting these abstractions. this is literally the first AI that has done this, with several lesser spooky AIs leading up to this since 2011. GTP was the very first AI to demonstrate that spooky reasoning and lucidity and its gotten us all this. how about version 2? 3? how about, now that capital is going to flow into this like a burst dam, version 1000? we are fucked. these AI training algorithms are going to strike gold quickly and before you know it, the models will be running the whole mining operation and then we will be fucking toast. someone on here said hes worried about 20% unemployment rate. people have noooooo idea whats going on.

machiaweliczny 4y ago

See hutter prize. Best way to compress data is by understanding it. I am not exactly sure how it manifests in transformer architecture.

jacquesm 4y ago
The future: You don't compress the movie frames, you supply a script and a list of actors and scenery and garb descriptions.
- baq 4y ago
  The Kolmogorov complexity, applied to entertainment. Yes, looks like we’re going there.
- agnosticmantis 4y ago
  Looks eerily like the past, when cameras didn’t exist and people wrote plays to be acted in theaters…

dannyz 4y ago

It would be interesting to see some example questions and answers. Since the test is multiple choice is it possible that the model has gotten very good at estimating how likely a possible answer is?

Analemma_ 4y ago

It's totally possible: Daniel Dennett's theory of sentient consciousness-- specifically, what we have that animals do not-- is that it is "ignited" by language acquisition. It's within the realm of possibility that LLMs provide empirical proof or disproof of this hypothesis.

smith7018 4y ago
I always find it outrageously pious to say we have "sentient consciousness" whereas animals don't. Animals have emotions; memories; wants; needs; the ability to use tools; personalities; an understanding of grief; an understanding of cause and effect; and much more. Just because they lack a formal language (that we can understand) doesn't mean they're any less "sentient" or "conscious."
- fnovd 4y ago
  Sentient consciousness, you mean that weird meatbag thinking style? AI consciousness will be so, so much more.
goatlover 4y ago
LLM's don't have any sensory modalities. All of our conscious experiences are built up on either perceptual, proprioceptual or emotional sensations. An LLM generating text of a sunset isn't seeing the colors.
goatlover 4y ago
LLM's don't have any sensory modalities. All of our conscious experiences are built up on either perceptual, proprioceptual or emotional sensations. An LLM generating text of a sunset isn't seeing the colors. Dennett thinks consciousness, in the sense of the hard problem/subjectivity, is some kind of trick of the brain. So he proposes a linguistic trick. Language fools us into thinking there is something more than a functional stream of information.
grantcas 4y ago
[dead]

macrolocal 4y ago

They do leverage emergent abstractions. For example, in [1] a transformer model learns the coset structure of a group to better grok its multiplication table. [1] https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper.pdf https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper....

zamadatix 4y ago

I recently finished a 100 level informatics course so threw in one of the assignment questions about optimizing shirt sales under constraints. A "you can make these colors but you only have 8 hours to make them with a limited amount of the different dyes" kind of thing but nothing crazy like mixing dyes just a straight "do you know how to apply linear programming" thing. GPT-4 knew to use linear programming and acknowledged the constraints, even without me formatting the tabular data so the labels were with the values and properly separated! It also ran all of the 2-3 digit integer multiplications/divisions/subtractions/additions correctly. It still failed to "put it all together" in the final step and forgot some constraints. I prompted it "won't I run out of time?" and it acknowledged it then redid it forgetting a different constraint. I wasn't able to get it to come to the right conclusion. It feels like it has learned a pattern for solving these types of questions but hasn't really gained any actual reasoning about whether it's applying the pattern in a way that makes sense. It confidently announces that it followed all of the constraints when the pattern it chose to follow didn't involve one of the constraints. It then acknowledges it was wrong but doesn't apply reason as much as knows to apply a different pattern that fixes that specific issue. Another example is I asked it to configure some network interfaces on a Cisco switch in a certain way. I gave it 3 VLANs to configure the interface with knowing 1 was incorrect (in the 5000s, VLANs are only 12 bits long). It created the answer with tagging VLAN 5031. I asked what problems I'd run into running the generated commands and it gave some hypothetical risks, one of which being that VLANs must be in a certain range, but didn't reason that the commands included an invalid VLAN. I told it "isn't VLAN 5031 invalid?" and it apologize and corrected it. I then told it "isn't VLAN 1000 invalid?" and it apologized for it not being a valid VLAN and corrected it all the same even though it was valid. All that testing the limits said... it may not have emergent deductive ability but I think this learned pattern matching approach based on training situations extends far past where most people would think it would. I think GPT-5 or GPT-6 may well avoid the above problems without necessarily gaining emergent logical reasoning for them as much as just having a larger depth in the patterns. Large number operations are still interesting though and I'm not sure how they fit in. 646864613385/41348.5 returns "approximately" 15652.172205 which has the right first 3 digits but is off by a factor of 1000 and the rest of the digits are made up. I'm not sure if this is similarly explained by applying a pattern without reasoning about it but it feels like it could be. All that said I really don't know much about how the system is constructed, I just use it :).

jaqalopes 4y ago

From the blog post: "A minority of the problems in the exams were seen by the model during training, but we believe the results to be representative—see our technical report for details." They have a chart where they broke out results for the model with versus without "vision" i.e. having trained on the exam questions before.

00F_ 4y ago

emergent deductive ability. lol. what do you call its writing, emergent writing ability? this is an algorithm where all the numbers are random, literally everything about it is emergent except the structure of the nodes. people have this stupid idea that GTP cant do this or cant do that. if GTP were just a set of nodes on paper, without any weights, in the year 2010, everyone in the world would say there is no way in hell that a structure of nodes such as that could write lucidly like a human being and perform as well as a human on various tests. they would say "you mean to tell me that if you just applied random numbers to those nodes, it would acquire some kind of emergent writing ability, some kind of emergent reading ability? it could have anything. it could have anything that is in the set of all algorithms that fit in those nodes. that could be AGI, it could be anything. there is zero doubt to anyone with any sense that it is finding in those random numbers some basic primitives or elements of conscious thought. while not demonstrating sentience or human logic, it clearly forms abstractions that are similar to ones used in animal minds and it clearly possess ways of reasoning about or connecting these abstractions. this is literally the first AI that has done this, with several lesser spooky AIs leading up to this since 2011. GTP was the very first AI to demonstrate that spooky reasoning and lucidity and its gotten us all this. how about version 2? 3? how about, now that capital is going to flow into this like a burst dam, version 1000? we are fucked. these AI training algorithms are going to strike gold quickly and before you know it, the models will be running the whole mining operation and then we will be fucking toast. someone on here said hes worried about 20% unemployment rate. people have noooooo idea whats going on.