4 ms·

> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vo

• lalos 4y ago
  Not to mention that the corpus mostly will have the correct case for most common words like apple. Train it with essays of ESL students and you'll get something else.
• prox 4y ago
  Wouldn’t this also be correlated somehow in its vector space?
• laszlokorte 4y ago
  From my naïve point of view it seems obvious that at any point where both an „a“ or a „an“ would fit the model randomly selects one of them and by doing so reduces the set of possible nouns to follow.
  • Sharlin 4y ago
    No, of course not randomly. If you RTA, it should be clear that some prompts are more likely to be followed by an “an” than others.
• kelnos 4y ago
  Yeah, my feeling here was that it's sort of tautological: if GPT predicts "a", then it must then predict a word that would follow "a" and not require "an" (and vice versa). And if you think about it from the opposite direction: if it's working out a response that is eventually going to have "apple" in it, then all the data it's trained on is going to cause it to predict "an" even before it needs to predict "apple". (Admittedly, all this ML/AI stuff is still beyond my current level of understanding, so I'm sure my thinking here is off.)
  • caf 4y ago
    I think that's essentially right - if it's already "thinking about" apples, then ' an' will be significantly upregulated at the expense of ' a', then if it does choose to output an ' an' token, ' apple' becomes a highly likely followup. You could probably test this by seeing if prompts containing a lot of nouns that start with a vowel sound results in output that contains a higher proportion of otherwise unrelated first-vowel nouns. (ie your prompt includes lots of apples, apricots, avocados, asparagus, aubergines, elderberries, eggplants, endives, oranges, olives, okras, onions and you count the proportion of non-food nouns in the result that start with a vowel and non-vowel sound).
  • clementneo 4y ago
    The main issue is that GPT is fundamentally an autoregressive language model — it's only predicting the next token based on the prompt at a single time. Every time it wants to predict the next word, it adds the previously predicted word into the prompt, repeating the cycle. We can intuitively guess that the model is 'working out a response that is eventually going to have "apple" in it', but we don't actually know how the model 'thinks' ahead about its response. To rephrase that for this case: what is the specific mechanism in GPT-2 that (1) makes it realise that the word 'apple' is significant in this prompt, and (2) use that knowledge to push the model to predict 'an'? Finding this neuron would only answer the some portion of (2). (And to rephrase this for the general case, which gives us the initial question: How does GPT-2 know when, given a suitable context, to predict 'an' over 'a'?)
    • IgorPartola 4y ago
      Do an improv exercise with a friend. Construct a story about a sentient apple one word at a time. I guarantee you that at one point one of you will set up the other with an “an” in your story. This is how I believe this works.
      • minkzilla 4y ago
        Your friend is thinking ahead that the next word would be apple When they say ‘an’. But GPT can’t think ahead what token it will add after the one it is on. Or can it? It could “predict” internally the word apple for the next “meaningful” word and output ‘an’ because of this.
• varispeed 4y ago
  My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple. Exactly what you mean by bigram.
  • caf 4y ago
    And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".
    • PeterisP 4y ago
      This is also why a bunch of words have changed - e.g. apron was originally called napron, but when people were saying "a napron" it got mangled to "an apron"..
      • caf 4y ago
        Maybe one day "knife" will become "ife".
      • hgsgm 4y ago
        And narrow from "an arrow"
    • chimpansteve 4y ago
      It's also not unheard of for native English speakers to use it differently (in speech, at least) as a consequence of regional accents. I would use "a history book", because my accent pronounces the h sound. My wife says "an istory book", because her accent drops the h. But yeah, it should be easier when you're just dealing with text I guess
  • 4y ago
• theptip 4y ago
  I think this would be a valid objection if they stopped there. But then: “ Testing the neuron on a larger dataset” If I follow correctly, they test a bunch of different completions that contain “an”. So they are not just detecting the bigram “an apple”, but the common activation among a bunch of “an X” activations where X is completely different.
• seppel 4y ago
  I don't think you are missing something. I think this whole "GPT-2 is predicting only one word at a time" is a red herring anyway. Of course it can only answer the next word, because there is only room in its outputs for the next word. But it has to compute much more. It a huge hidden internal state where it has to first encode what the given sentence is about, then predict some general concept in which the continuation goes, decide the locally correct syntactical structure and only from this you can predict the next word.
• josephmiller 4y ago
  Author here! I think this is reasonable but I have two responses. 1. It's kinda interesting because this is a clear case where the model must be thinking beyond the next token, whereas in most contexts it's hard to say whether the model thinks ahead at all (although I would guess that it does most of the time). 2. More importantly, the key question here is how it works. We're not surprised that it has this behavior, but we want to understand which exact weights and biases in the network are responsible. Note also that this is just the introductory sentence and the rest of the article would read exactly the same without it. > it doesn't seem that surprising that there might be cases where there is a dominant bigram [...] that would trigger an "an" prediction, without actually predicting the following word first btw I don't really understand what you mean by this. Bigrams can explain the second prediction in a two word pair but not the first.
  • defen 4y ago
    What happens if you ask it to complete "I climbed up the pear tree and picked a pear. I climbed up the apple tree and picked a"
    • eropple 4y ago
      I asked this to ChatGPT. Its answer? "Red apple". ...well played.
  • bilsbie 4y ago
    As a test, during text generation could you change an “a” to an “an” and see if it changes the noun. Or did it already have a noun in mind and it sticks with that.
  • tqi 4y ago
    Thanks for responding, and for the cool article! Agree this is pretty tangential to the rest of the article. So (again, I am very much not an expert so please correct me if I'm wrong) I guess my analogy would be if instead of predicting one word at a time, it predicted one letter at a time. At some point, would only be one word that could fit. So if prompted with "SENTE", and it returned "N", that doesn't mean that it's thinking ahead to the "CE" / knows that it is spelling "SENTENCE" already. Is that a correct way to think of it?
• VeninVidiaVicii 4y ago
  Word shmord, “an apple” is a lexical concept.