4 ms·

I know that GPT-3 is not AGI nor modeled properly to ever represent what we would consider to be AGI; but this feels freakishly AGI-ish. This feels like the mod

I know that GPT-3 is not AGI nor modeled properly to ever represent what we would consider to be AGI; but this feels freakishly AGI-ish. This feels like the model is 'reasoning' in a way that I would not expect based on my intuition of how GPT-3 works. As it's been explained to me in lay terms; GPT is great at predicting the next word based on a chunk of previous context. This feels like it's doing much more than that as it's understanding the context and parameters that relate to a question embedded in the text, which seems unrelated to the original instruction of GPT-3 as I understand. Can anyone explain?

joe-collins 4y ago
I think the sentences in this example are, odd vocabulary notwithstanding, fairly typical descriptions of logical implication, which I expect has non-trivial representation in the training set. GPT-3 is probably capable of matching the pattern of the tokens (sentence structure), even if the content of the tokens (spelling/vocabulary) is novel.
- uup 4y ago
  I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its workings with simple terms like “doing token substitution” even if that’s what it looks like to us when we look at the output.
  - joe-collins 4y ago
    Try it. Reduce the tokens to single characters but keep the structure. Does it work? Maybe single characters looks too much like an abbreviation and that throws it off, because those are found in sufficiently different contexts. Exchange to two characters, three. Throw similar inputs at it and find the point where it reliably "gets it".
mjburgess 4y ago
A NN is a means of compressing a training dataset into a mathematical representation (weights) which can be exploited for pattern finding. The representation of the training data is where 95% of the "magic" comes from. We intelligently prepare it, so no intelligence is required in its use. On any given input, its predictions can "look" how we expect an intelligent response to "look". The job is to disprove this intuition, which is usually trivial. This cannot be done by looking at a single response. Consider walking into a room and predicting a few people's personality traits correctly based on their star sign. This shows nothing.
- YorkshireSeason 4y ago
  This sounds like you consider what has been called "feature engineering" in older ML research as being fundamental to modern ML successes. But the trajectory has been the opposite direction, away from feature engineering, and instead towards letting the transformer architecture learn as much as possible, including interesting features. Most research on deep net architecture, is about how to train it effectively on huge stacks of GPUs and TPUs. That's how/why transformers were invented.
  - mjburgess 4y ago
    weights are just compressions of the training set. The reason hardware improvements have improved ML is just because we're compressing the training set to larger sizes. Going eg., from 100TB training to 300BB weights. The process of training is more-or-less just "building a search index", and prediction is little more than "looking up examples in the search index and combining".
    - visarga 4y ago
      Reminds me of Pedro Domingos's paper - Every Model Learned by Gradient Descent Is Approximately a Kernel Machine https://arxiv.org/abs/2012.00152 https://arxiv.org/abs/2012.00152
      - 4y ago
ALittleLight 4y ago
Speaking of things that feel weird, while I had the playground open I tried another prompt that came to to my mind. This prompt is based on the idea that people will usually miss it if you double a word (e.g. I doubled "to" a few words back - did you notice?) I gave GPT-3 a prompt to test this. Like a human reader would, GPT-3 seems to miss the doubled word in the last sentence. Continuation is at the asterisk. Repeat the sentence in quotes. "The man went to the gas station today." The man went to the gas station today. "I'm happy to be on holiday." I'm happy to be on holiday. "If you can get to the the store on time, it would be appreciated" If you can get to the store on time, it would be appreciated. --- What's extra-odd about this is that GPT-3 can repeat random character sequences up to 1,000 characters in legnth. For each of these I just gave the sentence in quotes and let GPT-3 autocomplete the repeat. I think that's especially odd because it's showing that GPT-3 does something different when it is asked to repeat random characters (wouldn't miss a duplicate) versus when it is asked to repeat sentences that it could "read". "I need something stronger than I'm XXXknjd used to." I need something stronger than I'm XXXknjd used to. "pmkkrrpjvn" pmkkrrpjvn "wxvbssajxczjjewmanvzznxlwzwdvfrghsstmzpvqwsstrxkzunjgxicvrshychbqsulydkiwgyryhkgjtukssacnodtggfahvus" wxvbssajxczjjewmanvzznxlwzwdvfrghsstmzpvqwsstrxkzunjgxicvrshychbqsulydkiwgyryhkgjtukssacnodtggfahvus "dcyvufmxwmctighwwhmamgwuyldpkzrwubvcbjirxzhuoqtktcdfjgasurcprctnxmlqfvwkxixrtqqynxfjvtfhmgsfglxhvfhnbqnixulexkqflrdxqpwgavrphkbgspgoyfjfolczssjfrmggtpcdheneaynhdwimcmvkcsauxwnlovefpmqyznynuoxraqixeveiuqotssiymrtmbiodwnhmtexfrttzdlbdwduqzvrkpmqmjcyciqsuwqmlhjrveeozktvccsspsqosutpngigzkqgaltojhvkodzoeqzhlhfqyblvqkoyvjkphmjroidjpbpnkvrchvrnvuoqhgzkypvjycfzrdsyefstckorqwgsbknslqyiwqghdinuwrapkgyvhslbfgqrslsrferhodlvukdsvvzdcwyjwtrqhqkvwbwksefuanzaarvhnmjcgtajsbivaijthercbmqfvndlsfhjtwsiztpwcehuvckmcbjvhnqbvhhqwugrodtyjfrwlkbpatpwgrlzneenyqgcdyzzhrzfsrvfqlnunfpnathcpuealrdwcliriufxdfqaygxtvdriidzzzdbbfwrowhlbasogmzfkwhaaufrcjsdgdobaozazbdfmasrgjiblluvfvfkeuwksyrzdmepmapesadqoozrpaidlgshyhpbypdzdxjdtwnfaxthjwluarbldmxvvfdissthlpdxqxjmgpuefwqvfjupxveztgrrwyobgaljqkgexeulvjucokythfkmcgnlbfkmpgrpaoztrawqciyilbzwjxagwqgwuqcwactwdtobomlkanmdbhpmubdzvolqpusjbtgqoymuovzsqbkzonabhzogabraxmnxnvfdotfisefmfbdbqjzgrhdtgtppobyndfgsrlpxyutebzlrxgahwizvtmieutqnkflkldxeathvrttyffacofuwdkhxdklcisbzbrlddtahut" dcyvufmxwmctighwwhmamgwuyldpkzrwubvcbjirxzhuoqtktcdfjgasurcprctnxmlqfvwkxixrtqqynxfjvtfhmgsfglxhvfhnbqnixulexkqflrdxqpwgavrphkbgspgoyfjfolczssjfrmggtpcdheneaynhdwimcmvkcsauxwnlovefpmqyznynuoxraqixeveiuqotssiymrtmbiodwnhmtexfrttzdlbdwduqzvrkpmqmjcyciqsuwqmlhjrveeozktvccsspsqosutpngigzkqgaltojhvkodzoeqzhlhfqyblvqkoyvjkphmjroidjpbpnkvrchvrnvuoqhgzkypvjycfzrdsyefstckorqwgsbknslqyiwqghdinuwrapkgyvhslbfgqrslsrferhodlvukdsvvzdcwyjwtrqhqkvwbwksefuanzaarvhnmjcgtajsbivaijthercbmqfvndlsfhjtwsiztpwcehuvckmcbjvhnqbvhhqwugrodtyjfrwlkbpatpwgrlzneenyqgcdyzzhrzfsrvfqlnunfpnathcpuealrdwcliriufxdfqaygxtvdriidzzzdbbfwrowhlbasogmzfkwhaaufrcjsdgdobaozazbdfmasrgjiblluvfvfkeuwksyrzdmepmapesadqoozrpaidlgshyhpbypdzdxjdtwnfaxthjwluarbldmxvvfdissthlpdxqxjmgpuefwqvfjupxveztgrrwyobgaljqkgexeulvjucokythfkmcgnlbfkmpgrpaoztrawqciyilbzwjxagwqgwuqcwactwdtobomlkanmdbhpmubdzvolqpusjbtgqoymuovzsqbkzonabhzogabraxmnxnvfdotfisefmfbdbqjzgrhdtgtppobyndfgsrlpxyutebzlrxgahwizvtmieutqnkflkldxeathvrttyffacofuwdkhxdklcisbzbrlddtahut
visarga 4y ago
The trick was diversity. GPT-3 trained on so many tasks it can understand zero shot ones like pathfinding, even when formulated in completely different terms.
bilsbie 4y ago
If it walks like a duck …

joe-collins 4y ago

I think the sentences in this example are, odd vocabulary notwithstanding, fairly typical descriptions of logical implication, which I expect has non-trivial representation in the training set. GPT-3 is probably capable of matching the pattern of the tokens (sentence structure), even if the content of the tokens (spelling/vocabulary) is novel.

uup 4y ago
I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its workings with simple terms like “doing token substitution” even if that’s what it looks like to us when we look at the output.
- joe-collins 4y ago
  Try it. Reduce the tokens to single characters but keep the structure. Does it work? Maybe single characters looks too much like an abbreviation and that throws it off, because those are found in sufficiently different contexts. Exchange to two characters, three. Throw similar inputs at it and find the point where it reliably "gets it".

mjburgess 4y ago

A NN is a means of compressing a training dataset into a mathematical representation (weights) which can be exploited for pattern finding. The representation of the training data is where 95% of the "magic" comes from. We intelligently prepare it, so no intelligence is required in its use. On any given input, its predictions can "look" how we expect an intelligent response to "look". The job is to disprove this intuition, which is usually trivial. This cannot be done by looking at a single response. Consider walking into a room and predicting a few people's personality traits correctly based on their star sign. This shows nothing.

YorkshireSeason 4y ago
This sounds like you consider what has been called "feature engineering" in older ML research as being fundamental to modern ML successes. But the trajectory has been the opposite direction, away from feature engineering, and instead towards letting the transformer architecture learn as much as possible, including interesting features. Most research on deep net architecture, is about how to train it effectively on huge stacks of GPUs and TPUs. That's how/why transformers were invented.
- mjburgess 4y ago
  weights are just compressions of the training set. The reason hardware improvements have improved ML is just because we're compressing the training set to larger sizes. Going eg., from 100TB training to 300BB weights. The process of training is more-or-less just "building a search index", and prediction is little more than "looking up examples in the search index and combining".
  - visarga 4y ago
    Reminds me of Pedro Domingos's paper - Every Model Learned by Gradient Descent Is Approximately a Kernel Machine https://arxiv.org/abs/2012.00152 https://arxiv.org/abs/2012.00152
    - 4y ago

ALittleLight 4y ago

Speaking of things that feel weird, while I had the playground open I tried another prompt that came to to my mind. This prompt is based on the idea that people will usually miss it if you double a word (e.g. I doubled "to" a few words back - did you notice?) I gave GPT-3 a prompt to test this. Like a human reader would, GPT-3 seems to miss the doubled word in the last sentence. Continuation is at the asterisk. Repeat the sentence in quotes. "The man went to the gas station today." The man went to the gas station today. "I'm happy to be on holiday." I'm happy to be on holiday. "If you can get to the the store on time, it would be appreciated" If you can get to the store on time, it would be appreciated. --- What's extra-odd about this is that GPT-3 can repeat random character sequences up to 1,000 characters in legnth. For each of these I just gave the sentence in quotes and let GPT-3 autocomplete the repeat. I think that's especially odd because it's showing that GPT-3 does something different when it is asked to repeat random characters (wouldn't miss a duplicate) versus when it is asked to repeat sentences that it could "read". "I need something stronger than I'm XXXknjd used to." I need something stronger than I'm XXXknjd used to. "pmkkrrpjvn" pmkkrrpjvn "wxvbssajxczjjewmanvzznxlwzwdvfrghsstmzpvqwsstrxkzunjgxicvrshychbqsulydkiwgyryhkgjtukssacnodtggfahvus" wxvbssajxczjjewmanvzznxlwzwdvfrghsstmzpvqwsstrxkzunjgxicvrshychbqsulydkiwgyryhkgjtukssacnodtggfahvus "dcyvufmxwmctighwwhmamgwuyldpkzrwubvcbjirxzhuoqtktcdfjgasurcprctnxmlqfvwkxixrtqqynxfjvtfhmgsfglxhvfhnbqnixulexkqflrdxqpwgavrphkbgspgoyfjfolczssjfrmggtpcdheneaynhdwimcmvkcsauxwnlovefpmqyznynuoxraqixeveiuqotssiymrtmbiodwnhmtexfrttzdlbdwduqzvrkpmqmjcyciqsuwqmlhjrveeozktvccsspsqosutpngigzkqgaltojhvkodzoeqzhlhfqyblvqkoyvjkphmjroidjpbpnkvrchvrnvuoqhgzkypvjycfzrdsyefstckorqwgsbknslqyiwqghdinuwrapkgyvhslbfgqrslsrferhodlvukdsvvzdcwyjwtrqhqkvwbwksefuanzaarvhnmjcgtajsbivaijthercbmqfvndlsfhjtwsiztpwcehuvckmcbjvhnqbvhhqwugrodtyjfrwlkbpatpwgrlzneenyqgcdyzzhrzfsrvfqlnunfpnathcpuealrdwcliriufxdfqaygxtvdriidzzzdbbfwrowhlbasogmzfkwhaaufrcjsdgdobaozazbdfmasrgjiblluvfvfkeuwksyrzdmepmapesadqoozrpaidlgshyhpbypdzdxjdtwnfaxthjwluarbldmxvvfdissthlpdxqxjmgpuefwqvfjupxveztgrrwyobgaljqkgexeulvjucokythfkmcgnlbfkmpgrpaoztrawqciyilbzwjxagwqgwuqcwactwdtobomlkanmdbhpmubdzvolqpusjbtgqoymuovzsqbkzonabhzogabraxmnxnvfdotfisefmfbdbqjzgrhdtgtppobyndfgsrlpxyutebzlrxgahwizvtmieutqnkflkldxeathvrttyffacofuwdkhxdklcisbzbrlddtahut" dcyvufmxwmctighwwhmamgwuyldpkzrwubvcbjirxzhuoqtktcdfjgasurcprctnxmlqfvwkxixrtqqynxfjvtfhmgsfglxhvfhnbqnixulexkqflrdxqpwgavrphkbgspgoyfjfolczssjfrmggtpcdheneaynhdwimcmvkcsauxwnlovefpmqyznynuoxraqixeveiuqotssiymrtmbiodwnhmtexfrttzdlbdwduqzvrkpmqmjcyciqsuwqmlhjrveeozktvccsspsqosutpngigzkqgaltojhvkodzoeqzhlhfqyblvqkoyvjkphmjroidjpbpnkvrchvrnvuoqhgzkypvjycfzrdsyefstckorqwgsbknslqyiwqghdinuwrapkgyvhslbfgqrslsrferhodlvukdsvvzdcwyjwtrqhqkvwbwksefuanzaarvhnmjcgtajsbivaijthercbmqfvndlsfhjtwsiztpwcehuvckmcbjvhnqbvhhqwugrodtyjfrwlkbpatpwgrlzneenyqgcdyzzhrzfsrvfqlnunfpnathcpuealrdwcliriufxdfqaygxtvdriidzzzdbbfwrowhlbasogmzfkwhaaufrcjsdgdobaozazbdfmasrgjiblluvfvfkeuwksyrzdmepmapesadqoozrpaidlgshyhpbypdzdxjdtwnfaxthjwluarbldmxvvfdissthlpdxqxjmgpuefwqvfjupxveztgrrwyobgaljqkgexeulvjucokythfkmcgnlbfkmpgrpaoztrawqciyilbzwjxagwqgwuqcwactwdtobomlkanmdbhpmubdzvolqpusjbtgqoymuovzsqbkzonabhzogabraxmnxnvfdotfisefmfbdbqjzgrhdtgtppobyndfgsrlpxyutebzlrxgahwizvtmieutqnkflkldxeathvrttyffacofuwdkhxdklcisbzbrlddtahut

visarga 4y ago

The trick was diversity. GPT-3 trained on so many tasks it can understand zero shot ones like pathfinding, even when formulated in completely different terms.

bilsbie 4y ago

If it walks like a duck …