5 ms·

I have been bearish on AGI, but GPT2 surprised me with the lucidity of its samples. My take from the past few years is that we're 99% done with the visual cort

by rjeli 7y ago

I have been bearish on AGI, but GPT2 surprised me with the lucidity of its samples. My take from the past few years is that we're 99% done with the visual cortex - convolutional nets can be trained to perform any visual task a human can in <100ms. Now I'm mostly convinced that GPT2 has solved the language cortex, and can babble as well as we will ever need it to. We just need a prefrontal cortex (symbolic processing / RL / whatever your pet theory is) to drive the components, which is a problem we have not even started to solve. I am 90% sure it is a different class of problem and we won't knock it out of the park in 5 years like the visual/language cortexes, but we can hope. edit: it's possible cognition follows from language, which would be convenient. is GPT2 smarter than a dog? I don't think so but I could be wrong ¯\_(ツ)_/¯

kragen 7y ago
I have been bearish on AGI, but GPT2 surprised me with the lucidity of both paths. I still maintain my support for the basic metric of the GPT-I. However, I have a number of requirements on how my proposal is to be funded to resolve concerns. First, I strongly believe that academic research should be the method of choice (that is, if we are to figure out how to make AGI possible), and I advocate funding to support results from the central bank community. Second, given that the GPT can be articulated in mathematical terms, this should be reflected in funding policy. A very serious concern is that if funding of GPT is disincentivized, investors may react similarly to the way they reacted to AGI. This is
- bonoboTP 7y ago
  You generated this with GPT, right?
  - kragen 7y ago
    Yes.
    - 1ris 7y ago
      While Markov chains sound like a schizophrenic, this sounds like a banker on coke.
    - 1ris 7y ago
      While Markov chains sound like a schizophrenic, this sounds like a banker on coke. Pretty impressive progress, i guess.
      - kragen 7y ago
        That's because of the word “bearish”. It can sound like just about anything. For example, prompted by your comment, it sounds like a human interest journalist. (Journalism and fanfic, in particular HPMOR, seem to have comprised a large part of its training corpus; it composes Harry Potter porn at the slightest provocation; “Hermione moaned”, say.) While Markov chains sound like a schizophrenic, this sounds like a spacial disjointed notebook, as if somebody was trying to write in two places at the same time. "Today is Monday, it's Saturday night, I forgot to write to my dad and he only leaves the house for a couple of hours." Yet, sometimes Markov chains turn out to be the most beautiful art form. His wife Jennifer Neil, whom he met at a barbecue and has been married to since 1998, attributes this creative process to the constant ups and downs in his old job. "His numbers are just insane," says Jennifer Neil. "I don't know where he keeps them, but they're very mind boggling." Somewhat disconnected from the actual

by rjeli 7y ago

kragen 7y ago

I have been bearish on AGI, but GPT2 surprised me with the lucidity of both paths. I still maintain my support for the basic metric of the GPT-I. However, I have a number of requirements on how my proposal is to be funded to resolve concerns. First, I strongly believe that academic research should be the method of choice (that is, if we are to figure out how to make AGI possible), and I advocate funding to support results from the central bank community. Second, given that the GPT can be articulated in mathematical terms, this should be reflected in funding policy. A very serious concern is that if funding of GPT is disincentivized, investors may react similarly to the way they reacted to AGI. This is

bonoboTP 7y ago
You generated this with GPT, right?
- kragen 7y ago
  Yes.
  - 1ris 7y ago
    While Markov chains sound like a schizophrenic, this sounds like a banker on coke.
  - 1ris 7y ago
    While Markov chains sound like a schizophrenic, this sounds like a banker on coke. Pretty impressive progress, i guess.
    - kragen 7y ago
      That's because of the word “bearish”. It can sound like just about anything. For example, prompted by your comment, it sounds like a human interest journalist. (Journalism and fanfic, in particular HPMOR, seem to have comprised a large part of its training corpus; it composes Harry Potter porn at the slightest provocation; “Hermione moaned”, say.) While Markov chains sound like a schizophrenic, this sounds like a spacial disjointed notebook, as if somebody was trying to write in two places at the same time. "Today is Monday, it's Saturday night, I forgot to write to my dad and he only leaves the house for a couple of hours." Yet, sometimes Markov chains turn out to be the most beautiful art form. His wife Jennifer Neil, whom he met at a barbecue and has been married to since 1998, attributes this creative process to the constant ups and downs in his old job. "His numbers are just insane," says Jennifer Neil. "I don't know where he keeps them, but they're very mind boggling." Somewhat disconnected from the actual