4 ms·

The imagined computers that looked so unrealistic in so many past science-fiction movies and TV shows no longer look... unrealistic. Those imagined computers n

by cs702 4y ago

The imagined computers that looked so unrealistic in so many past science-fiction movies and TV shows no longer look... unrealistic. Those imagined computers now look more like upgrades to current technology. Imagine an AI model 10x, 100x, 1000x larger than GPT4. That feels... surreal to me.

lm28469 4y ago
> Those imagined computers now look more like upgrades to current technology. I still don't get that argument. You can train a parrot for a week and he'll talk a few words, training it for a year won't make him become Shakespeare, LLMs will never ever automagically evolve into these AGI tier computers, not even their creators argue that, no one besides techbros argue that in fact.
- fiso64 4y ago
  >not even their creators argue that Yes, their creators take the far more defensible and scientific position of saying "we don't know".
  - lm28469 4y ago
    Not at all Same Altman (openAI ceo) : > The GPT-4 rumor mill is a ridiculous thing. I don’t know where it all comes from, people are begging to be disappointed and they will be. The hype is just like... We don’t have an actual AGI
    - letmevoteplease 4y ago
      He was responding to viral tweets[1] that said GPT-4 had "100 trillion parameters" and was "500x more powerful than GPT-3," which is indeed absurd hype. On the other hand, two weeks ago a team of Microsoft researchers put out a paper claiming that GPT-4 "could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system." [1] https://twitter.com/andrewsteinwold/status/1594889562526027777 https://twitter.com/andrewsteinwold/status/15948895625260277... [2] https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712
      - lm28469 4y ago
        You're missing the end of the quote, a bit further in the next sentence: > including the possible need for pursuing a new paradigm that moves beyond next-word prediction. So all in all "it's not AGI and we might need a paradigm shift to achieve it" A paradigm shift being: "a fundamental change in approach or underlying assumptions."
        fiso64 4y ago
        Exactly... In other words, they say that they don't know whether this will get us to AGI.
- WalterSear 4y ago
  We are hearing about new, improved parrots on an almost daily basis.
  - lm28469 4y ago
    And no matter how many W16 you put in a bugatti veyron it will never be a space shuttle I don't get how people, especially here of all places, fall for such things
    - WalterSear 4y ago
      Are you not familiar with the process of software engineering? Replacing parts until you get a space shuttle is exactly how systems are evolved.
- Name_Chawps 4y ago
  I can, right now, have GPT 4 write me sonnets on any topic. We are already beyond mere parroting. People say "it's just parroting what it's heard"! But did GPT4's training data really include a sonnet about Minecraft creepers? So they're not parroting. They're parroting PLUS something else. Plus some kind of analysis and transformation. Your claim is that this transformation is superficial and won't become deeper as the neural networks become deeper. We'll see.
  - marcosdumay 4y ago
    > But did GPT4's training data really include a sonnet about Minecraft creepers? The entire internet is one hell of a large knowledge base. It does surely look like it's doing some transformation. But I don't know how to get certainty on this.
  - lm28469 4y ago
    Yes you can cherry pick things that work very well, in the meantime it still can't rhyme in French, which is 12 years old type of intelligence It's wrong > 50% of the time for anything semi complex and semi niche, that's not AGI, no one in the field of AI argue it's AGI, only people on twitter and reddit with absolutely no background in tech and/or AI argue it's AGI It's a very very very good chatbot, built with very very impressive technology and amount of data, which is already quite impressive and definitely a big milestone, it's not AGI, not even close It's like saying the leather auger of my swiss army knife is a step towards an aerospace grade titanium cutting CNC machine
    - dandellion 4y ago
      I can't rhyme in French either. A lot of these comments just remind me of that Alan Kay quote: “Artificial intelligence is what we don’t know how to do yet.” In the past year we've got models that can draw, write poetry and code better than a significant percentage of people, and now the goal posts are moving so fast they're even hard to see. I'd probably agree that we're still a few breakthroughs away from something that could pass as AGI, but the current models are already better than just "parroting". And even if progress stales for the next 10 or 15 years like it has happened frequently with AI, everything that it has learned to do in the last couple years can be fine-tuned and made a lot more efficient, which I expect will have a big impact already.
- ravenstine 4y ago
  I've been noticing this pattern on HN where someone analogizes current generation LLMs to parrots while simultaneously claiming to know the future of LLMs without any justification. Am I a "techbro" because I don't believe that we know much about what the future of LLMs has in store for us, or because I find the parrot analogy to be utterly specious?
  - lm28469 4y ago
    You're free to believe whatever you want but then we're deep into religious territory, which is fine in itself but doesn't really matter here

by cs702 4y ago

lm28469 4y ago

> Those imagined computers now look more like upgrades to current technology. I still don't get that argument. You can train a parrot for a week and he'll talk a few words, training it for a year won't make him become Shakespeare, LLMs will never ever automagically evolve into these AGI tier computers, not even their creators argue that, no one besides techbros argue that in fact.

fiso64 4y ago
>not even their creators argue that Yes, their creators take the far more defensible and scientific position of saying "we don't know".
- lm28469 4y ago
  Not at all Same Altman (openAI ceo) : > The GPT-4 rumor mill is a ridiculous thing. I don’t know where it all comes from, people are begging to be disappointed and they will be. The hype is just like... We don’t have an actual AGI
  - letmevoteplease 4y ago
    He was responding to viral tweets[1] that said GPT-4 had "100 trillion parameters" and was "500x more powerful than GPT-3," which is indeed absurd hype. On the other hand, two weeks ago a team of Microsoft researchers put out a paper claiming that GPT-4 "could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system." [1] https://twitter.com/andrewsteinwold/status/1594889562526027777 https://twitter.com/andrewsteinwold/status/15948895625260277... [2] https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712
    - lm28469 4y ago
      You're missing the end of the quote, a bit further in the next sentence: > including the possible need for pursuing a new paradigm that moves beyond next-word prediction. So all in all "it's not AGI and we might need a paradigm shift to achieve it" A paradigm shift being: "a fundamental change in approach or underlying assumptions."
      - fiso64 4y ago
        Exactly... In other words, they say that they don't know whether this will get us to AGI.
WalterSear 4y ago
We are hearing about new, improved parrots on an almost daily basis.
- lm28469 4y ago
  And no matter how many W16 you put in a bugatti veyron it will never be a space shuttle I don't get how people, especially here of all places, fall for such things
  - WalterSear 4y ago
    Are you not familiar with the process of software engineering? Replacing parts until you get a space shuttle is exactly how systems are evolved.
Name_Chawps 4y ago
I can, right now, have GPT 4 write me sonnets on any topic. We are already beyond mere parroting. People say "it's just parroting what it's heard"! But did GPT4's training data really include a sonnet about Minecraft creepers? So they're not parroting. They're parroting PLUS something else. Plus some kind of analysis and transformation. Your claim is that this transformation is superficial and won't become deeper as the neural networks become deeper. We'll see.
- marcosdumay 4y ago
  > But did GPT4's training data really include a sonnet about Minecraft creepers? The entire internet is one hell of a large knowledge base. It does surely look like it's doing some transformation. But I don't know how to get certainty on this.
- lm28469 4y ago
  Yes you can cherry pick things that work very well, in the meantime it still can't rhyme in French, which is 12 years old type of intelligence It's wrong > 50% of the time for anything semi complex and semi niche, that's not AGI, no one in the field of AI argue it's AGI, only people on twitter and reddit with absolutely no background in tech and/or AI argue it's AGI It's a very very very good chatbot, built with very very impressive technology and amount of data, which is already quite impressive and definitely a big milestone, it's not AGI, not even close It's like saying the leather auger of my swiss army knife is a step towards an aerospace grade titanium cutting CNC machine
  - dandellion 4y ago
    I can't rhyme in French either. A lot of these comments just remind me of that Alan Kay quote: “Artificial intelligence is what we don’t know how to do yet.” In the past year we've got models that can draw, write poetry and code better than a significant percentage of people, and now the goal posts are moving so fast they're even hard to see. I'd probably agree that we're still a few breakthroughs away from something that could pass as AGI, but the current models are already better than just "parroting". And even if progress stales for the next 10 or 15 years like it has happened frequently with AI, everything that it has learned to do in the last couple years can be fine-tuned and made a lot more efficient, which I expect will have a big impact already.
ravenstine 4y ago
I've been noticing this pattern on HN where someone analogizes current generation LLMs to parrots while simultaneously claiming to know the future of LLMs without any justification. Am I a "techbro" because I don't believe that we know much about what the future of LLMs has in store for us, or because I find the parrot analogy to be utterly specious?
- lm28469 4y ago
  You're free to believe whatever you want but then we're deep into religious territory, which is fine in itself but doesn't really matter here