5 ms·

He is far from the only one. If you're interested in exploring this further I can really recommend taking a look at some of the papers that explore GPT-4's cap

He is far from the only one. If you're interested in exploring this further I can really recommend taking a look at some of the papers that explore GPT-4's capabilities. Most prominent among them are the "Sparks of AGI" paper from Microsoft, as well as the technical report from openai. Both of them are obviously to be taken with a grain of salt, but they serve as a pretty good jumping off point. There are some pretty good Videos on Youtube exploring these papers if you don't want to read them yourself. Also take a look at the stuff that Rob Miles has published over on Computerphile, as well as his own channel. He's an Alignment Researcher with a knack for explaining. He covers not just the theoretical dangers, but also real examples of misaligned ai, that alignment researchers have predicted would occur as capabilities grow. Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. It's still unclear that it will get us all the way there, but recent developments have made a lot of ai researchers rethink that possibility, with many of them significantly shortening their own estimates as to when and how we will get there. It's very unusual that the people that are better informed and closer to the research are more worried than the rest of the world and it's worth keeping this in mind as you explore the topic.

defgeneric 3y ago
> Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. This was basically the strategy of the OpenAI team if I understand them correctly. Most researchers in the field looked down on LLMs and it was a big surprise when they turned out to perform so well. It also seems to be the reason the big players are playing catch up right now.
- whimsicalism 3y ago
  I think it was a surprise the behaviors that were unlocked at different perplexity levels, but I don't really agree that LLMs were "looked down on."
  - TeMPOraL 3y ago
    Maybe not "looked down on", but more of "looked at as a promising avenue". I mean, 2-3 years ago, it felt LLMs are going to be nice storytellers at best. These days, we're wondering just how much of the overall process of "understanding" and "reasoning" can be reduced to adjacency search in sufficiently absurdly high-dimensional vector space.
    - whimsicalism 3y ago
      People certainly knew that language modeling was a key unsupervised objective to unlock inference on language. I agree that I think they underestimated quite how useful a product could be built around just the language modeling objective, but it's still been critical for most NLP advances of the last ~6+ years.
nradov 3y ago
I read that pre-print Microsoft paper. Despite the title, it doesn't actually show any real "sparks" of AGI (in the sense of something that could eventually pass a rigorous Turing test). What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness; our brains seem to be wired that way and this is likely the source of most superstition. https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 While there is no scientific evidence that LLMs can reach AGI, they will still be practically useful for many other tasks. A human mind paired with an LLM is a powerful combination.
- mirker 3y ago
  Agreed. Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making. Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical? Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.
  - ux-app 3y ago
    >Could you really be confident you understood how it worked by just running experiments on it as a black box the human brain is a black box, we can certainly learn a lot about it by prodding and poking it. >Things look like magic especially when you can’t look under the hood. imagine we had a 100% complete understanding of the mechanical/chemical/electrical functioning of the human brain. Would knowing the magic make it any less magical? in some sense, yes (the mystique would be gone, bye bye dualism), but in a practical sense, not really. It's still an astonishingly useful piece of grey matter.
- ux-app 3y ago
  >What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness I'm not saying that you're wrong, but... you'd have to provide a more rigorous rebuttal to be taken seriously. AGI can exist without sapience and intelligence is a continuum. you can't just hand wave away GPT's capabilities which is why the sharpest minds on the planet are poking this new machine to work out wtf is going on. human intelligence is a black box. we judge it by its outputs from given inputs. GPT is already producing human-like outputs. a common rebuttal is: "but it doesn't *really* think/understand/feel", to which my response is: ...and? ¯\_(ツ)_/¯ what does that even mean?

defgeneric 3y ago

> Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. This was basically the strategy of the OpenAI team if I understand them correctly. Most researchers in the field looked down on LLMs and it was a big surprise when they turned out to perform so well. It also seems to be the reason the big players are playing catch up right now.

whimsicalism 3y ago
I think it was a surprise the behaviors that were unlocked at different perplexity levels, but I don't really agree that LLMs were "looked down on."
- TeMPOraL 3y ago
  Maybe not "looked down on", but more of "looked at as a promising avenue". I mean, 2-3 years ago, it felt LLMs are going to be nice storytellers at best. These days, we're wondering just how much of the overall process of "understanding" and "reasoning" can be reduced to adjacency search in sufficiently absurdly high-dimensional vector space.
  - whimsicalism 3y ago
    People certainly knew that language modeling was a key unsupervised objective to unlock inference on language. I agree that I think they underestimated quite how useful a product could be built around just the language modeling objective, but it's still been critical for most NLP advances of the last ~6+ years.

nradov 3y ago

I read that pre-print Microsoft paper. Despite the title, it doesn't actually show any real "sparks" of AGI (in the sense of something that could eventually pass a rigorous Turing test). What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness; our brains seem to be wired that way and this is likely the source of most superstition. https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 While there is no scientific evidence that LLMs can reach AGI, they will still be practically useful for many other tasks. A human mind paired with an LLM is a powerful combination.

mirker 3y ago
Agreed. Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making. Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical? Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.
- ux-app 3y ago
  >Could you really be confident you understood how it worked by just running experiments on it as a black box the human brain is a black box, we can certainly learn a lot about it by prodding and poking it. >Things look like magic especially when you can’t look under the hood. imagine we had a 100% complete understanding of the mechanical/chemical/electrical functioning of the human brain. Would knowing the magic make it any less magical? in some sense, yes (the mystique would be gone, bye bye dualism), but in a practical sense, not really. It's still an astonishingly useful piece of grey matter.
ux-app 3y ago
>What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness I'm not saying that you're wrong, but... you'd have to provide a more rigorous rebuttal to be taken seriously. AGI can exist without sapience and intelligence is a continuum. you can't just hand wave away GPT's capabilities which is why the sharpest minds on the planet are poking this new machine to work out wtf is going on. human intelligence is a black box. we judge it by its outputs from given inputs. GPT is already producing human-like outputs. a common rebuttal is: "but it doesn't *really* think/understand/feel", to which my response is: ...and? ¯\_(ツ)_/¯ what does that even mean?