5 ms·

> The capabilities of these models are very likely to be radically different five years from now. This is an article of faith. Specifically, it believes that

> The capabilities of these models are very likely to be radically different five years from now. This is an article of faith. Specifically, it believes that at some point in the future the current paradigm will result in qualitatively different behaviors than the mimicry these systems (all GPT variants, "Attention is All You Need") currently perform. Much ado has been made of the previous Symbolic AI researchers "moving the goalposts." In this criticism, it is the old guard of AI who is constantly bemoaning the current state of affairs as not real AI. But there is no actual goalpost moving. They have said it wasn't real AI at the beginning, and they are saying it isn't real AI now. Whether or not the symbolists' model was real AI is irrelevant: when you bake in "this is a rapidly moving field" as a hand-wavy explanation for why this may result in AGI, you are the one implying a moving of the goalposts. If it ever turns out that these models need to be qualitatively different, then it will be clear that attention is not in fact all you need. In that eventuality, I fully expect the new guard to hem and haw and find some tricky sophistry to explain why they were right all along, despite qualitative shifts unattributable to adding mountains of new data or connectionist trickery.

dinkumthinkum 4y ago
This is actually very accurate.
modeless 4y ago
> connectionist trickery I don't know, man, you kind of come off as having a chip on your shoulder about this. I'm not predicting AGI specifically here, and I'm not making any argument about symbolic vs connectionist AI at all. Maybe the model of the future is half symbolic! I'm just saying that asserting that you know exactly which things AI models aren't going to be able to do is pretty foolish at this point.
- abernard1 4y ago
  > know exactly what these models aren't going to be able to do is pretty foolish at this point This can be said about absolutely any new technology but it does not make it true. It's simply the inscrutability of these tensors that allow people to imagine the intelligence is in there somewhere. The original comment was what humans "bring to the table" over ChatGPT specifically. And that's that they have real intelligence and not memorization. As others have said, these models have been around for many years. Their core innovation is to add more and more data to them like a dictionary and compress those basis functions in the network architecture. This memorization has a limit and is actually the opposite of intelligence. Intelligence or creativity can do more with less information. As per the original comment, this is what human intelligence and creativity is (currently) superior at and what people should prioritize if they don't want to be replaced.

dinkumthinkum 4y ago

This is actually very accurate.

modeless 4y ago

> connectionist trickery I don't know, man, you kind of come off as having a chip on your shoulder about this. I'm not predicting AGI specifically here, and I'm not making any argument about symbolic vs connectionist AI at all. Maybe the model of the future is half symbolic! I'm just saying that asserting that you know exactly which things AI models aren't going to be able to do is pretty foolish at this point.

abernard1 4y ago
> know exactly what these models aren't going to be able to do is pretty foolish at this point This can be said about absolutely any new technology but it does not make it true. It's simply the inscrutability of these tensors that allow people to imagine the intelligence is in there somewhere. The original comment was what humans "bring to the table" over ChatGPT specifically. And that's that they have real intelligence and not memorization. As others have said, these models have been around for many years. Their core innovation is to add more and more data to them like a dictionary and compress those basis functions in the network architecture. This memorization has a limit and is actually the opposite of intelligence. Intelligence or creativity can do more with less information. As per the original comment, this is what human intelligence and creativity is (currently) superior at and what people should prioritize if they don't want to be replaced.