4 ms·

It's missing generality. It can't do basic maths, or even follow instructions. It can't make valid moves in a game. It can only do what it was trained to do: ma

It's missing generality. It can't do basic maths, or even follow instructions. It can't make valid moves in a game. It can only do what it was trained to do: make a good guess at what word should come next. I don't mean to down-play how impressive models like GPT-4 are - they are very impressive and have great utility. The problem is we are talking about PR, and when you throw around words like "artificial intelligence" it misleads everyone who is not intimately familiar with the limitations of such models. > Personally I believe it's possible to have intelligence without sentience/self-awareness The general public doesn't make that distinction when you throw around terms like artificial intelligence. They hear artificial intelligence and imagine "thinking machines". No doubt people will say this is just moving the goalposts - that whatever these models achieve it will never be "AI", but that's demonstrably false. The concept of AI has existed in film and other media in the same form since forever, and it's pretty clear if you put those representations side by side with what we actually have today, that we are not even close yet. For some reason "advances in machine learning" just isn't hip enough to get funding, so "AI" gets thrown around instead.

Closi 3y ago
> It can only do what it was trained to do: make a good guess at what word should come next. Disagree - GPT's mechanics are that it guesses what word/token should come next, however it is now displaying emergent capabilities which do show an agent which can generalize. Maybe not to 'AGI' levels, but it is starting to show little signs of it. > It can't do basic maths, or even follow instructions. It can't make valid moves in a game. It can do basic maths - better than most humans (but it cannot do maths better than a calculator). If I ask GPT4 to calculate 51 times 102 it will get the answer, despite most humans struggling with this task. It does this just via internal representation without any specific 'calculator' API call or functionality. It can follow instructions - if I tell it to take the sentence "abcd" and then tell it to follow the following steps: "1) swap b and c around, 2) reverse the string, 3) ignore step 1", it will output "dcba" and describe how it reached that output It can make valid game moves - Without being formally taught chess it will make valid chess moves most of the time (see: https://www.lesswrong.com/posts/xyjhFCSSXZsW6HDBb/a-chess-game-against-gpt-4 https://www.lesswrong.com/posts/xyjhFCSSXZsW6HDBb/a-chess-ga...). It made 29 valid moves before making invalid moves. Again, GPT4 has not been specifically trained to play chess, it just has picked up some chess rules and games while reading the internet and this has somehow formed some sort of internal representations for chess within it's model. I can also describe a new/novel game or puzzle to GPT4 and it can have a go at playing and solving it. What it can't do is learn at the moment, although learning isn't part of a lot of definitions of intelligence, which is why we have to define intelligence before you can say if it is/isn't intelligent.
- Davidzheng 3y ago
  Yeah the previous comment is almost completely wrong. But on last point, it didn't get here by hard-coding and the reinforcement learning part at the end i think shows it definitely can "learn". I guess one thing I would still point to is it has a weakened sense of "self" still I think, which goes inline with it distinguishing things learned in training vs in deployment (though maybe this is just intentional idk)
  - Closi 3y ago
    Yes I agree - I really meant it doesn't directly learn from your interaction (i.e. it forgets you spoke as soon as you close the session, and only learns indirectly through the next training run - your words do not affect it's internal representation in real time like humans). The learning appears to require more information than humans too, who can do 'more with less'. You are right that it clearly can learn though, the learnings are just baked-in with each cycle.
    - nullsense 3y ago
      I think the Microsoft Tay incident is reason enough that OpenAI know it would be a bad idea to structure things in such a way that the internet can wreck it in a manner of hours, but I certainly think they could make it learn in real-time if they wanted to. They are running RLHF at least with the current ChatGPT, but in theory maybe they could do something like each user gets their own LoRA and it uses their conversations to fine-tune a version specific to them. I suspect we'll start seeing ideas akin to that tried in the open source community.
- orhmeh09 3y ago
  It doesn't even "understand" words and what they are made of. This is an extremely simple task that a child could do, and yet: - Give me a list of US and European officials with names ending in "-land" 1. Mary Landrieu - former US Senator from Louisiana 2. Harold Landry - US Congressman from Tennessee 3. Jacky Rosen - US Senator from Nevada 4. Chris Van Hollen - US Senator from Maryland 5. Brian Mast - US Congressman from Florida 6. Roland Rudd - British businessman and political campaigner 7. Sadiq Khan - Mayor of London, England 8. Martin Schulz - former President of the European Parliament 9. Angela Merkel - Chancellor of Germany 10. Mark Rutte - Prime Minister of the Netherlands 11. Stefan Löfven - Prime Minister of Sweden 12. Erna Solberg - Prime Minister of Norway 13. Alexander Stubb - former Prime Minister of Finland 14. Kaja Kallas - Prime Minister of Estonia 15. Xavier Bettel - Prime Minister of Luxembourg.

Closi 3y ago

> It can only do what it was trained to do: make a good guess at what word should come next. Disagree - GPT's mechanics are that it guesses what word/token should come next, however it is now displaying emergent capabilities which do show an agent which can generalize. Maybe not to 'AGI' levels, but it is starting to show little signs of it. > It can't do basic maths, or even follow instructions. It can't make valid moves in a game. It can do basic maths - better than most humans (but it cannot do maths better than a calculator). If I ask GPT4 to calculate 51 times 102 it will get the answer, despite most humans struggling with this task. It does this just via internal representation without any specific 'calculator' API call or functionality. It can follow instructions - if I tell it to take the sentence "abcd" and then tell it to follow the following steps: "1) swap b and c around, 2) reverse the string, 3) ignore step 1", it will output "dcba" and describe how it reached that output It can make valid game moves - Without being formally taught chess it will make valid chess moves most of the time (see: https://www.lesswrong.com/posts/xyjhFCSSXZsW6HDBb/a-chess-game-against-gpt-4 https://www.lesswrong.com/posts/xyjhFCSSXZsW6HDBb/a-chess-ga...). It made 29 valid moves before making invalid moves. Again, GPT4 has not been specifically trained to play chess, it just has picked up some chess rules and games while reading the internet and this has somehow formed some sort of internal representations for chess within it's model. I can also describe a new/novel game or puzzle to GPT4 and it can have a go at playing and solving it. What it can't do is learn at the moment, although learning isn't part of a lot of definitions of intelligence, which is why we have to define intelligence before you can say if it is/isn't intelligent.

Davidzheng 3y ago
Yeah the previous comment is almost completely wrong. But on last point, it didn't get here by hard-coding and the reinforcement learning part at the end i think shows it definitely can "learn". I guess one thing I would still point to is it has a weakened sense of "self" still I think, which goes inline with it distinguishing things learned in training vs in deployment (though maybe this is just intentional idk)
- Closi 3y ago
  Yes I agree - I really meant it doesn't directly learn from your interaction (i.e. it forgets you spoke as soon as you close the session, and only learns indirectly through the next training run - your words do not affect it's internal representation in real time like humans). The learning appears to require more information than humans too, who can do 'more with less'. You are right that it clearly can learn though, the learnings are just baked-in with each cycle.
  - nullsense 3y ago
    I think the Microsoft Tay incident is reason enough that OpenAI know it would be a bad idea to structure things in such a way that the internet can wreck it in a manner of hours, but I certainly think they could make it learn in real-time if they wanted to. They are running RLHF at least with the current ChatGPT, but in theory maybe they could do something like each user gets their own LoRA and it uses their conversations to fine-tune a version specific to them. I suspect we'll start seeing ideas akin to that tried in the open source community.
orhmeh09 3y ago
It doesn't even "understand" words and what they are made of. This is an extremely simple task that a child could do, and yet: - Give me a list of US and European officials with names ending in "-land" 1. Mary Landrieu - former US Senator from Louisiana 2. Harold Landry - US Congressman from Tennessee 3. Jacky Rosen - US Senator from Nevada 4. Chris Van Hollen - US Senator from Maryland 5. Brian Mast - US Congressman from Florida 6. Roland Rudd - British businessman and political campaigner 7. Sadiq Khan - Mayor of London, England 8. Martin Schulz - former President of the European Parliament 9. Angela Merkel - Chancellor of Germany 10. Mark Rutte - Prime Minister of the Netherlands 11. Stefan Löfven - Prime Minister of Sweden 12. Erna Solberg - Prime Minister of Norway 13. Alexander Stubb - former Prime Minister of Finland 14. Kaja Kallas - Prime Minister of Estonia 15. Xavier Bettel - Prime Minister of Luxembourg.