9 ms·

> When I say understanding I mean the meaning and context around it. Yes it is actually modelling the meaning and context around it. ChatGPT is a ~100 layer de

> When I say understanding I mean the meaning and context around it. Yes it is actually modelling the meaning and context around it. ChatGPT is a ~100 layer deep neural network that was trained specifically to solve "natural language understanding tasks". The term "understanding" is an important term in NLP research along with the concept of "semantic similarity". I believe it is more powerful and ultimately (eventually) more dangerous than you fear it is.

_8j50 3y ago
Based on what I have see it is making a really good approximation. Here is a thought problem: if memorized every single result of of a multiplication between two numbers up to the maximum a human can possibly think of, then do I understand multiplication? My point about mistakes is not that they were made but the way they were made indicates it was attempting to approximate. Someone mentioned a famous mathematician who got basic multiplication wrong, now if you are wrong because you missed some steps in the process that can be proven. But if your mistake is because you guessed the that can also proven by showing defects in your answer were arbitrary. Pretend you're grading a student, you can tell when they guessed and got it wrong as opposed to tried to follow the process bur misunderstood something or make a critical error.
- Mike_12345 3y ago
  > if memorized every single result of of a multiplication between two numbers up to the maximum a human can possibly think of, then do I understand multiplication? No, ChatGPT is not simply regurgitating. It actually understands the data. It is one of the largest deep neural networks ever constructed. Consider AlphaZero, the neural net that learned how to play Go and became the world champion. It started with nothing but the rules of the game. Deep Learning systems build a generalized model and reason about it. They are not simply regurgitating what they have already seen in their training set.

_8j50 3y ago

Based on what I have see it is making a really good approximation. Here is a thought problem: if memorized every single result of of a multiplication between two numbers up to the maximum a human can possibly think of, then do I understand multiplication? My point about mistakes is not that they were made but the way they were made indicates it was attempting to approximate. Someone mentioned a famous mathematician who got basic multiplication wrong, now if you are wrong because you missed some steps in the process that can be proven. But if your mistake is because you guessed the that can also proven by showing defects in your answer were arbitrary. Pretend you're grading a student, you can tell when they guessed and got it wrong as opposed to tried to follow the process bur misunderstood something or make a critical error.

Mike_12345 3y ago
> if memorized every single result of of a multiplication between two numbers up to the maximum a human can possibly think of, then do I understand multiplication? No, ChatGPT is not simply regurgitating. It actually understands the data. It is one of the largest deep neural networks ever constructed. Consider AlphaZero, the neural net that learned how to play Go and became the world champion. It started with nothing but the rules of the game. Deep Learning systems build a generalized model and reason about it. They are not simply regurgitating what they have already seen in their training set.