4 ms·

A human doing the same SATs will test roughly the same way every time. ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, wi

A human doing the same SATs will test roughly the same way every time. ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, will get 45% the following run etc. Stochastic randomisation makes the chatbot useful and provides these moments of impressive capability, it also means unlike humans that capability is in no way consistent.

pooper 4y ago
> A human doing the same SATs will test roughly the same way every time. > ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, will get 45% the following run etc. While I agree with your conclusion, I want to slightly modify the first sentence. I know this is just one anecdote but I took the SAT 1 twice and did somewhat better the second time. I think this is common as you are more familiar with the test the second time.
- hattmall 4y ago
  I don't think that make sense at all. Why would a human test the same multiple times? Everyone I know took the SAT multiple times with increasing results. I don't know of anyone that's achieved a perfect score with only one try and most people could get a perfect score with enough studying and attempts. My SAT score ranged from 980 to 1450 with 4 tries, increasing each time and had there been a reason or benefit I'm certain I could have eventually gotten a perfect score. I don't see how anyone could not get better over time.
hgsgm 4y ago
ChatGPT isn't a human, it's a mix of data sampled from many humans. Random humans are often bad at SAT.
- thomastjeffery 4y ago
  It isn't trained on the data from a random set of humans. It's trained on the human data that was chosen as more impressive, and trained to pull the most impressive parts out of that data. Fundamentally, while GPT may be "learned", it is not "logical". Instead of learning potential logical conclusions, it has learned potential semantic responses. It doesn't match the meaning of words to other meanings: it matchesa set of expressions to other expressions. The impressive part is that the logical conclusions already present in the training data get presented by GPT in a semantically coherent way.

pooper 4y ago

> A human doing the same SATs will test roughly the same way every time. > ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, will get 45% the following run etc. While I agree with your conclusion, I want to slightly modify the first sentence. I know this is just one anecdote but I took the SAT 1 twice and did somewhat better the second time. I think this is common as you are more familiar with the test the second time.

hattmall 4y ago
I don't think that make sense at all. Why would a human test the same multiple times? Everyone I know took the SAT multiple times with increasing results. I don't know of anyone that's achieved a perfect score with only one try and most people could get a perfect score with enough studying and attempts. My SAT score ranged from 980 to 1450 with 4 tries, increasing each time and had there been a reason or benefit I'm certain I could have eventually gotten a perfect score. I don't see how anyone could not get better over time.

hgsgm 4y ago

ChatGPT isn't a human, it's a mix of data sampled from many humans. Random humans are often bad at SAT.

thomastjeffery 4y ago
It isn't trained on the data from a random set of humans. It's trained on the human data that was chosen as more impressive, and trained to pull the most impressive parts out of that data. Fundamentally, while GPT may be "learned", it is not "logical". Instead of learning potential logical conclusions, it has learned potential semantic responses. It doesn't match the meaning of words to other meanings: it matchesa set of expressions to other expressions. The impressive part is that the logical conclusions already present in the training data get presented by GPT in a semantically coherent way.