3 ms·
We can assess this using verbal, logical and arithmetic reasoning tests. I gave ChatGPT 5 questions from a verbal reasoning test on sequence of words and it go
by HybridCurve 4y ago
We can assess this using verbal, logical and arithmetic reasoning tests.
I gave ChatGPT 5 questions from a verbal reasoning test on sequence of words and it got 2/5 right.
Here is sample question:
Arrange the words given below in a meaningful sequence.
1. Poverty 2. Population 3. Death
4. Unemployment 5. Disease
A) 2, 3, 4, 5, 1
B) 3, 4, 2, 5, 1
C) 2, 4, 1, 5, 3
D) 1, 2, 3, 4, 5
ChatGPT> B) 3, 4, 2, 5, 1
Unemployment can lead to poverty, which can lead to disease. Disease can lead to death. The population of an area is affected by all of these factors. Therefore, the correct order is 3, 4, 2, 5, 1.
Correct answer was C
I would expect if it was effective at reasoning it should be strongest on verbal associations. And while it is possible that enhancements to future language models might provided them with better reasoning ability I don't believe you can simply push the existing design to that limit. The poor results from the verbal reasoning tests demonstrate that reasoning is more complicated than what a transformer based language model is capable of.
Overall, the argument is moot because the statement made in the article was it was a reasoning engine which is not equivalent to being capable of reasoning.