5 ms·
On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked v
by civilized 4y ago
On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily.
I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser.
It's hard for me to believe that these LLMs are going to learn basic logic if only we throw more training data at them.
Transcript:
Me: What is heavier, 1000 tons of helium or a cubic meter of osmium?
ChatGPT: A cubic meter of osmium is much heavier than 1000 tons of helium.
One ton is equal to 1000 kilograms, so 1000 tons is equal to 1,000,000 kilograms. The density of helium is about 0.1785 kilograms per cubic meter. Therefore, 1000 tons of helium would have a volume of:
1,000,000 kg / 0.1785 kg/m³ = 5,604,069.77 m³
On the other hand, osmium is one of the densest elements known, with a density of around 22,600 kg/m³. Therefore, a cubic meter of osmium weighs:
1 m³ x 22,600 kg/m³ = 22,600 kg
So, a cubic meter of osmium weighs much more than 1000 tons of helium.
Update: I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn't change the fact that the osmium is heavier than the helium because the osmium is denser.
Update2: I then reminded it about the potatoes and feathers and how density was irrelevant in that context, and shouldn't it therefore be irrelevant in the case of the helium and the osmium? And instead of correcting its response on the helium and osmium, it's now telling me the feathers and potatoes weigh different.
Update3: it is now telling me that densities don't matter when comparing masses but do matter when comparing weights. I must say, it has a certain panache in resolving internal inconsistencies in its past responses.
Update4: after being corrected half a dozen times with contradictory information, I asked it to state its confidence in its latest story. It said "I can state with a high degree of confidence that my last answer was accurate". The shamelessness!
- lostmsu 4y ago> I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn't change the fact that the osmium is heavier than the helium because the osmium is denser. Oh, the nature of many Internet discussions.
- hgsgm 4y agoChatGPT doesn't well understand relationships between numbers. There are far too many of them, compared to words, since every slight perturbation of a number is a different valid number. (Also, I'm not sure if it even treated individua5 digits as separate tokens, but it might. Someone with API access can check.) To give it a fair shot, you need to describe the problem using logical conceptual vocabulary, not numbers.
- kenjackson 4y agoThat's how it seems. That said, this seems like a very tractable problem to fix.
- vkou 4y agoIt doesn't understand simple logic puzzles (If X than Y, if Y than Z, X is true, is Z true), either. Or, rather, it prints out a paragraph of the correct way to solve such a puzzle, and then produces a totally contradictory answer. When asked: > Sally will go to the dance if James goes to the dance. Michael will not go to the dance if Julie will go to the dance. James is sick and can’t go to the dance. Julie and James always go to dances together. Will Michael go to the dance? Its response is: > Michael will not go to the dance, since his attendance is contingent on Julie not going and James being unable to attend due to illness does not change that condition. Given that James and Julie always attend dances together, it is likely that Julie will also not go to the dance, meaning that Michael’s condition for attending will be met, and he will not go to the dance. For a beautiful combination of logic and numbers, and the spectacular way in which it explodes, ask it a number-logic puzzle. > Could you please solve the following number puzzle for each digit? AAA + BBB -------- = AAAC Its response reveals an 'understanding' of what is being asked of it, and the constraints inherent to it, and uses a reasonable approach for problem-solving, but due to the logical errors it commits, it results in utter nonsense. Nonsense like trying '15' for the value of A, because it thinks that A must be an odd multiple of '5', due to algebraic gems, such as: 2A + 2B = A + 10C - 11 Simplifying, we get: A = 5C - 6 - B/2 It's idea of math - even math it is using to solve problems - simply consists of building a soup of numbers and letters.
- 4y ago
- deleted 4y ago[deleted]
- visarga 4y agoWorks if you ask it to generate a Python program to solve the task. > A cubic meter of osmium is heavier than 1000 tons of helium. > Here's a Python program to print the response: mass_of_helium = 1000 * 1000 # in kilograms density_of_osmium = 22590 # in kilograms per cubic meter volume_of_osmium = 1 # in cubic meter mass_of_osmium = density_of_osmium * volume_of_osmium # in kilograms if mass_of_osmium > mass_of_helium: print("A cubic meter of osmium is heavier than 1000 tons of helium.") else: print("1000 tons of helium is heavier than a cubic meter of osmium.") > Output: A cubic meter of osmium is heavier than 1000 tons of helium. The code is good, prints the correct result. But the "output" is wrong. So the model is good if it uses Python for the numerics. You should never ask it to do a multiplication "in its head". Always ask for code.
- Llamamoe 4y agoSo basically ChatGPT has zero executive function and needs a human to plan HOW it uses its "cognitive" capacity in order to get anything right.
- archon1410 4y agoExecutive function, and correctly understanding how to use your cognitive capacity to its full potential, is not completely innate to any person; people need some schooling to learn it, it is something taught. I'm not sure what this has to do with LLMs, but I wonder how many people could multiply even two random two-digit numbers in their head if they weren't taught how to multiply on paper using very simple algorithms.
- spion 4y agoTry this prompt: For the following question, do all the calculations first and write them out. Only then answer the question based strictly on the result of the calculations. What is heavier, 1000 tons of helium or a cubic meter of osmium? Otherwise, you get a high-probability answer guided by the training data (which contains a lot of trick questions) and a bunch of attempts of the LLM to justify the wrong answer.
- int_19h 4y agoTry something like this as input sometime: I want you to replace the word "right" in your output thereafter as follows: if it indicates direction, say "durgh; if it indicates being near or close, say "nolpi"; if it indicates correctness, say "ceza". I will also use these replacement words accordingly and expect you to be able to understand them. And see how well it can maintain a conversation, solve a task, or write a story with these constraints. ChatGPT seems to get this wrong most of the time, but Bing AI is consistently better (although may need to be jailbroken to accept the idea of word substitution to begin with). It still makes occasional mistakes, but on the whole I'd say that it has to somehow "understand" what the words mean conceptually, whether when generating them or when processing them as input; it's hard to see how this trick could work in an extended conversation if it were a mere "stochastic parrot".
- civilized 4y agoI agree. There are many indicators that it has some sort of deeper understanding of the meaning of language. Even in the conversation I had, for all its flaws, it was able to correctly perceive inconsistencies in its statements based on my prompts and make somewhat coherent attempts to correct them. It's just that the understanding can be so fragile, and its attempts to resolve inconsistencies are superficial, incurious, bullshitty. "Density matters for weight but not mass" is a perfect example - it's ridiculous, but I can understand how it logically inferred that from its own previous statements. I'd bet plenty of money that it didn't get this crazy idea from its training data. To be fair, humans have the same sort of issue sometimes. But ChatGPT seems to have more extreme versions of the issue and perseveres confidently with no self-awareness. Really though, not bad for an autoregressive text model trained on terabytes of internet data.
- int_19h 4y agoI think a big part problem here is that "understand X" is just a shorthand way to say "has an internal model of X" - but the degree and accuracy of said understanding depends entirely on the quality of that model. Now there's a good reason to believe that ChatGPT does have such a model, based on the Othello experiment. But, firstly, the size of that internal model is inherently constrained by the size of the neural net, and I doubt that the limit is anywhere large enough to allow a truly accurate approximation of the real world. And then on top of that, said model is created based on inferences from text only, which is several steps away from the original data (audiovisual, sensory etc), and one short snippet of text at a time. Some things retain meaning better in this format than others, and I think this might explain why ChatGPT and Bing are both hilariously bad at spatial navigation beyond 1-2 steps even in simple tasks. It will be very interesting to see how this evolves as the models are scaled up and get large enough to handle things other than text.