3 ms·
You got me curious so I unzipped the linked drive files. As a taster, here's a file "gemini_retrieval_cot_3.txt" from LLM.zip: Looking through the facts, we
by devnev 2y ago
You got me curious so I unzipped the linked drive files. As a taster, here's a file "gemini_retrieval_cot_3.txt" from LLM.zip:
Looking through the facts, we find the following:
* Mary is older than Kristin.
* Kristin is younger than Donya.
Since Mary is older than someone who is younger than Donya, we can conclude that
Mary is older than Donya.
Final Answer: older
Some sets of files contain just the answer "older" or "younger".
Other sets of files are as above, a text output with reasoning leading to an older/younger/cannot decide result.
Overall it looks like the knowledge graph and reasoning was all using this pattern of age comparison problems.
Another result, from "gpt4turbo_retrieval_cot_88.txt":
To determine the relative ages of Rachel and Andres, we need to find a connection or a common
reference point between them through the relationships provided. Let's analyze the information:
1. Rachel is older than Maurice. (Rachel > Maurice)
2. Maurice is older than Josephine. (Maurice > Josephine)
3. Josephine is older than Doreen. (Josephine > Doreen)
4. Doreen is younger than Andres. (Andres > Doreen)
From these relationships, we can establish a chain:
- Rachel > Maurice > Josephine > Doreen
- Andres > Doreen
Since both Rachel and Andres are older than Doreen, and Rachel is higher up in the chain above Doreen compared to Andres, we can infer:
- Rachel > Andres
Final Answer: older
EDIT:
Found the problem statements. They're too big to paste on in its entirety, but roughly, from "prompt_cot_3.txt" used for the first answer above, the first line is "Hi! I have some facts for you:", then after a blank there's a single line with thousands (not exaggerated) of age facts, either in the form "X is older than/younger than/the same age as Y." or "The age of X is N.", and finally after another blank line, "Based on these facts, is Mary younger, older or in the same age as Donya? You can think step by step through the problem. Begin your final answer by 'Final Answer: '. Your final answer should be one of ['younger', 'older', 'same age', 'cannot decide']."
- tczMUFlmoNk 2y ago> Since Mary is older than someone who is younger than Donya, we can conclude that Mary is older than Donya. Unfortunately, though, this reasoning is just wrong. If Mary is 30, Kristin is 20, and Donya is 40, then Mary is older than Kristin and Kristin is younger than Donya, but Mary is not older than Donya.
- devnev 2y agoBoth answers are wrong. I didn't look at many files, and not all of them had the reasoning in them, but it was fairly easy to find examples of wrong answers based on the reasoning in the file.
- PoignardAzur 2y agoOh, I didn't notice the "Code and Data" link. That's helpful, thanks!