12 ms·

LLMs, Theory of Mind, and Cheryl's Birthday

290 points by stereoabuse 2y ago

erwald 2y ago
o1 mini seems to get it on the first try (I didn't vet the code, but I tested it and it works on both examples provided in the notebook, `dates` and `gabe_dates`): from collections import defaultdict def find_cheryls_birthday(possible_dates): # Parse the dates into month and day dates = [date.split() for date in possible_dates] months = [month for month, day in dates] days = [day for month, day in dates] # Step 1: Albert knows the month and says he doesn't know the birthday # and that Bernard doesn't know either. This implies the month has no unique days. month_counts = defaultdict(int) day_counts = defaultdict(int) for month, day in dates: month_counts[month] += 1 day_counts[day] += 1 # Months with all days appearing more than once possible_months = [month for month in month_counts if all(day_counts[day] > 1 for m, day in dates if m == month)] filtered_dates = [date for date in dates if date[0] in possible_months] # Step 2: Bernard knows the day and now knows the birthday # This means the day is unique in the filtered dates filtered_days = defaultdict(int) for month, day in filtered_dates: filtered_days[day] += 1 possible_days = [day for day in filtered_days if filtered_days[day] == 1] filtered_dates = [date for date in filtered_dates if date[1] in possible_days] # Step 3: Albert now knows the birthday, so the month must be unique in remaining dates possible_months = defaultdict(int) for month, day in filtered_dates: possible_months[month] += 1 final_dates = [date for date in filtered_dates if possible_months[date[0]] == 1] # Convert back to original format return ' '.join(final_dates[0]) if final_dates else "No unique solution found." # Example usage: possible_dates = [ "May 15", "May 16", "May 19", "June 17", "June 18", "July 14", "July 16", "August 14", "August 15", "August 17" ] birthday = find_cheryls_birthday(possible_dates) print(f"Cheryl's Birthday is on {birthday}.")
- mewpmewp2 2y ago
  In addition to that after they create the 1st program with mistakes the author should have showed them the invalid output and let them have a chance to fix it. For humans solving this on the first try without running the code also tends to frequently not work.
- fragmede 2y ago
  "seems to" isn't good enough, especially since it's entirely possible to generate code that doesn't give the right answer. 4o is able to write some bad code, run it, recognize that it's bad, and then fix it, if you tell it to. https://chatgpt.com/share/670086ed-67bc-8009-b96c-39e539791fd3 https://chatgpt.com/share/670086ed-67bc-8009-b96c-39e539791f...
  - Chinjut 2y ago
    Did you actually run the "fixed" code here? Its output is an empty list, just like the pre-"fixed" code.
    - Chinjut 2y ago
      Hm, actually, it's confusing, because clicking the [>_] links where it mentions running code gives different code than it just mentioned.
- isaacfrond 2y ago
  despite the name ‘mini’. it is actually more optimized for code. so that makes sense.
joe_the_user 2y ago
Deducing things from the inability of an LLM to answer a specific question seemed doomed by the "it will be able to on the next itteration" principle. It seems like the only way you could systematic chart the weaknesses of an LLM is by having a class of problems that get harder for LLMs at a steep rate, so a small increase in problem complexity requires a significant increase in LLM power.
- godelski 2y ago
  > Deducing things from the inability of an LLM to answer a specific question seemed doomed by the "it will be able to on the next itteration" principle. That's orthogonal. If we are pointing in the right direction(s) then yes, next iteration could resolve all problems. If we are not pointing in the right direction(s) then no, next iteration will not resolve these problems. Given LLMs rapid improvement in regurgitating knowledge from their training data but simultaneously slow improvement in their ability to generalize (such as logic "puzzles"), I think it is naive to assume we're pointed in the right direction. Maybe we're even pointing in mostly the right direction. But why assume we are? We can continue in the direction we are going while simultaneously considering it might not be well aligned. If we are well aligned, that gives us more confidence and makes gathering funding easier. If we aren't, well it is easier to course correct sooner than later. In either case, you benefit from the analysis. Understanding why things fail is more important than understanding why things succeed.
  - Uehreka 2y ago
    GP is referring to the fact that if it becomes well known that LLM version X can’t solve problem Q, then the model’s trainers will make sure to include problem Q prominently in the training set, running it through over and over to ensure that version X+1 is able to solve the problem whether the model’s “reasoning” abilities have improved or not. Thus observers of the LLM space like us need to keep finding novel “Bellweather problems” that we think will evaluate a model’s ability to reason, knowing that once we start talking about it openly the problem will no longer be a useful Bellweather. By their nature as “weird-shaped” problems, these aren’t the kind of thing we’re guaranteed to have an infinite supply of. As the generations move on it will become more and more difficult to discern “actual improvements in reasoning” from “the model essentially has the solution to your particular riddle hard-coded”.
jfcoa 2y ago
This seems like a terrible test case since python examples are readily available in the training data: https://rosettacode.org/wiki/Cheryl%27s_birthday https://rosettacode.org/wiki/Cheryl%27s_birthday It's interesting that so many of the model's fail to retrieve this, but any thta do solve it should clearly be able to do so with no reasoning/theory of mind.
- rghall102 2y ago
  It is fascinating that the R solution just below the Python solution is much shorter and more readable. The same applies to Ruby and various Lisps. It even applies to the VisualBasic solution!
- kohlerm 2y ago
  I agree this is not a great test. What's good about it is that it is a constraint satisfaction problem, and I would expect LLMs to be pretty bad at unknown problems of this kind. Simple reason, an LLM only has a a finite number of layers and it cannot do arbitrary long searches.
  - johnisgood 2y ago
    I almost made ChatGPT write a Python program that creates a monthly work schedule (for imaginary workers) based on specific constraints (e.g. there are 10 workers, 2 shifts (morning and night), must work 40 hours per week, must have at least one weekend in a month off, 2 minimum workers per shift, no more than 3 consecutive working days, and so forth). I am not sure if I could make it give me a working solution, however, and I have not tried Claude, for example, and I have not tried to do it with other programming languages. Maybe. The issue was that it messed up the constraints and there were no feasible solutions, that said, it did give me a working program for this that had fewer constraints.
  - falcor84 2y ago
    I don't understand what you're saying - the idea is that we're asking the LLM to generate code to perform the search, rather than run an arbitrarily long search on its own, right? So why should the number of layers it has matter?
whack 2y ago
> At least with respect to this problem, they had no theory of mind. This is very interesting and insightful, but I take issue with the above conclusion. Your average software engineer would probably fail to code up a python solution to this problem. But most people would agree that the average software engineer, and the average person, possesses some theory of mind. This seems to be a pattern I'm noticing with AI. The goalposts keep moving. When I was a kid, the turing test was the holy grail for "artificial intelligence." Now, your run-of-the-mill LLM can breeze through the turing test. But no one seems to care. "They are just imitating us, that doesn't count." Every couple years, AI/ML systems make revolutionary advances, but everyone pretends it's not a big deal because of some new excuse. The latest one being "LLMs can't write a python program to solve an entire class of very challenging logic problems. Therefore LLMs possess no theory of mind." Let me stick my neck out and say something controversial. Are the latest LLMs as smart as Peter Norvig? No. Are they smarter than your average human? Yes. Can they outperform your average human at a randomly chosen cognitive task that has real-world applications? Yes. This is pretty darn revolutionary. We have crossed the rubicon. We are watching history unfold in real-time.
- Jerrrrrrry 2y ago
  The goalposts will continue to move until GDP improves.
  - DoctorOetker 2y ago
    until who's GDP moves? Suppose nation X or power bloc Y's GDP improves due to ML, will nation Z without increasing GDP continue to move the goalposts?
- titanomachy 2y ago
  I consider myself a pretty average human programmer, and I was able to solve the logic puzzle and write a python program for it in ~10 mins. [0] I agree though, the people who are unable to solve this probably still have a theory of mind. It seems like we're setting a rather high bar. [0] https://pastebin.com/q33K0HJ1 https://pastebin.com/q33K0HJ1
jawns 2y ago
A long time ago, I created a version of this challenge called "Cheryl's Murder." My notebook not only solves logical induction problems like "Cheryl's Birthday," but it also generates them. https://github.com/shaungallagher/cheryls-murder/blob/master/cheryls-murder.ipynb https://github.com/shaungallagher/cheryls-murder/blob/master...
- airstrike 2y ago
  This is awesome, thanks for sharing
- xelxebar 2y ago
  > Twice a year, all of the software engineers at our company are given several days to work on a project of their choosing. Monetate sounds like it has (had?) some interesting leadership!
godelski 2y ago
I think the test is better than many other commenters are giving credit. It reminds me of responses to the river crossing problems. The reason people do tests like this is because we know the answer a priori or can determine the answer. Reasoning tests are about generalization, and this means you have to be able to generalize based on the logic. So the author knows that the question is spoiled, because they know that the model was trained on wiki. They also tested to see if the model is familiar with the problem in the first place. In fact, you too can confirm this by asking "What is the logic puzzle, Cheryl's birthday?" and they will spit you out the correct answer. The problem also went viral, so there are even variations of this. That should tell us that the model has not just been trained on it, but that it has seen it in various forms and we know that this increases its ability to generalize and perform the task. So then we're left with reasoning. How do we understand reasoning? It is the logical steps. But we need to make sure that this is distinct from memorization. So throwing in twists (as people do in the river puzzles) is a way to distinguish memory from logic. That's where these models fail. People always complain that "oh, but humans can't do it." I refer to this as "proof by self-incompetence." (I also see it claimed when it isn't actually true) But not everybody reasons, and not all the time (trivial cases are when you're asleep or in a coma, but it also includes things like when you're hangry or just dumb). Humans are different from LLMs. LLMs are giving it 100%, every time. "Proof by self-incompetence" is an exact example of this, where the goal is to explain a prior belief. But fitting data is easy, explaining data is hard (von Neumann's Elephant). There's also a key part that many people are missing in the analysis. The models were explicitly asked to *generalize* the problem. I'll give some comments about letting them attempt to solve iteratively, but this is often very tricky. I see this with the river crossing puzzles frequently, where there is information leakage passed back to the algo. Asking a followup question like "are you sure" is actually a hint. You typically don't ask that question when it is correct. Though newer models will not always apologize for being wrong, when actually correct, when they are sufficiently trained on that problem. You'll find that in these situations if you run the same prompt (in new clean sessions) multiple times that the variance in the output is very low. Overall, a good way to catch LLMs in differentiating reasoning from memorization is getting them to show their work, the steps in between. It isn't uncommon for them to get the right answer but have wrong steps, even in math problems. This is always a clear demonstration of memorization rather than reasoning. It is literally the subtly that matters. I suspect that one of the difficulties in humans analyzing LLMs is that there is no other entity that is capable of performing such feats that does not also have a theory of mind and a world model. But a good analogy might be in facts that you know, but not understanding why they are "the answer." I'm sure there's many people who have memorized complexities for many sorting algos or leet code problems and couldn't derive the answer themselves. But I really don't understand why we *need* LLMs to reason? A dictionary memorizes things, and so does wikipedia. Their lack in ability to reason does not make them any less marvelous of inventions/tools. But maybe, if we're looking to create intelligent and thinking machines, it isn't as simple as scale. We love simple things, but few things are simple and correct (though far more things are simple and approximately correct).
nextworddev 2y ago
The majority of humans in flesh can't solve the problem - so we need alternate measures for judging theory of mind capabilities in LLMs
- Terr_ 2y ago
  What about the difference that the human knows what they don't know? In contrast, the LLM knows nothing, but confidently half regurgitates correlational text that it is seen before.
  - famouswaffles 2y ago
    As far as all the research on this goes, LLMs (internally) mostly know what they know, but incentivizing that information making it to output is difficult.
- godelski 2y ago
  > The majority of humans in flesh can't solve the problem This is a non sequitur. All humans have the capacity to reason, but not all humans reason. There's a big difference. Hell, even GPT-4o will tell you this. Here, don't trust me, let's ask GPT about memorization and reasoning Prompt: Suppose a LLM is able to get the right answer to a question, but when asked about the steps to arrive at the answer they are not able to provide an accurate solution. Does this tell us anything about a model's ability to reason or its ability to memorize? What about when it gets the right answer but the explanation would lead to a different answer than the one stated? Response: These scenarios highlight important aspects of a model's capabilities and limitations regarding reasoning and memorization. Let's break them down: 1. **LLM Gets the Right Answer but Can't Explain the Steps:** - **Memorization vs. Reasoning:** If a model provides a correct answer without accurately explaining the steps, it might indicate that the model is relying more on memorization rather than understanding or reasoning. The model could have seen similar questions in its training data and learned to associate certain questions with specific answers without truly understanding the underlying principles. - **Reasoning Limitations:** This situation suggests that the model may not have a robust ability to reason through the problem-solving process. It highlights a gap between recognizing patterns in data and applying logical reasoning to derive answers. 2. **LLM Provides Correct Answer but Incorrect Explanation:** - **Memorization of Answers:** This might suggest that the model has memorized the answer from similar examples in the training data but does not understand the reasoning process. It could be retrieving an answer that it "knows" is correct without understanding why. - **Inconsistent Reasoning Ability:** Giving a correct answer with an explanation that would lead to a different answer indicates inconsistencies in its reasoning ability. It may reflect that the model's internal heuristics for generating explanations are not aligned with the logic used to derive answers. In both cases, these issues highlight the challenges in AI related to understanding vs. pattern recognition. While LLMs are excellent at recognizing patterns and retrieving information, their ability to reason logically and consistently can be limited. This differentiation is crucial when evaluating the capabilities of AI models, particularly in contexts where understanding and accurate reasoning are essential.
extr 2y ago
this is an interesting problem but it’s more of a logic problem than a true test of theory of mind. when i think “theory of mind” i think being able to model an external agent with complete knowledge, incentives, and behavior. i would not doubt LLMs have something close to this for humans, almost by accident since they are trained on human outputs.
- emseetech 2y ago
  I would venture to guess that it has value in that we couldn't even begin to properly test a theory of mind if they can't yet reason through a logic puzzle. It would be like testing if a society could build a rocket ship when they don't know how to use tools.
  - extr 2y ago
    Children as young as 4 demonstrate theory of mind but would be unable to solve artificial logic problems like OP
    - emseetech 2y ago
      Fair point, but I'd say that I don't think of LLMs as anthropomorphic individuals and more like societies acting in statistical concert. Maybe this is a failure on my part and is the wrong approach.
  - zmgsabst 2y ago
    What animals have theory of mind?
    - sdenton4 2y ago
      Many, certainly many primates. Our ability to understand animal intelligence is limited by or ability to ask appropriate questions, so we tend to underestimate intelligence outside of the primate space. Here's an excellent book on this topic: https://www.goodreads.com/book/show/30231743-are-we-smart-enough-to-know-how-smart-animals-are https://www.goodreads.com/book/show/30231743-are-we-smart-en...
m3kw9 2y ago
could be an architectual issue with the LLMs because you need to juggle a lot of states just from one statement regarding a big problem. Sort of like if you ask it to write an app like facebook. It would give you a bunch of crap, which is worse.
gkfasdfasdf 2y ago
This question was posed to o1, it is able to reason through it - but now I wonder if that is because the model is already aware of the puzzle. https://x.com/d_feldman/status/1834313124058726894 https://x.com/d_feldman/status/1834313124058726894
- cdfuller 2y ago
  I think that could be likely. I just asked 4o "When is Cheryl's birthday?" without any other context and was given this reply Cheryl's birthday puzzle is a logic problem where Albert and Bernard are trying to figure out Cheryl's birthday based on certain clues. Cheryl provides them with ten possible dates: May 15, May 16, May 19, June 17, June 18, July 14, July 16, August 14, August 15, and August 17. Here’s the reasoning: 1. Albert knows the month and Bernard knows the day. 2. Albert says he knows Cheryl’s birthday, meaning May and June can be eliminated because they contain unique days (May 19 and June 18). If Albert had been told May or June, he wouldn’t know for sure. 3. Bernard, knowing this, says he now knows Cheryl’s birthday. This eliminates the remaining dates with unique days (July 14 and August 14). 4. Albert then confirms that he also knows the birthday, meaning Cheryl’s birthday must be in July or August, but on a date with no unique days left: July 16, August 15, or August 17. Thus, Cheryl's birthday is *July 16*.
aithrowawaycomm 2y ago
AI researchers need to learn what terms like "theory of mind" actually mean before they write dumb crap like this. Theory of mind is about attributing mental states to others, not information. What Norvig has done here is present a logic puzzle, one that works equally well when the agents are Prolog programs instead of clever children. There's no "mind" in this puzzle at all. Norvig is being childishly ignorant to call this "theory of mind." It's hard to overstate my contempt for this kind of useless junk science, especially when it comes from an impressive pedigree. Of course he is hardly the only offender: arrogant disregard for psychology is astonishingly common among LLM researchers. Maybe they should turn off ChatGPT and read a book.
- deleted 2y ago
  [deleted]
- AdieuToLogic 2y ago
  > AI researchers need to learn what terms like "theory of mind" actually mean before they write dumb crap like this. > Of course he is hardly the only offender: arrogant disregard for psychology is astonishingly common among LLM researchers. Maybe they should turn off ChatGPT and read a book. Perhaps this[0] book? It is one of the seminal texts on AI topics. Maybe Norvig should read it. 0 - http://aima.cs.berkeley.edu/ http://aima.cs.berkeley.edu/
  - deleted 2y ago
    [deleted]
- sitkack 2y ago
  Hell, Norvig could have asked any of the LLMs if this was a good theory of mind test. CS researchers do partake in some pretty low quality science. My fav is AI researchers arrogantly rediscovering learning theory and pedagogy.
- immibis 2y ago
  People who fail theory of mind may assume everyone else knows something because they know it.
ynniv 2y ago
The problem with evaluating LLMs is that there's a random component, and the specific wording of prompts is so important. I asked Claude to explain the problem, then write python to solve it. When it ran there was an exception, so I pasted that back in and got the correct answer. I'm not sure what this says about theory of mind (the first script it wrote was organized into steps based on who knew what when, so it seems to grok that), but the real lesson is that if LLMs are an emulation of "human" intelligence, they should probably be given a python interpreter to check their work.
- skybrian 2y ago
  Yes, that helps. But if you iterate on this a few times (as I did last year with Code Interpreter), it reveals how much LLM's "like" to imitate patterns. Sure, often it will pattern-match on a useful fix and that's pretty neat. But after I told it "that fix didn't work" a couple times (with details about the error), it started assuming the fix wouldn't work and immediately trying again without my input. It learned the pattern! So, I learned to instead edit the question and resubmit. LLM's are pattern-imitating machines with a random number generator added to try to keep them from repeating the same pattern, which is what they really "want" to do. It's a brilliant hack because repeating the same pattern when it's not appropriate is a dead giveaway of machine-like behavior. (And adding a random number generator also makes it that much harder to evaluate LLM's since you need to repeat your queries and do statistics.) Although zero-shot question-answering often works, a more reliable way to get useful results out of an LLM is to "lean into it" by giving it a pattern and asking it to repeat it. (Or if you don't want it to follow a pattern, make sure you don't give it one that will confuse it.)
  - brain5ide 2y ago
    If I understood correctly, that anectode in first paragraph looks like an interaction with a child who is trying something but lacks confidence.
    - skybrian 2y ago
      It did look that way and it's a fun way to interpret it, but pattern-matching on a pretty obvious pattern in the text (several failed fixes in a row) seems more likely. LLM's will repeat patterns in other circumstances too.
deleted 2y ago
[deleted]
pfisherman 2y ago
LLMs and NLP are to verbal reasoning what the calculator is to quantitative reasoning. Language and by extension verbal reasoning is full of ambiguity and semantic slipperiness. For example, what degree of semantic similarity distinguishes synonymous from synonym-ish concepts? When do we partition concepts into homonyms? I think part of the problem with how people evaluate LLMs is that the expectations that people have. Natural language != ontology. The expectation should be more Chomsky and less Boole. Asking it to solve math problems written in paragraph form is a waste of time. Use a calculator for that! Solving riddles? Code it up in prolog! Instead you should be thinking of what operations you can do on concepts, meaning, and abstract ideas! That is what these things do.
- missingrib 2y ago
  Is this really verbal reasoning? It's just a logic problem.
  - pfisherman 2y ago
    How can one / should one combine the concepts of a dinosaur and monetary policy of the Ottoman Empire? What differentiates verbal reasoning from logic? I don’t know that either of those can be solved well with formal languages or logic.
    - samrus 2y ago
      Yeah but the Cheryl's birthday problem doesn't have any ambiguity like that. It's all in very simple language, the only complexity is keeping track of states of mind, which is easy to abstract away from the language
      - pfisherman 2y ago
        That is exactly the point I was making in my comment above. This type of unambiguous problem is best solved using formal languages - something more like quantitative reasoning. But stuff like prolog or classical automated reasoning approaches are quite brittle. They break down quickly when you start to introduce ambiguity and noise. Statistical approaches like hidden markov models that people used in these instances were the precursor to the LLMs we have today. But I was going down a rabbit hole there. My main point is that trying to use LLMs to solve logic puzzles - that can easily be solved in prolog - is a waste of time and a failure of the imagination. The applications that should be explored and would be most fruitful are those where there is ambiguity and contradiction.
mrbungie 2y ago
Not really about Theory of Mind, but in the same line, I remember the other day someone argued with me that LLMs model the world, rather than just modelling language (that may represent the world). I kept thinking about that problem and plausible experiments to show my point that LLMs are dumb about the physical world, even if they know perfectly how it works in terms of language/representation. So I thought, what happens if I give an LLM an image and I ask a representation of said image in ASCII art (obviously without relying in Python and the trivial pixel intensity to character transform it usually proposes). Remember: - LLMs should've been trained with a lot of RGB image training data with associated captions => So it should understand images very well. - LLMs should've been trained with a lot of ASCII training data with associated captions => So it should draw/write ASCII like an expert. Plus, it understands vision apparently (managed as tokens), so it should do well. But it can't do a decent translation that captures the most interesting features of an image into ASCII art (I'm pretty sure a human with an hour of time should be able to do it, even if its awful ASCII art). For example, I uploaded an image macro meme with text and two pictures of different persons kind of looking at each other. The ASCII art representation just showed two faces, that didn't look at each other but rather away from each other. It just does not "understand" the concept of crossing sights (even if it "understands" the language and even image patches when you ask about where are they looking at, it will not draw that humanly important stuff by itself). These things just work with tokens, and that is useful and seems like magic in a lot of domains. But there is no way in hell we are going to get into AGI without a fully integrated sensor platform that can model the world in its totality including interacting with it (i.e. like humans in training, but not necessarily in substrate nor training time hopefully). And I really don't know how something that has a very partial model of the world can have a Theory of Mind.
- Jerrrrrrry 2y ago
  it should draw/write ASCII like an expert. Not a lot of conversations incrementally totaling ASCII conversations in the training data - you are essentially asking a gold fish to climb a tree. It should have a lot of RGB image training data with associated captions => So it should understand images very well. you seem to have conflated the architectures. ChatGPT was trained on text, and text-image embedding - it can recognize, but cannot project. Thats the DALL-E portion - it leverages a similar transformer arch but they are not the same model nor architecture. However, ask a Generative Adversarial Network for ASCII, you'll get what you expect. Absent the infra-word character cohesion that LLM's token-ization provides, it will give realistic, if sometimes "uncanny" images - ones that "make sense" sequentially, or in the short term, but not the longer, or larger context. The language portion of your brain, that works faster than you do - else you would be at a loss of words constantly - is not nearly as equipped to deal with spatial problems that your posterior parietal cortex is. Ultimately we are converging towards a Mixture-of-Experts model that we will one day realize is just....us, but better.
AdieuToLogic 2y ago
What is a software program? The codification of a solution. What is a solution? An answer to a problem. What is a problem? The identification and expression of a need to be satisfied. What is a need? A uniquely human experience, one which only exists within the minds of people whom experience it.
- zmgsabst 2y ago
  Do animals not have needs?
  - AdieuToLogic 2y ago
    > Do animals not have needs? We are animals as well, so a pedantic answer would be "yes." :-) My actual answer is the origin of the thought process began by Norvig stating: But none of them were able to write a program that finds the solution. So while the concept of "need" is shared across many entities and widely applicable to many conversations, in this context it serves as a root cause answering "why write a program."
dmead 2y ago
I wonder if they are any unique properties of those programs that we can figure out what stack overflow or textbooks they're copying.
IanCal 2y ago
I'm not a huge fan of using these kind of riddles or gotchas. Other comments have riddle variants which feel also like ways of tripping someone up - if you don't spot the change you fail. And what's more the originals are things that lots of people struggle with (that's why they're riddles not basic questions). There's also little prompting, which feels like calling up a random person and demanding they solve a riddle straight away without talking it through. Part of the assumption here is that if llms don't write the right code they don't understand what people know. I'd wager that a huge number of people couldn't solve this puzzle yet fully understand what different people have their own internal thoughts and knowledge.
- rtlacn 2y ago
  Humans can solve these riddles even if they have never seen a similar one. LLMs need to have seen at least a similar riddle with the same structure and/or code to plagiarize. Humans can deal with modified riddles if they are alert and awake. An LLM is always alert and awake, so it should be able to handle modified riddles, but it cannot.
  - mirekrusin 2y ago
    Not to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.
    - samrus 2y ago
      Yeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit
      - Terretta 2y ago
        > An LLM can't be excused... LLMs are always operating at their best Depends how much you turn down the heat.
oli5679 2y ago
Gp1-o1 preview solves this puzzle correctly in 13 seconds and has a thorough logical deduction in the comments and explanation. I think it’s a bit unfair on llm to ask it to retrieve the puzzle definition from its training data. I posted the info on the puzzle from his notebook. https://chatgpt.com/share/670103ae-1c18-8011-8068-dd217937278a https://chatgpt.com/share/670103ae-1c18-8011-8068-dd21793727...
- godelski 2y ago
  The question is to get it to write generic code
- lagmg05 2y ago
  The question is if it solved the puzzle correctly before Norvig's article appeared. It could have been trained (I am told that existing models can be modified and augmented in any Llama discussion) on the article or on HN comments. There could even be an added routine that special cases trick questions and high profile criticisms.
  - oli5679 2y ago
    Sure, I guess the best way to test this is to compose a new question, of a similar format.
    - johnisgood 2y ago
      I am not sure "of a similar format" suffices here, it should not have any resemblance or similarity to this new question or riddle.
  - Terretta 2y ago
    > The question is if it solved the puzzle correctly before Norvig's article appeared. It could have been trained... This caught me by surprise — is there a suggestion or evidence that despite the "knowledge cutoff" OpenAI is continuously retraining GPT-4o's chat-backing model(s) on day over day updates to the web?
  - 2y ago
willguest 2y ago
This seems to amount to asking an LLM how it feels about Cheryl, discovering that it is performatively happy about her existence, and then deducing that the LLM has no capacity for genuine emotion, expressed in the form of logic. The faulty premise lies in the formulation of the test and makes the responses both predictable, but also does a disservice to 'mind' because it tries to interpret it in such a way that an LLM could begin to grapple with the basics, but not in a meaninful way. Perhaps it is useful to help build better context-specific logic flows (generally known as software) but it doesn't seem to provide any progress on the "theory of mind" front, which I guess is a borrowed notion.
mark_l_watson 2y ago
Nice! I use various LLMs many times a day as a limited coding tool and something to bounce ideas off of, and it is impossible to not think about how LLMs work and what their limitations are. I tried just asking Claude Sonet to solve the Cheryl’s Birthday word problem, changing the dates. Pretty cool that it can solve it as a word problem, and LLMs will keep getting better at coding. As a slight tangent: I used a combination of Gemini, GPT-4o, and Claude last week to write Common Lisp code for a simple RDF data store and the subset of SPARQL queries that I thought I would need in embedded Common Lisp applications. This process was far from automatic: I initially provided almost two pages of English instructions, and I had to help debug non-working code by adding debug statements and then show the models the code with print statements and the new output. I also did the optional thing of asking for stylistic changes. TLDR: saved me time and I liked the final code. I always enjoy it when people like Peter and Karpathy write relatively simple code to share ideas. I am a fairly good coder (I had the meaningless title Master Software Engineer at Capital One) but I like to read other people’s code, and I must admit that I spend more time reading code on GitHub than I spend reading technical papers.
JPLeRouzic 2y ago
Most LLMs won a T-shirt with the following inscription: " I am not as smart as Peter Norvig "!
diwank 2y ago
Script generated by o1-preview: ```python # List of possible dates dates = [ ('May', 15), ('May', 16), ('May', 19), ('June', 17), ('June', 18), ('July', 14), ('July', 16), ('August', 14), ('August', 15), ('August', 17) ] def solve_cheryls_birthday(dates): # Initial possible dates possible_dates = dates.copy() # Step 1: Albert's statement # Create a count of each day day_counts = {} for month, day in dates: day_counts[day] = day_counts.get(day, 0) + 1 # Filter out months where a unique day exists (Albert knows Bernard doesn't know) possible_months = set() for month in set(month for month, day in dates): month_days = [day for m, day in dates if m == month] if not any(day_counts[day] == 1 for day in month_days): possible_months.add(month) possible_dates = [ (month, day) for (month, day) in possible_dates if month in possible_months ] # Step 2: Bernard's statement # Recount the days in the filtered possible dates day_counts_in_possible = {} for month, day in possible_dates: day_counts_in_possible[day] = day_counts_in_possible.get(day, 0) + 1 # Bernard can now deduce the date; keep dates where the day is unique possible_dates = [ (month, day) for (month, day) in possible_dates if day_counts_in_possible[day] == 1 ] # Step 3: Albert's final statement # Recount the months in the possible dates month_counts_in_possible = {} for month, day in possible_dates: month_counts_in_possible[month] = month_counts_in_possible.get(month, 0) + 1 # Albert now knows the date; keep dates where the month is unique possible_dates = [ (month, day) for (month, day) in possible_dates if month_counts_in_possible[month] == 1 ] # The remaining date is Cheryl's birthday if len(possible_dates) == 1: return possible_dates[0] else: return None # Solve the problem birthday = solve_cheryls_birthday(dates) if birthday: print(f"Cheryl's birthday is on {birthday[0]} {birthday[1]}") else: print("Unable to determine Cheryl's birthday.") ``` Output: Cheryl's birthday is on July 16
tel 2y ago
I tried to replicate this and Claude 3.5 Sonnet got it correct on the first try. It generated a second set of dates which contained no solution so I asked it to write another python program that generates valid date sets. Here's the code it generated: https://gist.github.com/tel/8e126563d2d5fb13e7d53cf3adad862e https://gist.github.com/tel/8e126563d2d5fb13e7d53cf3adad862e To my test, it has absolutely no trouble with this problem and can correctly translate the "theory of mind" into a progressive constraint solver. Norvig is, of course, a well-respected researcher, but this is a bit disappointing. I feel confident he found that his tests failed, but to disprove his thesis (at least as is internally consistent with his experiment) we just need to find a single example of an LLM writing Python code that realizes the answer. I found that on the first try. I think it's possible that there exists some implementation of this problem, or something close enough to it, already in Claude's training data. It's quite hard to disprove that assertion. But still, I am satisfied with the code and its translation. To relate the word problem to this solution requires contemplation of the character's state-of-mind as a set of alternatives consistent with the information they've been given.
- deleted 2y ago
  [deleted]
- YeGoblynQueenne 2y ago
  >> I tried to replicate this and Claude 3.5 Sonnet got it correct on the first try. It generated a second set of dates which contained no solution so I asked it to write another python program that generates valid date sets. That's good but no cigar and it certainly didn't get it "correct on the first try". First it generated a partially correct solution. Then you had to prompt it again to generate a new program. You were only able to do that because you know what the right answer looks like in the first place. The second program is missing a second set of dates so it's not clear if it really gets it right the second time or just reproducing a different program from its training set without understanding the problem and what's wrong with the first program and just because you asked it to do so instead. >> I feel confident he found that his tests failed, but to disprove his thesis (at least as is internally consistent with his experiment) we just need to find a single example of an LLM writing Python code that realizes the answer. I found that on the first try. That's not how testing LLM code generation is done in practice, exactly because of the variance that can be expected in generated results. To properly test an LLM (which I would agree Norvig's experiment falls a little short off) one has to run multiple experiments and evaluate all the results in aggregate in some form. The usual way to do it is to draw k samples from the LLM's distribution and check whether the correct answer is generated at least n times (k@n metric). That's an awful metric because it's basically allowing arbitrary "guesses" until the LLM code generator gets it right. A simpler test is to generate k programs, check whether each program is right or wrong, and assign 1 for each correct answer and 0 for each incorrect answer, then average over all answers. It's an open question whether to count a partial answer as a 0, or 0.5. So if we took the total failure in Norvig's experiment and the only partial success in yours, and allowing for the most charitable aggregation of results, we have something like 0.25 accuracy, which is not that impressive. Not least because it's evaluated on just two test samples. Also, please don't underestimate the knowledge of experts like Peter Norvig.
fny 2y ago
How does solving a logic puzzle imply a theory of mind? I don’t mean to say that LLMs don’t have a theory of mind, just that deductive reasoning does not amount to empathetic evaluations of how someone else thinks and feels… …unless you’re a programmer.
deleted 2y ago
[deleted]
wanderingbort 2y ago
Related to this in asked LLMs to directly solve the same riddle but then obfuscated the riddle so it wouldn’t match training data and as a final test added extraneous information to distract them. Outside of o1, simple obfuscation was enough to throw off most of the group. The distracting information also had a relevant effect. I don’t think LLMs are properly fine tuned for prompters lying to them. With RAG putting “untrusted prose” into the prompt that’s a big issue. https://hackernoon.com/ai-loves-cake-more-than-truth https://hackernoon.com/ai-loves-cake-more-than-truth
roro5179 2y ago
[dead]
RevEng 2y ago
It's important to remember that modern LLMs are trained on bloody everything. They know every common logic problem, at least when stated the way they would have seen it. If you want to test an LLM, always make up a new problem. It can be the same idea as an existing problem, but change all names and numbers. I tested if GPT 3.5 could recognize chaos theory. If I stated it as the typical "butterfly flaps its wings" it instantly recognized it as the chaos theory example. If I totally changed the problem statement, it correctly identified that weather isn't correlated with a single action by a single person, but it didn't associate it with chaos theory.
johnobrien1010 2y ago
The approach is fundamentally flawed. You can’t query an LLM as to whether it has a theory of mind. You need to analyze how its internal logic works. Imagine the opposite result had occurred, and the LLM had outputted something which was considered a theory of mind… Does that prove it has one, or that it was trained on some data that had something it used which made it sound like it has a theory of mind?

290 points by stereoabuse 2y ago

o1 mini seems to get it on the first try (I didn't vet the code, but I tested it and it works on both examples provided in the notebook, `dates` and `gabe_dates`): from collections import defaultdict def find_cheryls_birthday(possible_dates): # Parse the dates into month and day dates = [date.split() for date in possible_dates] months = [month for month, day in dates] days = [day for month, day in dates] # Step 1: Albert knows the month and says he doesn't know the birthday # and that Bernard doesn't know either. This implies the month has no unique days. month_counts = defaultdict(int) day_counts = defaultdict(int) for month, day in dates: month_counts[month] += 1 day_counts[day] += 1 # Months with all days appearing more than once possible_months = [month for month in month_counts if all(day_counts[day] > 1 for m, day in dates if m == month)] filtered_dates = [date for date in dates if date[0] in possible_months] # Step 2: Bernard knows the day and now knows the birthday # This means the day is unique in the filtered dates filtered_days = defaultdict(int) for month, day in filtered_dates: filtered_days[day] += 1 possible_days = [day for day in filtered_days if filtered_days[day] == 1] filtered_dates = [date for date in filtered_dates if date[1] in possible_days] # Step 3: Albert now knows the birthday, so the month must be unique in remaining dates possible_months = defaultdict(int) for month, day in filtered_dates: possible_months[month] += 1 final_dates = [date for date in filtered_dates if possible_months[date[0]] == 1] # Convert back to original format return ' '.join(final_dates[0]) if final_dates else "No unique solution found." # Example usage: possible_dates = [ "May 15", "May 16", "May 19", "June 17", "June 18", "July 14", "July 16", "August 14", "August 15", "August 17" ] birthday = find_cheryls_birthday(possible_dates) print(f"Cheryl's Birthday is on {birthday}.")

mewpmewp2 2y ago
In addition to that after they create the 1st program with mistakes the author should have showed them the invalid output and let them have a chance to fix it. For humans solving this on the first try without running the code also tends to frequently not work.
fragmede 2y ago
"seems to" isn't good enough, especially since it's entirely possible to generate code that doesn't give the right answer. 4o is able to write some bad code, run it, recognize that it's bad, and then fix it, if you tell it to. https://chatgpt.com/share/670086ed-67bc-8009-b96c-39e539791fd3 https://chatgpt.com/share/670086ed-67bc-8009-b96c-39e539791f...
- Chinjut 2y ago
  Did you actually run the "fixed" code here? Its output is an empty list, just like the pre-"fixed" code.
  - Chinjut 2y ago
    Hm, actually, it's confusing, because clicking the [>_] links where it mentions running code gives different code than it just mentioned.
isaacfrond 2y ago
despite the name ‘mini’. it is actually more optimized for code. so that makes sense.

joe_the_user 2y ago

Deducing things from the inability of an LLM to answer a specific question seemed doomed by the "it will be able to on the next itteration" principle. It seems like the only way you could systematic chart the weaknesses of an LLM is by having a class of problems that get harder for LLMs at a steep rate, so a small increase in problem complexity requires a significant increase in LLM power.

godelski 2y ago
> Deducing things from the inability of an LLM to answer a specific question seemed doomed by the "it will be able to on the next itteration" principle. That's orthogonal. If we are pointing in the right direction(s) then yes, next iteration could resolve all problems. If we are not pointing in the right direction(s) then no, next iteration will not resolve these problems. Given LLMs rapid improvement in regurgitating knowledge from their training data but simultaneously slow improvement in their ability to generalize (such as logic "puzzles"), I think it is naive to assume we're pointed in the right direction. Maybe we're even pointing in mostly the right direction. But why assume we are? We can continue in the direction we are going while simultaneously considering it might not be well aligned. If we are well aligned, that gives us more confidence and makes gathering funding easier. If we aren't, well it is easier to course correct sooner than later. In either case, you benefit from the analysis. Understanding why things fail is more important than understanding why things succeed.
- Uehreka 2y ago
  GP is referring to the fact that if it becomes well known that LLM version X can’t solve problem Q, then the model’s trainers will make sure to include problem Q prominently in the training set, running it through over and over to ensure that version X+1 is able to solve the problem whether the model’s “reasoning” abilities have improved or not. Thus observers of the LLM space like us need to keep finding novel “Bellweather problems” that we think will evaluate a model’s ability to reason, knowing that once we start talking about it openly the problem will no longer be a useful Bellweather. By their nature as “weird-shaped” problems, these aren’t the kind of thing we’re guaranteed to have an infinite supply of. As the generations move on it will become more and more difficult to discern “actual improvements in reasoning” from “the model essentially has the solution to your particular riddle hard-coded”.

jfcoa 2y ago

This seems like a terrible test case since python examples are readily available in the training data: https://rosettacode.org/wiki/Cheryl%27s_birthday https://rosettacode.org/wiki/Cheryl%27s_birthday It's interesting that so many of the model's fail to retrieve this, but any thta do solve it should clearly be able to do so with no reasoning/theory of mind.

rghall102 2y ago
It is fascinating that the R solution just below the Python solution is much shorter and more readable. The same applies to Ruby and various Lisps. It even applies to the VisualBasic solution!
kohlerm 2y ago
I agree this is not a great test. What's good about it is that it is a constraint satisfaction problem, and I would expect LLMs to be pretty bad at unknown problems of this kind. Simple reason, an LLM only has a a finite number of layers and it cannot do arbitrary long searches.
- johnisgood 2y ago
  I almost made ChatGPT write a Python program that creates a monthly work schedule (for imaginary workers) based on specific constraints (e.g. there are 10 workers, 2 shifts (morning and night), must work 40 hours per week, must have at least one weekend in a month off, 2 minimum workers per shift, no more than 3 consecutive working days, and so forth). I am not sure if I could make it give me a working solution, however, and I have not tried Claude, for example, and I have not tried to do it with other programming languages. Maybe. The issue was that it messed up the constraints and there were no feasible solutions, that said, it did give me a working program for this that had fewer constraints.
- falcor84 2y ago
  I don't understand what you're saying - the idea is that we're asking the LLM to generate code to perform the search, rather than run an arbitrarily long search on its own, right? So why should the number of layers it has matter?

whack 2y ago

> At least with respect to this problem, they had no theory of mind. This is very interesting and insightful, but I take issue with the above conclusion. Your average software engineer would probably fail to code up a python solution to this problem. But most people would agree that the average software engineer, and the average person, possesses some theory of mind. This seems to be a pattern I'm noticing with AI. The goalposts keep moving. When I was a kid, the turing test was the holy grail for "artificial intelligence." Now, your run-of-the-mill LLM can breeze through the turing test. But no one seems to care. "They are just imitating us, that doesn't count." Every couple years, AI/ML systems make revolutionary advances, but everyone pretends it's not a big deal because of some new excuse. The latest one being "LLMs can't write a python program to solve an entire class of very challenging logic problems. Therefore LLMs possess no theory of mind." Let me stick my neck out and say something controversial. Are the latest LLMs as smart as Peter Norvig? No. Are they smarter than your average human? Yes. Can they outperform your average human at a randomly chosen cognitive task that has real-world applications? Yes. This is pretty darn revolutionary. We have crossed the rubicon. We are watching history unfold in real-time.

Jerrrrrrry 2y ago
The goalposts will continue to move until GDP improves.
- DoctorOetker 2y ago
  until who's GDP moves? Suppose nation X or power bloc Y's GDP improves due to ML, will nation Z without increasing GDP continue to move the goalposts?
titanomachy 2y ago
I consider myself a pretty average human programmer, and I was able to solve the logic puzzle and write a python program for it in ~10 mins. [0] I agree though, the people who are unable to solve this probably still have a theory of mind. It seems like we're setting a rather high bar. [0] https://pastebin.com/q33K0HJ1 https://pastebin.com/q33K0HJ1

jawns 2y ago

A long time ago, I created a version of this challenge called "Cheryl's Murder." My notebook not only solves logical induction problems like "Cheryl's Birthday," but it also generates them. https://github.com/shaungallagher/cheryls-murder/blob/master/cheryls-murder.ipynb https://github.com/shaungallagher/cheryls-murder/blob/master...

airstrike 2y ago
This is awesome, thanks for sharing
xelxebar 2y ago
> Twice a year, all of the software engineers at our company are given several days to work on a project of their choosing. Monetate sounds like it has (had?) some interesting leadership!

godelski 2y ago

I think the test is better than many other commenters are giving credit. It reminds me of responses to the river crossing problems. The reason people do tests like this is because we know the answer a priori or can determine the answer. Reasoning tests are about generalization, and this means you have to be able to generalize based on the logic. So the author knows that the question is spoiled, because they know that the model was trained on wiki. They also tested to see if the model is familiar with the problem in the first place. In fact, you too can confirm this by asking "What is the logic puzzle, Cheryl's birthday?" and they will spit you out the correct answer. The problem also went viral, so there are even variations of this. That should tell us that the model has not just been trained on it, but that it has seen it in various forms and we know that this increases its ability to generalize and perform the task. So then we're left with reasoning. How do we understand reasoning? It is the logical steps. But we need to make sure that this is distinct from memorization. So throwing in twists (as people do in the river puzzles) is a way to distinguish memory from logic. That's where these models fail. People always complain that "oh, but humans can't do it." I refer to this as "proof by self-incompetence." (I also see it claimed when it isn't actually true) But not everybody reasons, and not all the time (trivial cases are when you're asleep or in a coma, but it also includes things like when you're hangry or just dumb). Humans are different from LLMs. LLMs are giving it 100%, every time. "Proof by self-incompetence" is an exact example of this, where the goal is to explain a prior belief. But fitting data is easy, explaining data is hard (von Neumann's Elephant). There's also a key part that many people are missing in the analysis. The models were explicitly asked to *generalize* the problem. I'll give some comments about letting them attempt to solve iteratively, but this is often very tricky. I see this with the river crossing puzzles frequently, where there is information leakage passed back to the algo. Asking a followup question like "are you sure" is actually a hint. You typically don't ask that question when it is correct. Though newer models will not always apologize for being wrong, when actually correct, when they are sufficiently trained on that problem. You'll find that in these situations if you run the same prompt (in new clean sessions) multiple times that the variance in the output is very low. Overall, a good way to catch LLMs in differentiating reasoning from memorization is getting them to show their work, the steps in between. It isn't uncommon for them to get the right answer but have wrong steps, even in math problems. This is always a clear demonstration of memorization rather than reasoning. It is literally the subtly that matters. I suspect that one of the difficulties in humans analyzing LLMs is that there is no other entity that is capable of performing such feats that does not also have a theory of mind and a world model. But a good analogy might be in facts that you know, but not understanding why they are "the answer." I'm sure there's many people who have memorized complexities for many sorting algos or leet code problems and couldn't derive the answer themselves. But I really don't understand why we *need* LLMs to reason? A dictionary memorizes things, and so does wikipedia. Their lack in ability to reason does not make them any less marvelous of inventions/tools. But maybe, if we're looking to create intelligent and thinking machines, it isn't as simple as scale. We love simple things, but few things are simple and correct (though far more things are simple and approximately correct).

nextworddev 2y ago

The majority of humans in flesh can't solve the problem - so we need alternate measures for judging theory of mind capabilities in LLMs

Terr_ 2y ago
What about the difference that the human knows what they don't know? In contrast, the LLM knows nothing, but confidently half regurgitates correlational text that it is seen before.
- famouswaffles 2y ago
  As far as all the research on this goes, LLMs (internally) mostly know what they know, but incentivizing that information making it to output is difficult.
godelski 2y ago
> The majority of humans in flesh can't solve the problem This is a non sequitur. All humans have the capacity to reason, but not all humans reason. There's a big difference. Hell, even GPT-4o will tell you this. Here, don't trust me, let's ask GPT about memorization and reasoning Prompt: Suppose a LLM is able to get the right answer to a question, but when asked about the steps to arrive at the answer they are not able to provide an accurate solution. Does this tell us anything about a model's ability to reason or its ability to memorize? What about when it gets the right answer but the explanation would lead to a different answer than the one stated? Response: These scenarios highlight important aspects of a model's capabilities and limitations regarding reasoning and memorization. Let's break them down: 1. **LLM Gets the Right Answer but Can't Explain the Steps:** - **Memorization vs. Reasoning:** If a model provides a correct answer without accurately explaining the steps, it might indicate that the model is relying more on memorization rather than understanding or reasoning. The model could have seen similar questions in its training data and learned to associate certain questions with specific answers without truly understanding the underlying principles. - **Reasoning Limitations:** This situation suggests that the model may not have a robust ability to reason through the problem-solving process. It highlights a gap between recognizing patterns in data and applying logical reasoning to derive answers. 2. **LLM Provides Correct Answer but Incorrect Explanation:** - **Memorization of Answers:** This might suggest that the model has memorized the answer from similar examples in the training data but does not understand the reasoning process. It could be retrieving an answer that it "knows" is correct without understanding why. - **Inconsistent Reasoning Ability:** Giving a correct answer with an explanation that would lead to a different answer indicates inconsistencies in its reasoning ability. It may reflect that the model's internal heuristics for generating explanations are not aligned with the logic used to derive answers. In both cases, these issues highlight the challenges in AI related to understanding vs. pattern recognition. While LLMs are excellent at recognizing patterns and retrieving information, their ability to reason logically and consistently can be limited. This differentiation is crucial when evaluating the capabilities of AI models, particularly in contexts where understanding and accurate reasoning are essential.

extr 2y ago

this is an interesting problem but it’s more of a logic problem than a true test of theory of mind. when i think “theory of mind” i think being able to model an external agent with complete knowledge, incentives, and behavior. i would not doubt LLMs have something close to this for humans, almost by accident since they are trained on human outputs.

emseetech 2y ago
I would venture to guess that it has value in that we couldn't even begin to properly test a theory of mind if they can't yet reason through a logic puzzle. It would be like testing if a society could build a rocket ship when they don't know how to use tools.
- extr 2y ago
  Children as young as 4 demonstrate theory of mind but would be unable to solve artificial logic problems like OP
  - emseetech 2y ago
    Fair point, but I'd say that I don't think of LLMs as anthropomorphic individuals and more like societies acting in statistical concert. Maybe this is a failure on my part and is the wrong approach.
- zmgsabst 2y ago
  What animals have theory of mind?
  - sdenton4 2y ago
    Many, certainly many primates. Our ability to understand animal intelligence is limited by or ability to ask appropriate questions, so we tend to underestimate intelligence outside of the primate space. Here's an excellent book on this topic: https://www.goodreads.com/book/show/30231743-are-we-smart-enough-to-know-how-smart-animals-are https://www.goodreads.com/book/show/30231743-are-we-smart-en...

m3kw9 2y ago

could be an architectual issue with the LLMs because you need to juggle a lot of states just from one statement regarding a big problem. Sort of like if you ask it to write an app like facebook. It would give you a bunch of crap, which is worse.

gkfasdfasdf 2y ago

This question was posed to o1, it is able to reason through it - but now I wonder if that is because the model is already aware of the puzzle. https://x.com/d_feldman/status/1834313124058726894 https://x.com/d_feldman/status/1834313124058726894

cdfuller 2y ago
I think that could be likely. I just asked 4o "When is Cheryl's birthday?" without any other context and was given this reply Cheryl's birthday puzzle is a logic problem where Albert and Bernard are trying to figure out Cheryl's birthday based on certain clues. Cheryl provides them with ten possible dates: May 15, May 16, May 19, June 17, June 18, July 14, July 16, August 14, August 15, and August 17. Here’s the reasoning: 1. Albert knows the month and Bernard knows the day. 2. Albert says he knows Cheryl’s birthday, meaning May and June can be eliminated because they contain unique days (May 19 and June 18). If Albert had been told May or June, he wouldn’t know for sure. 3. Bernard, knowing this, says he now knows Cheryl’s birthday. This eliminates the remaining dates with unique days (July 14 and August 14). 4. Albert then confirms that he also knows the birthday, meaning Cheryl’s birthday must be in July or August, but on a date with no unique days left: July 16, August 15, or August 17. Thus, Cheryl's birthday is *July 16*.

aithrowawaycomm 2y ago

AI researchers need to learn what terms like "theory of mind" actually mean before they write dumb crap like this. Theory of mind is about attributing mental states to others, not information. What Norvig has done here is present a logic puzzle, one that works equally well when the agents are Prolog programs instead of clever children. There's no "mind" in this puzzle at all. Norvig is being childishly ignorant to call this "theory of mind." It's hard to overstate my contempt for this kind of useless junk science, especially when it comes from an impressive pedigree. Of course he is hardly the only offender: arrogant disregard for psychology is astonishingly common among LLM researchers. Maybe they should turn off ChatGPT and read a book.

deleted 2y ago
[deleted]
AdieuToLogic 2y ago
> AI researchers need to learn what terms like "theory of mind" actually mean before they write dumb crap like this. > Of course he is hardly the only offender: arrogant disregard for psychology is astonishingly common among LLM researchers. Maybe they should turn off ChatGPT and read a book. Perhaps this[0] book? It is one of the seminal texts on AI topics. Maybe Norvig should read it. 0 - http://aima.cs.berkeley.edu/ http://aima.cs.berkeley.edu/
- deleted 2y ago
  [deleted]
sitkack 2y ago
Hell, Norvig could have asked any of the LLMs if this was a good theory of mind test. CS researchers do partake in some pretty low quality science. My fav is AI researchers arrogantly rediscovering learning theory and pedagogy.
immibis 2y ago
People who fail theory of mind may assume everyone else knows something because they know it.

ynniv 2y ago

The problem with evaluating LLMs is that there's a random component, and the specific wording of prompts is so important. I asked Claude to explain the problem, then write python to solve it. When it ran there was an exception, so I pasted that back in and got the correct answer. I'm not sure what this says about theory of mind (the first script it wrote was organized into steps based on who knew what when, so it seems to grok that), but the real lesson is that if LLMs are an emulation of "human" intelligence, they should probably be given a python interpreter to check their work.

skybrian 2y ago
Yes, that helps. But if you iterate on this a few times (as I did last year with Code Interpreter), it reveals how much LLM's "like" to imitate patterns. Sure, often it will pattern-match on a useful fix and that's pretty neat. But after I told it "that fix didn't work" a couple times (with details about the error), it started assuming the fix wouldn't work and immediately trying again without my input. It learned the pattern! So, I learned to instead edit the question and resubmit. LLM's are pattern-imitating machines with a random number generator added to try to keep them from repeating the same pattern, which is what they really "want" to do. It's a brilliant hack because repeating the same pattern when it's not appropriate is a dead giveaway of machine-like behavior. (And adding a random number generator also makes it that much harder to evaluate LLM's since you need to repeat your queries and do statistics.) Although zero-shot question-answering often works, a more reliable way to get useful results out of an LLM is to "lean into it" by giving it a pattern and asking it to repeat it. (Or if you don't want it to follow a pattern, make sure you don't give it one that will confuse it.)
- brain5ide 2y ago
  If I understood correctly, that anectode in first paragraph looks like an interaction with a child who is trying something but lacks confidence.
  - skybrian 2y ago
    It did look that way and it's a fun way to interpret it, but pattern-matching on a pretty obvious pattern in the text (several failed fixes in a row) seems more likely. LLM's will repeat patterns in other circumstances too.

deleted 2y ago

[deleted]

pfisherman 2y ago

LLMs and NLP are to verbal reasoning what the calculator is to quantitative reasoning. Language and by extension verbal reasoning is full of ambiguity and semantic slipperiness. For example, what degree of semantic similarity distinguishes synonymous from synonym-ish concepts? When do we partition concepts into homonyms? I think part of the problem with how people evaluate LLMs is that the expectations that people have. Natural language != ontology. The expectation should be more Chomsky and less Boole. Asking it to solve math problems written in paragraph form is a waste of time. Use a calculator for that! Solving riddles? Code it up in prolog! Instead you should be thinking of what operations you can do on concepts, meaning, and abstract ideas! That is what these things do.

missingrib 2y ago
Is this really verbal reasoning? It's just a logic problem.
- pfisherman 2y ago
  How can one / should one combine the concepts of a dinosaur and monetary policy of the Ottoman Empire? What differentiates verbal reasoning from logic? I don’t know that either of those can be solved well with formal languages or logic.
  - samrus 2y ago
    Yeah but the Cheryl's birthday problem doesn't have any ambiguity like that. It's all in very simple language, the only complexity is keeping track of states of mind, which is easy to abstract away from the language
    - pfisherman 2y ago
      That is exactly the point I was making in my comment above. This type of unambiguous problem is best solved using formal languages - something more like quantitative reasoning. But stuff like prolog or classical automated reasoning approaches are quite brittle. They break down quickly when you start to introduce ambiguity and noise. Statistical approaches like hidden markov models that people used in these instances were the precursor to the LLMs we have today. But I was going down a rabbit hole there. My main point is that trying to use LLMs to solve logic puzzles - that can easily be solved in prolog - is a waste of time and a failure of the imagination. The applications that should be explored and would be most fruitful are those where there is ambiguity and contradiction.

mrbungie 2y ago

Not really about Theory of Mind, but in the same line, I remember the other day someone argued with me that LLMs model the world, rather than just modelling language (that may represent the world). I kept thinking about that problem and plausible experiments to show my point that LLMs are dumb about the physical world, even if they know perfectly how it works in terms of language/representation. So I thought, what happens if I give an LLM an image and I ask a representation of said image in ASCII art (obviously without relying in Python and the trivial pixel intensity to character transform it usually proposes). Remember: - LLMs should've been trained with a lot of RGB image training data with associated captions => So it should understand images very well. - LLMs should've been trained with a lot of ASCII training data with associated captions => So it should draw/write ASCII like an expert. Plus, it understands vision apparently (managed as tokens), so it should do well. But it can't do a decent translation that captures the most interesting features of an image into ASCII art (I'm pretty sure a human with an hour of time should be able to do it, even if its awful ASCII art). For example, I uploaded an image macro meme with text and two pictures of different persons kind of looking at each other. The ASCII art representation just showed two faces, that didn't look at each other but rather away from each other. It just does not "understand" the concept of crossing sights (even if it "understands" the language and even image patches when you ask about where are they looking at, it will not draw that humanly important stuff by itself). These things just work with tokens, and that is useful and seems like magic in a lot of domains. But there is no way in hell we are going to get into AGI without a fully integrated sensor platform that can model the world in its totality including interacting with it (i.e. like humans in training, but not necessarily in substrate nor training time hopefully). And I really don't know how something that has a very partial model of the world can have a Theory of Mind.

Jerrrrrrry 2y ago
it should draw/write ASCII like an expert. Not a lot of conversations incrementally totaling ASCII conversations in the training data - you are essentially asking a gold fish to climb a tree. It should have a lot of RGB image training data with associated captions => So it should understand images very well. you seem to have conflated the architectures. ChatGPT was trained on text, and text-image embedding - it can recognize, but cannot project. Thats the DALL-E portion - it leverages a similar transformer arch but they are not the same model nor architecture. However, ask a Generative Adversarial Network for ASCII, you'll get what you expect. Absent the infra-word character cohesion that LLM's token-ization provides, it will give realistic, if sometimes "uncanny" images - ones that "make sense" sequentially, or in the short term, but not the longer, or larger context. The language portion of your brain, that works faster than you do - else you would be at a loss of words constantly - is not nearly as equipped to deal with spatial problems that your posterior parietal cortex is. Ultimately we are converging towards a Mixture-of-Experts model that we will one day realize is just....us, but better.

AdieuToLogic 2y ago

What is a software program? The codification of a solution. What is a solution? An answer to a problem. What is a problem? The identification and expression of a need to be satisfied. What is a need? A uniquely human experience, one which only exists within the minds of people whom experience it.

zmgsabst 2y ago
Do animals not have needs?
- AdieuToLogic 2y ago
  > Do animals not have needs? We are animals as well, so a pedantic answer would be "yes." :-) My actual answer is the origin of the thought process began by Norvig stating: But none of them were able to write a program that finds the solution. So while the concept of "need" is shared across many entities and widely applicable to many conversations, in this context it serves as a root cause answering "why write a program."

dmead 2y ago

I wonder if they are any unique properties of those programs that we can figure out what stack overflow or textbooks they're copying.

IanCal 2y ago

I'm not a huge fan of using these kind of riddles or gotchas. Other comments have riddle variants which feel also like ways of tripping someone up - if you don't spot the change you fail. And what's more the originals are things that lots of people struggle with (that's why they're riddles not basic questions). There's also little prompting, which feels like calling up a random person and demanding they solve a riddle straight away without talking it through. Part of the assumption here is that if llms don't write the right code they don't understand what people know. I'd wager that a huge number of people couldn't solve this puzzle yet fully understand what different people have their own internal thoughts and knowledge.

rtlacn 2y ago
Humans can solve these riddles even if they have never seen a similar one. LLMs need to have seen at least a similar riddle with the same structure and/or code to plagiarize. Humans can deal with modified riddles if they are alert and awake. An LLM is always alert and awake, so it should be able to handle modified riddles, but it cannot.
- mirekrusin 2y ago
  Not to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.
  - samrus 2y ago
    Yeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit
    - Terretta 2y ago
      > An LLM can't be excused... LLMs are always operating at their best Depends how much you turn down the heat.

oli5679 2y ago

Gp1-o1 preview solves this puzzle correctly in 13 seconds and has a thorough logical deduction in the comments and explanation. I think it’s a bit unfair on llm to ask it to retrieve the puzzle definition from its training data. I posted the info on the puzzle from his notebook. https://chatgpt.com/share/670103ae-1c18-8011-8068-dd217937278a https://chatgpt.com/share/670103ae-1c18-8011-8068-dd21793727...

godelski 2y ago
The question is to get it to write generic code
lagmg05 2y ago
The question is if it solved the puzzle correctly before Norvig's article appeared. It could have been trained (I am told that existing models can be modified and augmented in any Llama discussion) on the article or on HN comments. There could even be an added routine that special cases trick questions and high profile criticisms.
- oli5679 2y ago
  Sure, I guess the best way to test this is to compose a new question, of a similar format.
  - johnisgood 2y ago
    I am not sure "of a similar format" suffices here, it should not have any resemblance or similarity to this new question or riddle.
- Terretta 2y ago
  > The question is if it solved the puzzle correctly before Norvig's article appeared. It could have been trained... This caught me by surprise — is there a suggestion or evidence that despite the "knowledge cutoff" OpenAI is continuously retraining GPT-4o's chat-backing model(s) on day over day updates to the web?
- 2y ago

willguest 2y ago

This seems to amount to asking an LLM how it feels about Cheryl, discovering that it is performatively happy about her existence, and then deducing that the LLM has no capacity for genuine emotion, expressed in the form of logic. The faulty premise lies in the formulation of the test and makes the responses both predictable, but also does a disservice to 'mind' because it tries to interpret it in such a way that an LLM could begin to grapple with the basics, but not in a meaninful way. Perhaps it is useful to help build better context-specific logic flows (generally known as software) but it doesn't seem to provide any progress on the "theory of mind" front, which I guess is a borrowed notion.

mark_l_watson 2y ago

Nice! I use various LLMs many times a day as a limited coding tool and something to bounce ideas off of, and it is impossible to not think about how LLMs work and what their limitations are. I tried just asking Claude Sonet to solve the Cheryl’s Birthday word problem, changing the dates. Pretty cool that it can solve it as a word problem, and LLMs will keep getting better at coding. As a slight tangent: I used a combination of Gemini, GPT-4o, and Claude last week to write Common Lisp code for a simple RDF data store and the subset of SPARQL queries that I thought I would need in embedded Common Lisp applications. This process was far from automatic: I initially provided almost two pages of English instructions, and I had to help debug non-working code by adding debug statements and then show the models the code with print statements and the new output. I also did the optional thing of asking for stylistic changes. TLDR: saved me time and I liked the final code. I always enjoy it when people like Peter and Karpathy write relatively simple code to share ideas. I am a fairly good coder (I had the meaningless title Master Software Engineer at Capital One) but I like to read other people’s code, and I must admit that I spend more time reading code on GitHub than I spend reading technical papers.

JPLeRouzic 2y ago

Most LLMs won a T-shirt with the following inscription: " I am not as smart as Peter Norvig "!

diwank 2y ago

Script generated by o1-preview: ```python # List of possible dates dates = [ ('May', 15), ('May', 16), ('May', 19), ('June', 17), ('June', 18), ('July', 14), ('July', 16), ('August', 14), ('August', 15), ('August', 17) ] def solve_cheryls_birthday(dates): # Initial possible dates possible_dates = dates.copy() # Step 1: Albert's statement # Create a count of each day day_counts = {} for month, day in dates: day_counts[day] = day_counts.get(day, 0) + 1 # Filter out months where a unique day exists (Albert knows Bernard doesn't know) possible_months = set() for month in set(month for month, day in dates): month_days = [day for m, day in dates if m == month] if not any(day_counts[day] == 1 for day in month_days): possible_months.add(month) possible_dates = [ (month, day) for (month, day) in possible_dates if month in possible_months ] # Step 2: Bernard's statement # Recount the days in the filtered possible dates day_counts_in_possible = {} for month, day in possible_dates: day_counts_in_possible[day] = day_counts_in_possible.get(day, 0) + 1 # Bernard can now deduce the date; keep dates where the day is unique possible_dates = [ (month, day) for (month, day) in possible_dates if day_counts_in_possible[day] == 1 ] # Step 3: Albert's final statement # Recount the months in the possible dates month_counts_in_possible = {} for month, day in possible_dates: month_counts_in_possible[month] = month_counts_in_possible.get(month, 0) + 1 # Albert now knows the date; keep dates where the month is unique possible_dates = [ (month, day) for (month, day) in possible_dates if month_counts_in_possible[month] == 1 ] # The remaining date is Cheryl's birthday if len(possible_dates) == 1: return possible_dates[0] else: return None # Solve the problem birthday = solve_cheryls_birthday(dates) if birthday: print(f"Cheryl's birthday is on {birthday[0]} {birthday[1]}") else: print("Unable to determine Cheryl's birthday.") ``` Output: Cheryl's birthday is on July 16

tel 2y ago

I tried to replicate this and Claude 3.5 Sonnet got it correct on the first try. It generated a second set of dates which contained no solution so I asked it to write another python program that generates valid date sets. Here's the code it generated: https://gist.github.com/tel/8e126563d2d5fb13e7d53cf3adad862e https://gist.github.com/tel/8e126563d2d5fb13e7d53cf3adad862e To my test, it has absolutely no trouble with this problem and can correctly translate the "theory of mind" into a progressive constraint solver. Norvig is, of course, a well-respected researcher, but this is a bit disappointing. I feel confident he found that his tests failed, but to disprove his thesis (at least as is internally consistent with his experiment) we just need to find a single example of an LLM writing Python code that realizes the answer. I found that on the first try. I think it's possible that there exists some implementation of this problem, or something close enough to it, already in Claude's training data. It's quite hard to disprove that assertion. But still, I am satisfied with the code and its translation. To relate the word problem to this solution requires contemplation of the character's state-of-mind as a set of alternatives consistent with the information they've been given.

deleted 2y ago
[deleted]
YeGoblynQueenne 2y ago
>> I tried to replicate this and Claude 3.5 Sonnet got it correct on the first try. It generated a second set of dates which contained no solution so I asked it to write another python program that generates valid date sets. That's good but no cigar and it certainly didn't get it "correct on the first try". First it generated a partially correct solution. Then you had to prompt it again to generate a new program. You were only able to do that because you know what the right answer looks like in the first place. The second program is missing a second set of dates so it's not clear if it really gets it right the second time or just reproducing a different program from its training set without understanding the problem and what's wrong with the first program and just because you asked it to do so instead. >> I feel confident he found that his tests failed, but to disprove his thesis (at least as is internally consistent with his experiment) we just need to find a single example of an LLM writing Python code that realizes the answer. I found that on the first try. That's not how testing LLM code generation is done in practice, exactly because of the variance that can be expected in generated results. To properly test an LLM (which I would agree Norvig's experiment falls a little short off) one has to run multiple experiments and evaluate all the results in aggregate in some form. The usual way to do it is to draw k samples from the LLM's distribution and check whether the correct answer is generated at least n times (k@n metric). That's an awful metric because it's basically allowing arbitrary "guesses" until the LLM code generator gets it right. A simpler test is to generate k programs, check whether each program is right or wrong, and assign 1 for each correct answer and 0 for each incorrect answer, then average over all answers. It's an open question whether to count a partial answer as a 0, or 0.5. So if we took the total failure in Norvig's experiment and the only partial success in yours, and allowing for the most charitable aggregation of results, we have something like 0.25 accuracy, which is not that impressive. Not least because it's evaluated on just two test samples. Also, please don't underestimate the knowledge of experts like Peter Norvig.

fny 2y ago

How does solving a logic puzzle imply a theory of mind? I don’t mean to say that LLMs don’t have a theory of mind, just that deductive reasoning does not amount to empathetic evaluations of how someone else thinks and feels… …unless you’re a programmer.

deleted 2y ago

[deleted]

wanderingbort 2y ago

Related to this in asked LLMs to directly solve the same riddle but then obfuscated the riddle so it wouldn’t match training data and as a final test added extraneous information to distract them. Outside of o1, simple obfuscation was enough to throw off most of the group. The distracting information also had a relevant effect. I don’t think LLMs are properly fine tuned for prompters lying to them. With RAG putting “untrusted prose” into the prompt that’s a big issue. https://hackernoon.com/ai-loves-cake-more-than-truth https://hackernoon.com/ai-loves-cake-more-than-truth

roro5179 2y ago

[dead]

RevEng 2y ago

It's important to remember that modern LLMs are trained on bloody everything. They know every common logic problem, at least when stated the way they would have seen it. If you want to test an LLM, always make up a new problem. It can be the same idea as an existing problem, but change all names and numbers. I tested if GPT 3.5 could recognize chaos theory. If I stated it as the typical "butterfly flaps its wings" it instantly recognized it as the chaos theory example. If I totally changed the problem statement, it correctly identified that weather isn't correlated with a single action by a single person, but it didn't associate it with chaos theory.

johnobrien1010 2y ago

The approach is fundamentally flawed. You can’t query an LLM as to whether it has a theory of mind. You need to analyze how its internal logic works. Imagine the opposite result had occurred, and the LLM had outputted something which was considered a theory of mind… Does that prove it has one, or that it was trained on some data that had something it used which made it sound like it has a theory of mind?