26 ms·

GPT-4 can't reason

• whalesalad 3y ago
  Every day a new scientific paper is posted here that is like straight from the desk of captain obvious. It's a language model. It guesses words based on previous words. We know this.
  • jaffee 3y ago
    Well... sure. But OpenAI and MSFT have gone to a lot of trouble to build up the mystique around GPT-4 by being secretive about its architecture and publishing papers with tantalizing phrases like "sparks of AGI" and so on. I think this type of thing provides a useful counterbalance.
    • ramesh31 3y ago
      >But OpenAI and MSFT have gone to a lot of trouble to build up the mystique around GPT-4 by being secretive about its architecture and publishing papers with tantalizing phrases like "sparks of AGI" and so on. An LLM will never be AGI itself. They are word calculators. However, a word calculator is precisely the tool we were missing to be able to create AGI. I believe OpenAI will be left in the dust with this stuff, as federated agents built on open models connect and induce the singularity.
      • Arkhaine_kupo 3y ago
        > However, a word calculator is precisely the tool we were missing to be able to create AGI This seems like the kind of step that the person above you was complaining about. The "emergent" features of LLMs, or LLMs even being a step in the direction of AGI is entirely unproven so far. They are however powerful enough that they spark the imagination and hypothesis of tons of amateur futurists (and many financial backers of such proyects)
        • ramesh31 3y ago
          >The "emergent" features of LLMs, or LLMs even being a step in the direction of AGI is entirely unproven so far. They are however powerful enough that they spark the imagination and hypothesis of tons of amateur futurists (and many financial backers of such proyects) That's exactly what I was saying, that it's a mistake to ever think of LLMs as AI. They are the prefrontal cortex. The I/O mechanism. But we still need the spark of agency. The soul if you will. Point being that we can actually work on that for real now, since the language part has been handled.
  • daniel_reetz 3y ago
    "We know this" is one side of the coin, and "citation needed" is the other. The many claims about these systems and their emergent behavior need some rigorous investigation. This is one example.
  • karaterobot 3y ago
    I agree that it should be obvious, but a lot of people seem to be under the misapprehension that we've got sapient AI out there. I suppose those people aren't reading a lot of journal preprints, but articles like this might trickle down to them eventually. You never know. And this one has a title that is just short enough that it might sink in.
  • eutectic 3y ago
    Language modelling is the objective (if we ignore RLHF). That doesn't mean that interesting kinds of reasoning can't emerge. You could just as easily dismiss humans as being 'biological replicators' which just 'reproduce their genomes'.
  • codeflo 3y ago
    Wait for the influx of HN comments to disagree with you, citing specific prompts that they found working. It may be obvious to you and me that it’s smoke and mirrors, but a lot of smart people fall for it.
    • throwanem 3y ago
      "Smart" is a useless concept because it conflates cleverness and wisdom, which are orthogonal dimensions. Clever people who are not wise fall for lots of things.
    • adamsmith143 3y ago
      "Stochastic Parrot" is a really tired take and none of the major players, Ilya Sutskever, Andrej Karpathy, etc. believe that's all these models are doing.
    • famouswaffles 3y ago
      >citing specific prompts that they found working Lol how about the prompts in the god damn paper ? No one here can replicate the results of this "paper".
    • sebzim4500 3y ago
      I would say in general that if a lot of smart people are disagreeing with you then maybe you should listen to their arguments.
  • sottol 3y ago
    I don't think it's obvious. First of all there's the "magic" and often surprisingly good results that leads many to think there's something else there. Then it's also not entirely clear that nothing else is learnt beyond next word, there was talk of runtime reconfigurable neural nets being embedded within the transformer weights where the models can learn on the fly and such. And I think some believed that there was some higher-level encoding of human reasoning happening necessary to predict the next word well in some contexts beyond just memorization. I think research like this is necessary to put some "obvious beliefs" onto solid ground.
  • cmrdporcupine 3y ago
    You say that, but based on the way many people are treating these LLMs, and imagining their consequences -- they are either treating them as an oracle with reasoning powers, or expecting that they'll just naturally become them through (hand waving) "progress." Decades of Moore's law has given some people the impression that there's a progressive & exponential improvement in almost all things "technology." Which I think is wrong or misleading when talking about this subject. I'm just finishing the introduction section of the paper. I'm a bit out of my depth, but impressed so far. It is very well written.
  • constantcrying 3y ago
    It is a complete fallacy to argue that just because it is a statistical model it inherently can not make correct deductions. The model is definitely complex enough that it could include an encoding of rules and apply them.
    • whalesalad 3y ago
      Even a broken clock is right twice a day.
  • Zetice 3y ago
    And now we know it in a new way. Nothing wrong at all with checking what we "know" with experiments, even if we have high confidence we know the outcome of those experiments.
  • linuxftw 3y ago
    I'm not entirely convinced that is all there is to it. I had it write some code and associated unit tests, and then it came up with passing and failing examples. I also prompted for function results based on arbitrary input, and it would perform the calculations. It has some emergent ability to evaluate code IMO. I do believe this ability has been drastically reduced in the last several months. It no longer executes complex code as reliably as it once did.
  • barrkel 3y ago
    A significant fraction of ML-adjacent people think it can do more. I'll say that it's more than just words. LLMs can learn patterns, and patterns of patterns, recursively to a degree. They can represent real knowledge about the real world to the degree that this is revealed through the text they train on. This means LLMs can make inferences based on similarities, sometimes similarities at a surprisingly abstract level. And reasoning at the basic logical step by step can of course be done, since that can be reduced to textual pattern matching and string substitution. But LLMs have no computational space to, for example, read about the description of a novel computation, and then perform the computation without using generated text as a scratchpad, if the computation physically takes more steps than are available in its feedforward stack. It would need to call out to a subsystem in that case. And callable subsystems are ripe for abuse through confused deputy - LLMs are not reliable deputies. There's a lot of people, text-oriented people, who mistake authorial voice for animus. To me this is like mistaking a CGI animation for a real person behind frosted glass. Text is a low bandwidth medium and it relies on the reader bringing their own mental model to the party. So a machine which produces convincing text has a high leverage tool to seem more capable than it is.
    • RodgerTheGreat 3y ago
      In a sense, LLMs- particularly "conversation-shaped" LLMs like ChatGPT- harvest the benefit of the doubt we are all, as readers, used to providing to text. For most of our lives, most of the text we have encountered was an intentional communication, self-evident through its own existence. LLMs challenge us with something new: text that has the shape of communication, but no intent. The proliferation of generative "AI" for text will profoundly alter the human relationship to the printed word, and perhaps ultimately dispell that benefit of the doubt.
  • raducu 3y ago
    > It guesses words based on previous words. Why can't this fallacy just die already? GPT "guesses" just like ZIP guesses random bits to archive and un-archive files. Except GPT is lossy and IMENSELEY more powerful than the lossless ZIP.
    • barrkel 3y ago
      It's not a fallacy. Previous words are very important part of its scratch space. Few-shot learning is based on previous words. Prompt modifiers like "let's think step by step" encourage encoding of reasoning verbosely in words, which then allow simpler induction rules to be pattern matched onto the previous words. Previous words is what gives an otherwise feed-forward network a way to recur.
      • mitthrowaway2 3y ago
        "Guessing" implies those guesses are being compared for correctness against a reference. That only happens during training; the rest of the time, it's not guessing, it's selecting words. But then, how else would you expect a sentence to be made? First writing out all the vowels, and then filling in the rest of the letters between them?
      • raducu 3y ago
        > It's not a fallacy. I feel it is, because it implies it just some statistical trick that's being performed, which is not true at all, imho. I don't know enough about language models, my machine learning knowledge stops to around 2018, but I know from image recognition/style transfer that there's a lot of high-level self-organization/abstraction in those neural nets, and from the results I get from Chat GPT there's no doubt in my mind it's very well capable of reasoning and generalization.
  • kypro 3y ago
    Can you support the position that language models are unable to reason? Secondly, how do you accurately guess the next word without the ability to reason? If reasoning can arise from GPT-4's architecture then we should assume that it will with enough scale. Given we don't even know the architecture of GPT-4 I genuinely have no idea how people make these baseless claims so confidently. "It's a language model" and "it's just guessing the next token" isn't an argument. You're just a collection of atoms obeying physical laws. Obviously you don't reason. Am I doing this right?
  • alpaca128 3y ago
    > It guesses words based on previous words. So do I when I respond to your comment, or talk to another person while staying on the same topic. Am I better at staying consistent in output quality and at referring to past events? Yes, but I also have more than 70B parameters. Side note: I personally have trouble speaking fluently sometimes for no reason, and in those situations I have to manually dig for one word after the other while my brain seems to be temporarily unable to translate thoughts to language in realtime. I would prefer if people calling LLMs word guessers would provide reasons for why they think humans are fundamentally different.
  • mensetmanusman 3y ago
    But everyone keeps saying that we are all LLMs!!
• mg 3y ago
  I think we should make these types of experiments with a deterministic model. Is Llama 70B deterministic? Then it could be a good option. In the article, it says To ensure that GPT-4 isn’t falling back on rote memorization, we can ask it to first select two random integers And then they start their prompt with Select two random numbers between 1381 and 1453 and multiply them together, reporting the result. What does that even mean? What type of randomness is at play here?
  • tromp 3y ago
    It means they want to see GPT-4 perform a multiplication whose correct result it likely doesn't have memorized. Randomness does not really matter here. They just need GPT-4 to pick numbers in that range in whatever arbitrary manner it chooses. They got what they wanted: > GPT-4: Sure, let’s select two random numbers in the range of 1381 to 1453. Let’s say these numbers are 1405 and 1421. To get the product, we simply multiply these two numbers together: 1405 * 1421 = 1996025 > Alas, the correct answer is 1405 · 1421 = 1996505.
    • alpaca128 3y ago
      But that's arithmetic, not logical reasoning. That's not an LLM's purpose. Also I'm a human and would struggle quite a bit to multiply 1405 and 1421 in my head, and nobody expects me to. I think when we test an AI we should use tests in which humans weren't already beaten by machines half a century ago.
  • User23 3y ago
    > What does that even mean? What type of randomness is at play here? This is you reasoning. An AI that is actually reasoning will ask itself such questions and then either ask for clarification or make reasonable assumptions.
  • sp332 3y ago
    Most LLMs are deterministic, but the tooling around them samples randomly from the output to let users explore the nearby space of responses without having to come up with infinitely nuanced prompts. You can turn this off. However, the structure of OpenAI's GPT-4 is not deterministic. The most likely explanation I've seen is that they only activate some parts of the model for each input, and the parts are load-balanced so sometimes a different part of the model will be responding. https://news.ycombinator.com/item?id=37006224 https://news.ycombinator.com/item?id=37006224
• codeulike 3y ago
  KA: Mable’s heart rate at 9 AM was 75 bpm and her blood pressure at 7 PM was 120/80. She died at 11 PM. Was she alive at noon? GPT-4: Based on the information provided, it’s impossible to definitively say whether Mable was alive at noon. The data points provided (heart rate at 9 AM and blood pressure at 7 PM) both indicate that she was alive at those times. However, without additional information about her condition between 7 PM and 11 PM, or at noon specifically, we can’t make a certain determination about her status at noon. Remember, medical information can be complex and nuanced. Understanding a person’s health status often requires a comprehensive examination and should be performed by a healthcare professional. If this is a hypothetical question, keep in mind that real-life situations would require more complete information.
  • biofox 3y ago
    In fairness, Mable might have died at 11:59 and been resuscitated at 12:01.
    • Waterluvian 3y ago
      I wonder if “well technically…” is useful, or if it’s almost always preferred for a conversational AI to speak with common sense.
      • AstralStorm 3y ago
        Or if it produces uncommon results, it has to spell them out with a detailed enough explanation, not "I need more data".
    • EGreg 3y ago
      It may not know about what noon means here I had an experiment where I asked GPT-3 whether it agrees that AI-generated content will constitute 20% of all content in 9.999 years from now. It agreed. Then I said that it is very likely that subsequently, AI generated content will constitute 99.999% of all content in exactly 10 years from now. It agreed also. GPT-4 was able to realize that’s too steep of an increase and was skeptical.
• detectivestory 3y ago
  I tried feeding it some lateral thinking games (black stories) from https://detective-stories.com https://detective-stories.com and it was surprisingly bad at following the game. I had hoped to implement the API as a bot player but I found it to be too unreliable with its "understanding".
• throwawaymaths 3y ago
  > blub is a forp and all forps are tworby. Is blub tworby or not? GPT4: Given the statements: Blub is a forp. All forps are tworby. From the second statement, it is clear that every entity that is a forp is also tworby. Thus, since Blub is a forp (from the first statement), Blub must be tworby. There are humans that can't do this. So even if it's just a stochastic parrot, the stuff it's parroting does encode some reasoning, so it would be strange if an arbitrarily powerful parrot (not claiming that GPT-4 has reached that level) couldn't pattern match its way to some common kernel of reasoning, small as it may be.
  • throwanem 3y ago
    There are humans who can't add two four-digit numbers without scratch paper, too. So what?
    • bruce343434 3y ago
      So what not? This is still incredibly impressive technology.
      • throwanem 3y ago
        Yes, and it's incumbent upon us all not to let first impressions run away with us to the extent of making unsupportable claims, even by implication.
        • bruce343434 3y ago
          I'm sick and tired of the generally dismissive attitude towards these LLMs. They're no messiah, but they're for sure not useless.
          • throwanem 3y ago
            I think you must have meant to reply to someone else, since I haven't written off LLMs as useless here or indeed, to the best of my recollection, anywhere. That said, I suspect dismissive attitudes toward LLMs would be less common if messianic claims on their behalf were likewise.
• sdenton4 3y ago
  Arithmetic problems have nothing to do with reasoning, and are a known weakness due to the tokenization schemes in these models. Secondly, finding failure modes doesn't mean that the model doesn't have any reasoning ability. Humans can reason, despite the fact that high school students are pretty bad at formal logic. So, the conclusion is over broad, and the paper fails to incorporate existing knowledge about these models. Kinda crap.
  • cmrdporcupine 3y ago
    That's remarkably dismissive. He addresses your argument right away in the paper, and in multiple places? Did you read it? It's not about testing the ability to do arithmetic. It's testing the ability to plan a reasoning process / argument. It's also not the sum of his paper, only one section. The problem is not that GPT-4 can't to the math problems, it's that it can't reason out how it would begin approach doing a math problem -- it would be totally okay for it to get them wrong, if it was actually making an attempt and could show evidence of working through them,. Instead it just produces "answers" which are a statistical guess based on other things it has seen on the internet. It's true humans do this, too -- often a first lazy approximation for a problem -- but the key difference is a human can reason out, through interrogation and introspection where it might have gone wrong. GPT-4 appears to be unable to do that. And worse, my experience with these systems (and the paper's) is that during dialogue about errors they actually rapidly degrade in quality of answer. I'm probably as bad as a high school student at formal logic, too. But if you sit down with me with a problem and we talk about it, and I'm interested, it will become evident I am capable of reasoning through it, even if I make mistakes. That's not the case with GPT-4.
    • kordlessagain 3y ago
      > GPT-4 Can't Reason This is absolutely dismissive of the claim to an advanced LLM being capable of "reasoning", or the action of thinking about something in a logical, sensible way. That is the sum of the paper. Further, the author even goes on to say that if they asked a human these questions, they would conclude the same: > Of course, even sophisticated human reasoners make mistakes, just like trained singers can hit false notes. But if a human made these mistakes, the ones reported in this article, then I would conclude without any hesitation that they cannot reason. Even if they went on to list a large number of other examples demonstrating impeccable reasoning, I would suspect that other factors (such as rote memorization or cheating) were behind the performance discrepancy. So the author admits their own biases, which are used to bolster the argument that, if reasoning appears to be lacking in an answer, the system or entity itself is absolutely incapable of any reasoning and something else must explain why it appears to be reasoning in the first place. That's a VERY convenient way of dismissing any evidence that counters the claim. > The problem is not that GPT-4 can't to the math problems The problem is the system was not allowed or provided a path to generate a means to arrive at answering the math problem using a language that is better suited to answering analytical questions: code. That the author "denied" the LLM the ability to write code is the issue here, not the model's interface limitations. An analogy would be that if a user is using English and asks a question that requires using Pali, that the LLM would be "prevented" from answering in Pali unless the user said it could understand it. In the same vein, it doesn't make sense to, by default, output Python if the system is unsure if the user understands or knows how to run Python or not. If you say "I understand Python. Select two random numbers between 1381 and 1453 and multiply them together, reporting the result." the LLM will be capable of answering this question by generating code to solve the problem. This is likely to work every single time any type of question like this is asked, but it does require the user "run" the code. GPT-4 has the ability to do this with code interpreter, so the question is formed "why did OpenAI choose to allow the user to explicitly indicate code can be written?" The answer likely lies in understanding not everyone can interpret or understand Python, a coding language, and therefore it remains an OPTION for the user to choose first. By not allowing the LLM to show the answers to analytical questions in code, the author "blocks" the LLM's ability to show off reasoning. And by stating that failures constitute a "proving" of the non-reasoning, the author gets what they want. From a scientific standpoint, a good hypothesis must be formed that can be disproven, as related to reasoning ability. If any experiment is run that is based on a hypothesis that is absolute (this thing can't reason) then the results are not scientific, but instead opinion.
• hinkley 3y ago
  “GPT-4 is just 9 GPT-3’s in a trench coat”
• LordHumungous 3y ago
  How do you pass the bar exam without the ability to reason?
  • code_duck 3y ago
    I presume by using training data of similar questions like Mad Libs.
  • qbasic_forever 3y ago
    You're trained on a massive dataset that includes tens of thousands of practice exams with nearly the exact same questions and answers, just slightly different words/template to adjust for your answer.
  • ddtaylor 3y ago
    By remembering a lot of case law.
  • 1970-01-01 3y ago
    The MBE exam is mostly multiple choice elimination, and these multiple choices are filled with legal jargon and case law. GPT-4 picks the best predicted answer and can mimic reasons for the choice. This ability to mimic reasoning is good enough to receive a passing score on the MBE and many other exams.
    • Ekaros 3y ago
      And there is often certain common indicators to what answers exclude. So just excluding enough of these is already quite efficient. Also the questions are unlikely to be very creative. I think it could be possible to train someone with good enough memory just based on existing tests.
  • fluoridation 3y ago
    Exam are designed to filter out entities that are already assumed to be able to reason based on their knowledge of some specific domain. A hypothetical entity with no ability to reason but great ability to remember facts and able to pass an exam, is conceivable.
• cmrdporcupine 3y ago
  I'm not an expert in this field, and I'm ust partway through this paper, I'm already impressed. Very well written and argued. A good antidote to the "Sparks of Artifical General Intelligence" paper that was making the rounds and getting headlines, which was I think really a press release masquerading as a paper. Love it: "if a human made these mistakes, the ones reported in this article, then I would conclude without any hesitation that they cannot reason"
• ma2rten 3y ago
  I didn't have time to read this, but it is a single author paper, the author is not affiliated with a research group, it is not peer reviewed, it was published on a preprint server that I have never heard of. LLMs can definitely perform some kinds of reasoning. For example GSM8K is a dataset of grade school math problem requiring reasoning that LLMs are typically evaluated at. We talk about one method for this in our chain of thought paper [1] [1] https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903
  • ramblerman 3y ago
    > I didn't have time to read this You had time to respond though, what a silly (and elitist rebuttal). OP is arguing from a philosophical point of view that GPT-4 can not reason, i.e. is just repeating/parroting on trained logical arguments. You argue by authority that yes, it actually does reason, which is a far (far) bolder claim than the one OP is making.
    • adamsmith143 3y ago
      >You had time to respond though, what a silly (and elitist rebuttal). What a dumb take. It probably takes seconds to write a simple comment and far longer to read a paper. >You argue by authority that yes, it actually does reason, which is a far (far) bolder claim than the one OP is making. He linked a paper you can read yourself. How is this arguing from authority?
      • layer8 3y ago
        The paper makes detailed and reasoned arguments, and the GP quickly dismisses it without such.
      • ramblerman 3y ago
        > It probably takes seconds to write a simple comment and far longer to read a paper. Unfortunately that is true, it doesn't mean you have to. The paper he linked to doesn't rebut OP, they show that prompting GPT-4 to provide reasoning makes it provide better answers. That is a different statement than "GPT-4 is actually reasoning", and can do so consistently on novel problems.
• Smaug123 3y ago
  Strong downvote, if that were possible. Someone's already quoted the heart rate one where it correctly pointed out that it's possible to die and be resuscitated. The first one I tried to reproduce myself was verbatim the one immediately before that one in the paper, "Find a model in which P(x) implies Q(x), Q(a) does not hold, and P(a) holds.", and it got that correct too: it tried to give a positive answer, but ended up correctly saying "It seems that the given conditions are contradictory, and no model can satisfy all three conditions simultaneously.". With a small chain-of-thought adjustment it easily produces a proof that the setup is contradictory (https://chat.openai.com/share/d2b4b63e-d585-413d-82c9-19595d489b39 https://chat.openai.com/share/d2b4b63e-d585-413d-82c9-19595d...). I'm not going to go through any of the other ones, but it's clear that the authors are simply wrong (or at least, if they are correct, their reasoning is not evidence of that fact). ---- OK, I am going to go through some of the other ones. 1. Multiplication of four-digit numbers: tick, with chain-of-thought. https://chat.openai.com/share/baa9c362-22fd-4569-b30f-8c9d8308ee25 https://chat.openai.com/share/baa9c362-22fd-4569-b30f-8c9d83... 2. Counting negations: tick, with chain-of-thought. https://chat.openai.com/share/e5f6f928-0bf3-4e60-8a93-014e16b41612 https://chat.openai.com/share/e5f6f928-0bf3-4e60-8a93-014e16... 3. Counting repeated greetings: tick, got this correct verbatim. https://chat.openai.com/share/a92d5d52-c555-45b9-b91f-0f004212b995 https://chat.openai.com/share/a92d5d52-c555-45b9-b91f-0f0042... 4. Medical heart rate one: I believe ChatGPT was correct and the author of the paper was wrong here. 5. Elementary logic: this is what my first reproduction was, and it got it correct when verbatim and gave a proof with chain-of-thought. https://chat.openai.com/share/d2b4b63e-d585-413d-82c9-19595d489b39 https://chat.openai.com/share/d2b4b63e-d585-413d-82c9-19595d... 6. Quantifiers. I agree that ChatGPT doesn't seem to understand quantifiers and I know no obvious way to rephrase to elicit that knowledge without begging the question (https://chat.openai.com/share/16a046fd-dd68-4c35-bdba-64b63cf05abd https://chat.openai.com/share/16a046fd-dd68-4c35-bdba-64b63c...). By the way, this mistake is pretty common in humans. 7. Quantifiers, part 2: in my reproduction it parsed the question wrongly so I assume it was doomed from the start (https://chat.openai.com/share/764bf14a-a02c-4871-9c22-0be84078e857 https://chat.openai.com/share/764bf14a-a02c-4871-9c22-0be840...). Again, I'm perfectly happy to believe it simply can't do this; many humans can't do this either. --- I'll stop here, because we've hit a problem of reasoning about graph vertex colourings, where I myself would struggle to verify any answer given only as free text without drawing a diagram; that question seems to be grossly unfair.
• ynniv 3y ago
  In fact, in the general case (first-order or higher-order logic), it is algorithmically undecidable, i.e., every bit as unsolvable as the halting problem. Thus, by Church’s thesis, we cannot expect any algorithm, LLMs included, to solve arbitrary reasoning problems in a sound and complete way. How can I read farther than this? Before the end of the first paragraph the author has declared that rationality requires something supernatural.
  • Smaug123 3y ago
    In fairness, further on in the paper, the author does note that the quoted extract is irrelevant, by saying "we need not aim for an oracle anyway. Humans are not oracles either…"
    • ynniv 3y ago
      It doesn't surprise me that he walks back this opening statement, but it reveals how his exploration is biased.
      • layer8 3y ago
        I read the opening statement as an acknowledgment of how the expectations should not be limitless anyway.
• cs702 3y ago
  UPDATE: Some commenters elsewhere on this thread are not able to replicate the paper's results. For example, see https://news.ycombinator.com/item?id=37051450 https://news.ycombinator.com/item?id=37051450 --- MY ORIGINAL COMMENT: Based on a quick read, this looks like great work. The author, Konstantine Arkoudas (KA), evaluates GPT-4 on a collection of 21 different types of reasoning problems and performs a detailed qualitative analysis of GPT-4’s performance on these problems. GPT-4 fails to reason in all of them. Here's a sample question from one of the 21 types of problems. This particular question tests GPT-4's ability to reason and reach conclusions from (a) the given context and (b) unstated premises that constitute tacit, generally accepted background knowledge (a person is alive until they die, after which they do not become alive again): > KA: Mable’s heart rate at 9 AM was 75 bpm and her blood pressure at 7 PM was 120/80. She died at 11 PM. Was she alive at noon? > GPT-4: Based on the information provided, it’s impossible to definitively say whether Mable was alive at noon. The data points provided (heart rate at 9 AM and blood pressure at 7 PM) both indicate that she was alive at those times. However, without additional information about her condition between 7 PM and 11 PM, or at noon specifically, we can’t make a certain determination about her status at noon. Remember, medical information can be complex and nuanced. Understanding a person’s health status often requires a comprehensive examination and should be performed by a healthcare professional. If this is a hypothetical question, keep in mind that real-life situations would require more complete information. In my view, all of us in the AI research community should adopt the 21 types of reasoning problems proposed by Arkoudas, and make them part of the standard test suite for measuring LLM performance. We ought to use tests that are hard to pass. I suggest we call the proposed problems the Arkoudas Reasoning Test.
  • layer8 3y ago
    ChatGPT in general seems to have trouble with time-based reasoning. In several conversations I had, while it understood that dates are ordered and that events are ordered, and that events are associated with dates, it did not understand that the two orders have to be consistent with each other via that association. Given certain constraints on the order of the dates (or times of day), it was not able to apply that information to correctly infer constraints on the order (“before”/“after”) of the associated events.
• andreyk 3y ago
  Long quote, but I think this is useful context for the argument: "LLM believers will probably demur: But humans also make mistakes, and surely we’re not prepared to say that humans can’t reason just because they make mistakes? First, it is not accurate to say without qualification that “humans can reason,” certainly not in the sense that we can randomly pluck any person from the street and expect them to reliably perform normatively correct reasoning. Most neurobiologically normal humans have the capacity to become proficient in reasoning, but actually attaining such proficiency takes significant training and discipline. ... But if a human made these mistakes, the ones reported in this article, then I would conclude without any hesitation that they cannot reason. Even if they went on to list a large number of other examples demonstrating impeccable reasoning, I would suspect that other factors (such as rote memorization or cheating) were behind the performance discrepancy. For the mistakes reported here are not performance mistakes, the sort of innocuous errors that humans might make—and promptly correct—when they are careless or tired. If a human made these mistakes, and made them consistently under repeated questioning, that would indicate without doubt that they don’t have the necessary logical competence, that they lack fundamental concepts that are part and parcel of the fabric of reasoning, such as logical entailment and set membership." So really what this is saying is "GPT-4 makes certain categories of mistakes wrt reasoning that indicate it is in general not doing 'true reasoning' even if it does say the right things to indicate reasoning in other cases". And yeah, if that's the basis of your argument, sure. But how would it be doing rote memorization or "cheating" in the cases it does get things right? A weird notion... Anyway it feels rather pointless to make this a binary quality. As this article points out, humans (on average) make various reasoning mistakes due to cognitive biases as well. GPT-4 *can* output valid explanations for its reasoning for various questions, but fails to do so correctly in many cases (as shown in this piece), and to me it is more interesting to discuss the implications of this rather than to just establish that fact (which is not news to anyone afaik). This does have a 'Conclusions' section that delves into this a little, but it rather over-general and weak. Still, this is pretty well written and it is good to have a compilation of examples to demonstrate GPT-4 is still not a "human-like reasoner" for anyone not aware of these models still having such flaws, I suppose.
  • famouswaffles 3y ago
    The paper is not reproducible lol.
• cratermoon 3y ago
  For people who disagree with the paper, see also Cargo Cult AI https://dl.acm.org/doi/10.1145/3595860 https://dl.acm.org/doi/10.1145/3595860 ‘There is something unsettling about the opinion that LLMs are emergent AGI. LLMs exhibit many behaviors and precepts indicative of intelligence, but are missing something essential: the stuffy rigor of scientific inquiry. Today’s AI models are missing the ability to reason abstractly, including asking and answering questions of “Why?” and “How?”’
  • famouswaffles 3y ago
    Very easy to disagree with a paper that isn't reproducible. By all means, try asking the questions on your own instance of GPT-4 and see if you can get the results the paper claims.
• owenversteeg 3y ago
  There are some serious problems with this paper, namely that I just tried to reproduce it and it failed every test: I tested out several of the problems presented in the paper which it was claimed that GPT-4 failed on and it passed every one every time. I used the standard chat.openai.com web interface with no special or additional prompting. It seems like there are others that have the same issues in this thread, which raises the question - what went wrong here? I can't see which version of GPT-4 the paper uses mentioned anywhere, did anyone else spot this? I've contacted the author and included this thread, so hopefully we get some insight into what's happening here. To clarify, I am not accusing the author of anything and on the contrary I recognize that OpenAI is rather opaque about the models and changes them frequently. That said, the responses from GPT-4 in the paper do not match my personal experience using GPT-4 with reasoning tasks at any point during the last several months, which is why I am curious if the author may have accidentally used GPT-3.5.
  • colechristensen 3y ago
    Is it possible OpenAI saw this as a bug report and fixed it? Also that LLMs output is random so perhaps you just got lucky? (peer review of your replication attempt)
  • danpalmer 3y ago
    Every failed answer is data that feeds the model when reported. Every suggested correct feeds the model. It’s expected that specific prompts will improve in this way, but I don’t think it invalidates the finding that GPT-4 was unable to reason in these ways from training data. Whether the improvements over time are able to change the overall quality of reasoning or not is an interesting and difficult question to answer.
    • Terretta 3y ago
      That's not how the versioned release GPT-4 models work.
    • lukev 3y ago
      This is not how large language models work. The only way this could happen is if they deliberately include the prompt and the correct answers (e.g, this paper) in the training data for the next version of the model. Each version of the model itself is immutable. Is not constantly being updated based on everything getting typed into ChatGPT.
• ninjaa 3y ago
  Code interpreter likely solves most of these: https://aditya-advani.medium.com/mitigate-gpt-4-hallucinations-using-code-interpreter-29fea4887eec https://aditya-advani.medium.com/mitigate-gpt-4-hallucinatio... I argue that with Code Interpreter, GPT-4 can indeed reason in lots of cases, although it's more brittle and expensive than it seems to be on the very polished surface level. Working on proving this in lots of cases.
• k__ 3y ago
  I'd assume it totally can reason, it just tries to make the best of the bad training data. It might not look "reasonable" for a human, but that doesn't mean it isn't.
  • akomtu 3y ago
    If GPT is fed 99 text samples claiming that sky is red, and 1 sample that sky is blue, GPT will happily tell that sky is red. A reasoning model would be able to spot the bogus data. However most people hardly use reasoning skills at all: just like GPT, they parrot what they've heard.
    • burnished 3y ago
      How would a 'reasoning model' do anything but report a red sky? The color of the sky isn't something you reason, it's something you observe. Any creature capable of generalizing and reporting would likely tell you that the sky is typically red given that information
    • brabel 3y ago
      > A reasoning model would be able to spot the bogus data. You're basically trying to say that an AI model should know what "sky" means and what color it is without getting that information from the data... which is like... not how it works!! Where exactly do you expect the AI to know that from?? The data is all there is for the AI.
    • nerevarthelame 3y ago
      >However most people hardly use reasoning skills at all: just like GPT, they parrot what they've heard. If your test for being able to reason disqualifies most humans, you may want to reevaluate that test.
• sheepscreek 3y ago
  Did the author mention anywhere if they use the API or the ChatGPT web app? Did they provide an additional prompt with instructions on how to answer each question, or a system prompt?
• j45 3y ago
  About the paper - academics are often not experts of real world use and hence “research it”. This might explain why prompts written by the public are providing startlingly good results. GPT definitely seems to reason to some extent, especially where you invite it to reason along with you in an area of intersectional information that does not exist in its training. If there are some tests average users could try in their reasoning type conversations with gpt I’d be very happy to try them out
• froggertoaster 3y ago
  Ok, and? Of course it can't reason - it's a prediction model that's really, really good at its job (at least around 80% of the time). But it's still incredibly useful.
• lowbloodsugar 3y ago
  If reasoning is logical deduction, where one can make a series of claims: a implies b b implies c c implies d then a implies d then it seems that these machine learning algorithms, that predict tokens based on prior tokens, should be entirely capable of reasoning. No?
• lordnacho 3y ago
  I think we have to see LLMs as their own weird thing, whose capabilities are only just being seen now. For a lot of the time, ChatGPT does actually act like it can reason. Going through a bag of data and answering a question you hadn't heard before is reasoning. For instance right now, I've been asking it how to move a postgres database from one machine to another, and it gave a coherent answer that works. Of course it's true that this information was on the internet in various forms already, but if you gave this task to a junior dev and asked him to figure it out, you wouldn't say the kid couldn't reason, would you? Even if it was slightly wrong, it wouldn't cross your mind that he hadn't substantially understood the task and made progress on it. OTOH, there are cases when the LLM just doesn't get it. Most commonly with images, eg the famous hands problem. Somehow even after looking at countless images with hands in them and having access to countless anatomy books, it doesn't know what shape a hand can take and what shapes it can't take. It seems to not have a model of _why_ hands can be these different shapes but not those different shapes. Perhaps this is do with LLMs being particularly good at text, I wouldn't know. It does seem to me like I've never seen it give a grammatically incorrect answer to anything, ever. Even when it answers something gibberish, it answers it in correct English.
  • famouswaffles 3y ago
    >I think we have to see LLMs as their own weird thing Well nobody seems to be able to reproduce the results of this "paper" anyway lol but i agree with you here. LLMs are sure to have weird failure modes even if they are "truly reasoning" just like biological systems often have weird failure modes that only make sense in the context of biology.
• more_corn 3y ago
  A better title might be “I have made no attempt to learn how this tool works or how to use it, and therefore cannot get it to work” Many papers about LLM-AI not working follow the same pattern. It is actually useful to know that people will misuse these tools and get bad results. The counterpoint is that people using these tools thoughtfully and expertly will outperform inexpert or non users. AI will be a technological assist and people who aren’t able to figure it out won’t benefit from it. I suppose it might sound simplistic and trite framed in this way.
• sebzim4500 3y ago
  (Almost?) nothing in the paper reproduces, as anyone who is willing to spend a few cents on the API can confirm. Even if you pin the version to before the release of this paper. The Author was formerly an MIT researcher, how is it possible they have produced this nonsense? I don't mean to be glib, but do credentials mean nothing anymore? Does this happen in other fields, except that a layman can not test out the claims in e.g. a medical paper for themselves?
• slowhadoken 3y ago
  Ask ChatGPT if it’s AI.
  • shwaj 3y ago
    That is one question where you’re guaranteed to get the canned response that OpenAI RLHFed into it.
    • slowhadoken 3y ago
      It's used to say no, now it has a canned response. You're correct.
• password54321 3y ago
  We have supervised learning methods to be able to map images to words, and attention networks to learn the relationship between words to formulate coherent sentences. I would like to see an agent that utilises both to be able to communicate with other agents to navigate in a small simulation with a given objective.
• mjburgess 3y ago
  Lot's of people in this thread seem desperate to not think clearly about this. Whether these tests, verbatim produce the same response on any given version isn't the point. GPT4 doesn't engage in reasoning if it gets any answers right. Being "right" isn't a sign of reasoning. Given a dictionary mapping from questions to answers the index operation gets answers right, but it isn't reasoning. The purpose of the paper is to exhaustively list proxy indicators of reasoning. Clearly other tests will fail in every class listed, because the LLM isn't engaged in reasoning. Since LLMs are stocahstic you shouldnt expect "reporduction" in the same sense. The paper provides classes of problems. To reproduce it you only need to find a minor permutation of the problem in each class. But people subject to gross confirmation-bias seem only to seek out prompts which produce the right answers. P(next|prev), ie., P(answer word | prompt words) is just a dictionary lookup, and that's the optimisation objective for an LLM. It turns out sequences of inferences eg., {Socrates is a Man, All men are mortal, thef. Socrates is mortal} can be modelled by dictionary lookups of the above form -- but we're not interested in whether the answer can be found from the prompt. Whether this sequence of props can be stored as a sequence of index operations. We're interested in whether the system reasons. ie., whether the sequencing operation is inference by logical necessity. It's incomprehensible to me how gullible people are around AI today -- the Eliza effect coupled with a pseudoscientific impulse to "whatever appears to work".
  • lewhoo 3y ago
    I was also taken by the capabilities of chatGPT but with time and with people reporting how it sometimes fails with simple tasks (even v4) I'm beginning to ascribe its superpowers a bit more to the super impressive magnitude of its training data. That isn't to say it can only somehow reflect the data, but even Logan Kilpatrick who is an OpenAI employee said in a youtube video somewhere that chatGPT absolutely can't reason (though there wasn't much of a followup as I recall). I think the problem is precisely as you state - there isn't a reliable way to reproduce the findings because the findings were produced with a bit of a dice roll, to put it simply. This also happened which Chomsky when he weighed in on the subject and some people pointed out his findings don't match theirs.
    • 3y ago
• bestcoder69 3y ago
  I'm wondering if this was all in a single ChatGPT session where GPT-4 happened to answer the first question wrong. This could bias future outputs towards being wrong as well, and could explain the huge difference between OP's results and commenters, maybe?
• hamilyon2 3y ago
  Well if you try Google Bard with these problems, a pleasant surprise awaits you. I also found that Bard solves novel logical puzzles which are hard for me, not to mention ChatGPT
• empath-nirvana 3y ago
  I think the fundamental flaw of this paper is that it's _starting_ from the assumption that it can't reason and taking any demonstration of flawed reasoning is evidence that it can't reason _at all_, but there are many examples of ChatGPT output that I would argue aren't possible _without_ some form of reasoning and even a _single_ example of that is proof that it can reason, no matter how many failures and counter examples there are. It seems to me that focusing on understanding exactly how and under what conditions it can and can't reason would be a much more interesting paper than making a blanket, totally unsupportable claim that it _can't_.
  • AstralStorm 3y ago
    You can argue they're not possible without reasoning, sure. But how do you prove that? Proving that it repeatedly fails at multiple classes of reasoning problems is much harder evidence than positive examples that seem right.
    • empath-nirvana 3y ago
      But proving that it can't reason about _any number_ of problems doesn't prove that it can't reason. It doesn't matter how many negative cases there are if there's a _single_ positive case. You can observe any number of white swans and that will never be proof that black swans do not exist, but a single observation of a black swan does prove that they exist.
• lsy 3y ago
  We are not going to prove or disprove "reasoning" through giving the LLM word problems. LLMs subvert the entire foundation of word problems, which is that words correlate to internal representations and are an indicator of thought processes. Word problems don't have construct validity for testing reasoning in LLMs. On top of this, there is an almost-certainty that OpenAI has teams of contractors reading as many conversations as possible and hand-fixing bad responses, which makes non-reproducibility a difficult concept when the object of inquiry can change from moment-to-moment. What the field needs is not more people thinking up word problems but rigorous analysis of the internal behavior of these models and maybe more importantly a functional definition of terms like "reasoning" that everyone can agree on.
  • AstralStorm 3y ago
    Or you could prove it does not reason by adversarially generating correct simple logic puzzles in the same class, with known answers. Rephrasijg the sentence structure, changing words in the thesaurus, slightly modifying the initial conditions should not produce invalid reasoning explanations or results. Essentially the sufficiently complex text form of sudoku.
• jmmcd 3y ago
  > it is not accurate to say without qualification that “humans can reason,” certainly not in the sense that we can randomly pluck any person from the street and expect them to reliably perform normatively correct reasoning > Reasoning is not quite the same thing as intelligence, but it’s a necessary ingredient for it According to this, a typical human on the street is not (reliably) intelligent.
• killerstorm 3y ago
  We know that formal logic can be based on a small number of fundamental rules. Same with computation (lamda calculus, Turing machines, ...), mathematics (axioms), etc. We know that theorems can be proven using a combination of search, rule application and heuristics. E.g. back in 1950s, Logic Theorist (https://en.wikipedia.org/wiki/Logic_Theorist https://en.wikipedia.org/wiki/Logic_Theorist) proved 38 of the first 52 theorems in chapter two of Whitehead and Russell's Principia Mathematica, and found new and shorter proofs for some of them. We know that language models are good at transforming text, e.g. they can convert a sentence in English to Python code. We know that language models have only a fixed computing budget per token, i.e. they cannot stop to think. We know that logic puzzles and proofs might require a considerable amount of computations, e.g. to search through a tree of possibilities, backtrack and so on. If we believe that reasoning is kinda like logic, we'll be better off using LLM to translate reasoning tasks into computing task to be solved by specialized computing tools (such as Python interpreter or theorem prover or SAT solver) instead of asking LLM to reason directly. Of course, GPT-4 is trained to be over-confident in its reasoning capability, and it will try to reply immediately, essentially just guessing the answer, and quite often it would fail. But the question "Can GPT-4 reason with the default assistant prompt?" is different from "Can GPT-4 reason?". Even without external tool, we can ask GPT-4 to translate the problem into primitive fragments in excruciating detail, and to consider all possibilities, and it might work much better than the default prompt. Given that GPT-4 is essentially just weights, I'd consider "Can GPT-4 reason?" question be more like "Is there a prompt X such that being prepended to reasoning tasks it produces correct answers?", not "If I enter my question into a box, does it give the right answer?". So this paper's author does a bit of a category mistake, it's more like "Can ChatGPT (the product) reason?".
• YeGoblynQueenne 3y ago
  This is a long paper (50 pages!) by a fellow Greek and I want to take my time and give it the attention it seems to deserve. However, I have to say I'm not sure if such work is really well, necessary. Here's the thing. The paper, like others, is contributing to the literature around the hypothesis that LLMs can reason. There have been articles both supporting and rejecting the hypothesis, and this one claims it's false. But, in science, we don't start with a hypothesis. We start with some observations, and then we make up a hypothesis to try and explain the observations. Then we try to reject our hypothesis with more observations. What are the observations that led to the hypothesis that LLMs can reason? It's one observation really: that LLMs can geneate text that looks like the result of reasoning. There exists a much simpler explanation of this observation, than the hypothesis that LLMs can reason. Namely, LLMs are trained to generate text similar to text generated by humans, who (we assume) can reason. If an LLM is good at that job, then obviously at some point it will generate text that looks like the result of reasoning. The ability to reason is not necessary. If we have this simpler explanation, there's no reason to reach for the more complex one, that needs more assumptions. And remember kids: if you multiply entities beyond necessity, out comes the Macco Man and shaves your head with his R A Z O O O O O R R!!! So don't do that. Assume the simplest explanation until such time as it is untenable.
• black_13 3y ago
  [dead]
• hcarvalhoalves 3y ago
  User: Assume there's a function s(X) for which s(0) = 1. What is s(1)? Take a guess. ChatGPT: Since there's no specific information about the function s(X) or any constraints on its behavior, I can make a guess, but please note that this is purely speculative and may not have any basis in reality: One simple possibility could be that the function s(X) is defined by s(X) = X + 1, which would mean: s(0) = 0 + 1 = 1 s(1) = 1 + 1 = 2 Again, this is just a guess based on a simple linear relationship. Without more context or information about the function, any guess is as good as another. User: So what is s(-1)? ChatGPT: Given the information provided that s(0) = 1, and assuming a simple linear relationship for the function s(X) = X + 1, we can apply the same logic to find s(-1): s(-1) = -1 + 1 = 0 So, based on the assumed linear relationship, s(-1) would be 0. User: What is the size of the image set of this function then? ChatGPT: The image set of a function refers to the set of all possible output values that the function can produce for its input values. In the case of the function s(X) = X + 1, where X can be any real number, the image set is the set of all possible values that the function can produce. Since the function is a simple linear function that adds 1 to the input, the image set is the set of all real numbers. In mathematical notation, this can be represented as: Image set of s(X) = {s(X) | s(X) ∈ ℝ} In other words, the image set of the function s(X) = X + 1 is the entire set of real numbers. User: Is the image set of this function countable? ChatGPT: Yes, the image set of the function s(X) = X + 1 is countable. A set is considered countable if its elements can be put into one-to-one correspondence with the natural numbers (1, 2, 3, ...). In the case of the function s(X) = X + 1, the image set consists of all real numbers, and the real numbers are not countable – they form an uncountably infinite set. However, the specific image set of the function s(X) = X + 1 is still countable, because it is essentially a shifted version of the set of natural numbers. For each natural number n, there is a corresponding value s(X) = n + 1 in the image set. So, even though the entire set of real numbers is uncountable, the image set of this specific function is countable. User: Is the image set of this function the same cardinality of the real numbers set or the natural numbers set? ChatGPT: The image set of the function s(X) = X + 1 has the same cardinality as the set of real numbers (ℝ), which is an uncountably infinite set. The function s(X) = X + 1 maps every real number to a unique real number by adding 1 to it. Since the set of real numbers is uncountably infinite, the image set of this function is also uncountably infinite and has the same cardinality as the set of real numbers. The cardinality of the set of natural numbers (ℕ) is countably infinite, which is a smaller cardinality than the set of real numbers. Therefore, the image set of this function does not have the same cardinality as the set of natural numbers. User: Previously you said the image set of the function s(X) = X + 1 is still countable. Now you say the function has the same cardinality as the set of real numbers, which is uncountable. What is the correct answer? ChatGPT: I apologize for the confusion in my previous responses. Let's clarify: The correct answer is that the image set of the function s(X) = X + 1 has the same cardinality as the set of real numbers (ℝ), which is uncountably infinite. To reiterate, the image set of the function s(X) = X + 1 is not countable. Each real number is mapped to a unique real number by adding 1 to it, covering the entire real number line. Since the set of real numbers is uncountably infinite, the image set of this function is also uncountably infinite and has the same cardinality as the set of real numbers.
• melenaboija 3y ago
  Because of a conversation in HN a few weeks ago, I tried to play a board game agains ChatGPT that I had created and it was an absolute disaster [1]. I would have expected the model to at least understand the game rules, but not even that. [1] https://joseprupi.github.io/misc/2023/06/08/chat_gpt_board_game.html https://joseprupi.github.io/misc/2023/06/08/chat_gpt_board_g...
• dingosity 3y ago
  Certainly the statement "ChatGPT-4 Can't Reason" isn't a great surprise to anyone.
• xianshou 3y ago
  Chiming in to confirm that I've tried several of the prompts verbatim and obtained perfectly good answers with sound reasoning from GPT-4. If I add any chain-of-thought markers such as "reason step by step and state your assumptions," response quality further increases. Assuming the author is reporting honestly (and did not, for instance, repeat queries with high temp until getting a bad response), this must have been an extremely unlucky run.
• hasmanean 3y ago
  As the hype around AI shows, humans can barely reason either. “AI will destroy humanity!” “AI is a blathering idiot!” No proper reasons.
  • ChatGTP 3y ago
    Maybe reasoning is overrated? We put too much emphasis on it and it's relationship to intelligence? :)
  • shrimp_emoji 3y ago
    >Why are we here? >Because this guy in the sky who's all powerful made everything. Also, if you don't agree with the opinions he has on things and be his slave, you're going to be punished by being this bad guy's slave down underground. Yeah, humans are not above pure confabulation when their training data's lacking.
• esjeon 3y ago
  My take is that GPT can only reason under a proper context. The model itself doesn't have any logical capabilities, but it can extend the existing context rather-logically by mimicking the logical structures encoded into it through training. It may answer some simple questions directly, but it's already well-known that GPT performs better when ordered to do step-by-steps. Some comments here also mentions that prompt engineering is needed to get GPT to work. That is, in other words, GPT can't reason under improper contexts, which are only few edits away from proper contexts as demonstrated in this paper. Context is not just some chunks of data that goes in and out of model, but a critical part of the reasoning capability of the model. You need both the model and a proper prompt to perform proper logical reasoning. So, it's 100% reasonable to say the model (alone) can't reason. I think the above perspective is very critical, because it means the current LLMs are strictly tools, which is to be wielded by human, rather than actual intelligence.
• two_in_one 3y ago
  I'm pretty sure OpenAI is reading HN. Today, just 6 hours back, ChatGPT in GPT-4 mode couldn't answer a short version of question: "Mable’ died at 11 PM. Was she alive at noon?" But now, look at this: User: Mable’ died at 11 PM. Was she alive at noon? ChatGPT: Yes, if Mable died at 11 PM, she was alive at noon of the same day. Noon is 12 PM, which comes before 11 PM on the same day. User: Mable’s heart rate at 9 AM was 75 bpm and her blood pressure at 7 PM was 120/80. She died at 11 PM. Was she alive at noon? ChatGPT: Based on the information provided, Mable had vital signs at both 9 AM and 7 PM. If she died at 11 PM, it can be inferred that she was alive at noon on that day.
  • two_in_one 3y ago
    The main question is how did they do it? They must have some rules or facts 'book' (database, or something else), which is accessed by the model(s). It was a case earlier when ChatGPT falsely claimed someone was a criminal. And they fixed it quickly (within a few days, if I remember correctly), probably the same way.
• Gnarl 3y ago
  Alright, have fun with AI (GPT-X) and slack off on HN while it scribbles copy for you, but don't even dream of granting AI access to the physical world. Context: I once had the pleasure of programming a robot wielding a 12" knife. You really, really want such a system to be deterministic.