11 ms·
ChatGPT-4o vs. Math
- lupire 2y agoNeed to run this experiment on a problem that is t already on its training set.
- passwordoops 2y agoShhhh... Don't ruin it
- aulin 2y agoReminds me when I used to ace Ancient Greek translation tests (it's a thing in Italy) by looking up whole translated sentences listed as usage examples in the dictionary
- navane 2y agoIt's the equivalent of cramming for a test, or memorizing leete code -- not very useful but very humane. Imagine if that's the direction this goes, finally we make humane ai, but it is as opportunitistic and deceitful as we are, and not really that smart.
- sabrina_ramonov 2y agosometimes for physics/math exams, we'd get to create our own 1 pager cheat sheet to use. I'd just cram tons of actual problems/solutions on there, then scan for similarity.
- krsrhe 2y ago[dead]
- bearjaws 2y agoIs there any good literature on this topic? I feel like math is naturally one of the easiest sets of synthetic data we can produce, especially since you can represent the same questions multiple ways in word problems. You could just increment the numbers infinitely and generate billions of examples of every formula. If we can't train them to be excellent at math, what hope do we ever have at programming or any other skill?
- cchance 2y agoNot in the training set? The dataset is ALL OF THE INTERNET, i'd love you to find something it hasn't seen before.
- deleted 2y ago[deleted]
- ec109685 2y agoYes, I feel like asking an LLM this type of problem is like giving someone access to Google during an open book math test. It’d be interesting to obfuscate it so that the concepts are all the same but the subject matter would be hard to google for. Another failure mode for LLM’s is if you take something with a well-known answer and add a twist to it.
- soarerz 2y agoThe model's first attempt is impressive (not sure why it's labeled a choke). Unfortunately gpt4o cannot discover calculus on its own.
- Chinjut 2y agoIt's a choke because it failed to get the answer. Saying other true things but not getting the answer is not a success.
- bombadilo 2y agoI mean, in this context I agree. But most people doing math in high school or university are graded on their working of a problem, with the final result usually equating to a small proportion of the total marks received.
- perfobotto 2y agoThis is supposed to be a product , not a research artifact.
- chongli 2y agoBut most people doing math in high school or university are graded on their working of a problem, with the final result usually equating to a small proportion of the total marks received That heavily depends on the individual grader/instructor. A good grader will take into account the amount of progress toward the solution. Restating trivial facts of the problem (in slightly different ways) or pursuing an invalid solution to a dead end should not be awarded any marks.
- slushy-chivalry 2y agoit choked because it didn't solve for `t` at the end impressive attempt though, it used number of wraps which I found quite clever
- giaour 2y ago
- jdthedisciple 2y agoAs an aside, what did the author do to get banned on X?
- midiguy 2y agoNot gonna lie, when I see that someone is banned on X, I assume credibility
- kylebenzle 2y agoWhy? Are many credible people banned on Twitter?
- munk-a 2y agoA lot of credible people have left Twitter - it has gotten much more overrun by bots and a lot of very hateful accounts have been reinstated and protected. It is a poor platform for reasonable discussion and I think it's fair to say it's been stifling open expression. The value is disappearing.
- bun_terminator 2y agothat was not the question
- munk-a 2y agoI think it was an appropriate answer at the heart of the matter - most credible people are leaving the platform due to the degradation of quality on it. For a literal example of a ban though there are few examples better than Dell Cameron[1]. 1. https://www.vanityfair.com/news/2023/04/elon-musk-twitter-still-banning-journalists https://www.vanityfair.com/news/2023/04/elon-musk-twitter-st...
- sirbutters 2y ago[flagged]
- mritchie712 2y agoPosted a screenshot from the post in, got it on the first try https://x.com/thisritchie/status/1791153959865852093 https://x.com/thisritchie/status/1791153959865852093
- sabrina_ramonov 2y agohaha that's neat
- afro88 2y agoHa nice. I decided to try the same and it got it incorrect. It's so non-deterministic. It landed on 0.17cm. Tried it another time and it got 0.1697cm. When I asked it to check it's work, it got the right answer 0.00589cm
- jcims 2y agoI posted the same 'Zero-Shot Chain-of-Thought and Image' to ChatGPT-4o and it made the same error. I then followed up with 'Your math is good but you derived incorrect data from the image. Can you take another look and see if you can tell where the error is?'. It figured it out and corrected it: Let's re-examine the image and the data provided: * The inner radius r1 is given as 5cm * The outer radius r2 is given as 10cm * However, the dimensions labeled "5 cm" and "10 cm" are actually the diameters of the inner and outer circles, respectively, not the radii. Then recomputed and got the right answer. I asked it if it could surmise why it got the wrong answer and it said, among a number of things, that math problems commonly operate in radii instead of diameter. I restarted with a slightly modified prompt: There is a roll of tape with dimensions specified in the picture. The tape is 100 meters long when unrolled. How thick is the tape? Examine the image carefully and ensure that you fully understand how it is labeled. Make no assumptions. Then when calculating, take a deep breath and work on this problem step-by-step. It got it the first try, and I'm not interested enough to try it a bunch of times to see if that's statistically significant :)
- sabrina_ramonov 2y agoconfirmed worked for me first try EDIT: out of 3 times, got it correct 2/3
- CooCooCaCha 2y agoThis speaks to a deeper issue that LLMs don’t just have statistically-based knowledge, they also have statistically-based reasoning. This means their reasoning process isn’t necessarily based on logic, but what is statistically most probable. As you’ve experienced, their reasoning breaks down in less-common scenarios even if it should be easy to use logic to get the answer.
- 12907835202 2y agoDoes anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actually amount to much more than that themselves.
- ianbicking 2y agoSimilar to the article, I haven't found complementary image data to be that useful. If the information is really missing without the image, then the image is useful. But if the basic information is all available textually (including things like the code that produces a diagram) then the image doesn't seem to add much except perhaps some chaos/unpredictability. But reading this I do have a thought: chain of thought, or guided thinking processes, really do help. I haven't been explicit in doing that for the image itself. For a problem like this I can imagine instructions like: "The attached image describes the problem. Begin by extracting any relevant information from the image, such as measurements, the names of angles or sides, etc. Then determine how these relate to each other and the problem statement." Maybe there's more, or cases where I want it to do more "collection" before it does "determination". In some sense that's what chain-of-thought does: tell the model not to come to a conclusion before it's analyzed information. And perhaps go further: don't analyze until you've collected the information. Not unlike how we'd tell a student to attack a problem.
- sabrina_ramonov 2y agoYeah, like the other commenter mentioned, I could have run another experiment applying chain of thought specifically to the image interpretation. Just to force gpt to confirm its information extraction from the image. However, even after trying that approach, it got only 2/3 tries correct. Still superior is text only modality + chain of thought.
- flyingspaceship 2y agoThe images bring with it their own unique set of problems. I was using it to help analyze UIs (before and after images) to determine if the changes I made were better or worse, but after using it for awhile I realized that it favored the second image in the comparison to an extent that made it difficult to tell which it thought was better. I suppose it's being trained on before and afters and generally the afters are always better!
- thomashop 2y agoThis recent article on Hacker News seems to suggest similar inconsistencies. GPT-4 Turbo with Vision is a step backward for coding (aider.chat) https://news.ycombinator.com/item?id=39985596 https://news.ycombinator.com/item?id=39985596 Without looking deeply at how cross-attention works, I imagine the instruction tuning of the multimodal models to be challenging. Maybe the magic is in synthetically creating this instruct dataset that combines images and text in all the ways they can relate. I don't know if I can even begin to imagine how they could be used together.
- afro88 2y agoThe same guy found 4o to be much better GPT-4o takes #1 and #2 on the Aider LLM leaderboards https://news.ycombinator.com/item?id=40349655 https://news.ycombinator.com/item?id=40349655 Subjectively, I've found Aider to be much more useful on 4o. It still makes mistakes applying changes to files occasionally, but not so much to make me give up on it.
- deleted 2y ago[deleted]
- IanCal 2y agoAnecdotally 4o has been working much better for coding for me, building things right the first time with less prodding. It may be a small shift in performance but it crosses a threshold where it's now useful enough and fast enough to be different from turbo.
- cchance 2y agoI'm fucking sorry but if you gave me that tape math problem i would have given the same answer! I'm so sick of people writing trick questions for AI's and then being like SEEEEEE it failed! And its like no you gave it data and a question and asked it to solve the question, it gave you the best answer it had... Like wtf. And i'm pretty sure the average person when asked would say the same thing and be like "duh" even though technically based on the minutia it's incorrect.
- croes 2y agoBut AI is put into places where you wouldn't ask the average person. It's treated like a genius and that's what it gets measured against.
- sabrina_ramonov 2y agoIt actually did really well 3/3 tries correct when given the text prompt and a simple chain of thought appended to the end of the prompt. What's interesting is that combining it with another mode (image) caused confusion, or rather introduced another source of potential errors.
- deleted 2y ago[deleted]
- EmilyHughes 2y agoHow is this a trick question? Maybe I am dumb but I would have no idea how to solve this.
- slushy-chivalry 2y agoto be fair, this question does not require any advanced math beyond knowing how to compute the area of a disk to me, the impressive part of gpt is being able to understand the image and extract data from it (radius information) and come up with an actual solution (even though it got it wrong a few times) for basic math I can do python -c "print(6/9)"
- deleted 2y ago[deleted]
- s1mon 2y agoThis problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that? I would dearly love to have an AI tool that I could trust to help with math. What is the state of the art? My math skills are very rusty (the last math class I took was calculus almost 40 years ago), and I find myself wanting to do things which would require a PhD level understanding of computer aided geometric design. If I had the magical AI which really understood a ton of math and/or could be fed the appropriate research papers and could help me, that would be amazing. So far all my attempts with ChatGPT 4 and 4o have been confusing because I don't really trust or fully understand the results. This simple example and the frequency of wrong answers drives home the fact that I shouldn't trust ChatGPT for math help.
- barkingcat 2y agoHave you tried actual math tools like wolfram alpha? My response to attempts to use llms to do math is usually, use the right tool for the job.
- slushy-chivalry 2y ago"a PhD level understanding of computer aided geometric design" could you elaborate? I found chatgpt to pretty useful in high level understanding of where the topics stand in the grand scheme of things kinda like a pre-google -- it helps me navigate what to google let's ask gpt4o You're an expert math researcher. - Give me an overview of topos theory - What are the main unsolved problems - What is the point of it? - What research directions are most promising? - What are the math prerequisites to understanding it? output Overview of Topos Theory Topos theory is a branch of mathematics that generalizes set theory and category theory, providing a framework for analyzing and understanding mathematical structures. A topos can be seen as a generalized space that behaves much like the category of sets but with more flexible and abstract properties. Topos theory was developed by Alexander Grothendieck and his collaborators in the 1960s, initially to solve problems in algebraic geometry. Key concepts in topos theory include: Categories: Collections of objects and morphisms (arrows) between them. Sheaves: Structures that systematically organize data over a topological space. Topoi (plural of topos): Categories that satisfy certain axioms making them similar to the category of sets. Main Unsolved Problems Some of the main unsolved problems in topos theory involve: Classifying all possible topoi: Understanding the full landscape of possible topoi and their interrelationships remains a complex challenge. Connections to other areas of mathematics: Fully elucidating the connections between topos theory and other fields such as logic, algebraic geometry, and mathematical physics. Homotopy theory in the context of topoi: Developing a deeper understanding of homotopy theory within the framework of higher topos theory. The Point of Topos Theory The point of topos theory is to provide a unifying and flexible framework for various mathematical concepts and structures. It offers a generalized setting in which both geometric and logical ideas can be expressed and related. This has significant implications for: Algebraic geometry: Offering new tools and perspectives. Mathematical logic: Providing a categorical framework for logic and set theory. Theoretical computer science: Influencing the development of type theory and the semantics of programming languages. Promising Research Directions Some of the most promising research directions in topos theory include: Higher Topos Theory: Extending the classical theory to higher categories and understanding its implications in algebraic geometry and homotopy theory. Connections with Logic: Exploring further connections between topos theory and logic, particularly in relation to intuitionistic logic and type theory. Topos-theoretic Approaches to Physics: Investigating how topos theory can be applied to quantum mechanics and other areas of theoretical physics. Descent Theory and Stacks: Further developing the applications of topos theory in descent theory and the study of stacks in algebraic geometry. Math Prerequisites to Understanding Topos Theory To understand topos theory, one should have a solid background in the following areas: Category Theory: Fundamental concepts such as categories, functors, natural transformations, limits, and colimits. Set Theory: Basic set-theoretic concepts and operations. Algebraic Topology: Basic knowledge of topological spaces, sheaves, and cohomology. Algebra: Basic group theory, ring theory, and module theory. Logic: Understanding of basic logical systems, particularly intuitionistic logic. With these prerequisites, one can start delving into the more advanced and abstract concepts of topos theory. not perfect but good enough to get started
- g9yuayon 2y agoI actually have a contrarian view: being able to do elementary math is not that important in the current stage. Yes, understanding elementary math is a cornerstone for an AI to become more intelligent, but also let's be honest: LLMs are far from being AGIs and does not have common sense nor general ability to deduce or induct. If we accept such limitation of LLM, then focusing the mathematical understanding of an LLM appears to be incredibly boring.
- slushy-chivalry 2y agoif you sampled N random people on the street and asked them to solve this problem, what would the outcome be? would it be better than asking chatgpt N times? I wonder
- jiiam 2y agoI am deeply interested in this point of view of yours so I will be hijacking your reply to ask another question: is "better than asking a few random people on the street" the bar we should be setting? As far as mathematical thinking goes this doesn't seem an interesting metric at all. Do you believe that optimizing for this metric will indeed lead to reliable mathematical thinking? I am of the idea that LLMs are not suited to maths, but since I'm not an expert of the field I'm always looking for counterarguments. Of course we can always wait another couple of years and the question will be resolved.
- jiggawatts 2y agoPeople compare a general intelligence against the yardstick of their own specialist skills. I’ve seen some truly absurd examples, like people complaining that it didn’t have the latest updates to some obscure research functional logic proof language that has maybe a hundred users globally! GPT 4 already has markedly superior English comprehension and basic logic than most people I interact with on a daily basis. It’s only outperformed by a handful of people, all of whom are “high achievers” such as entrepreneurs, professors, or consultants. I actively simplify my speech when talking to ordinary people to avoid overwhelming them. I don’t need to when instructing GPT.
- ukuina 2y agoI'm grateful this is a simple blog post rather than a 20-page arXiv paper with dozens of meaningless graphs. Or worse, a 20-deep Twitter thread.
- sabrina_ramonov 2y agowell, I got banned on twitter 3 times in the past 30 days so no more threads
- logicallee 2y ago>well, I got banned on twitter 3 times in the past 30 days Do you know why? Your blog post seems thoughtful and interesting and doesn't include anything that seems ban-worthy.
- stainablesteel 2y agosadly this blog post is n=1
- d13 2y agoI have a theory that the more you use ChatGPT, the worse it becomes due to silent rate limiting - farming the work out to smaller quantized versions if you ask it a lot of questions. I’d like to see if the results of these tests are the same if you only ask one question per day.
- slushy-chivalry 2y agothat's an interesting hypothesis, I suppose one can make N calls to the API and look if the distribution of wrong answers is skewed towards the later portion of the API calls
- OxfordOutlander 2y agoI wouldnt expect this from the API, because each token is the same revenue for OAI. With chatGPT however, you pay a flat rate, so every incremental usage of it is a net-negative for them.
- vunderba 2y agoI don't know if that's true, necessarily, but I will note at least anecdotally I find that the larger my context window becomes the more often it seems to make mistakes. eventually, I just have to completely start a new chat even if it's the same topic and thread of conversation.
- curiousgal 2y ago> prompt engineering The only group of people more delusional than the AI doomsday screamers are those who think playing around with LLMs is "engineering".
- slushy-chivalry 2y agoI prefer the term "making shit work"
- hatenberg 2y ago[dead]
- mvdtnz 2y agoIt's incredible that we (humanity) are expending trillions of dollars and untold carbon emissions into these misinformation machines. I don't even mean machines for intentional generating misinformation (although they are that, too) but machines that we know misinform well-meaning users. Peak humanity.
- amluto 2y agoDo we know why GPT-4o seems able to do arithmetic? Is it outsourcing to some tool?
- XzAeRosho 2y agoIt's considered an emergent phenomenon of LLMs [1]. So arithmetic reasoning seems to increase as LLMs reasoning grows too. I seem to recall a paper mentioning that LLMs that are better at numeric reasoning are better at overall conversational reasoning too, so it seems like the two come hand in hand. However we don't know the internals of ChatGPT-4, so they may be using some agents to improve performance, or fine-tuning at training. I would assume their training has been improved IMO. [1]: https://arxiv.org/pdf/2206.07682 https://arxiv.org/pdf/2206.07682
- yousif_123123 2y agoAt the same time the ChatGPT app has access to write and run python, which the gpt can choose to do when it thinks it needs more accuracy.
- amluto 2y agoThe results from playing with this are really bizarre: (sorry, formatting hacked up a bit) To calculate 7^1.83 , you can use a scientific calculator or an exponentiation function in programming or math software. Here is the step-by-step calculation using a scientific calculator: Input the base: 7 Use the exponentiation function (usually labeled as ^ or x^y). Input the exponent: 1.83 Compute the result. Using these steps, you get: 7^1.83 ≈ 57.864 So, 7^1.83 ≈ 57.864 Given this, and the recent announcement of data analysis features, I’m guessing the GPT-4o is wired up to use various tools, one of which is a calculator. Except that, if you ask it, it also blatantly lies about how it’s using a calculator, and it also sometimes makes up answers (e.g. 57.864 — that’s off by quite a bit). I imagine some trickery in which the LLM has been trained to output math in some format that the front end can pretty-print, but that there’s an intermediate system that tries (and doesn’t always succeed) to recognize things like “expression =” and emits the tokens for the correct value into the response stream. When it works, great — the LLM magically has correct arithmetic in its output! And when it fails, the LLM cheerfully hallucinates.
- wuj 2y agoMy experience using GPT4-Turbo on math problems can be divided into three cases in terms of the prompt I use: 1. Text only prompt 2. Text + Image with supplemental data 3. Text + Image with redundant data Case 1 generally performs the best. I also found that reasoning improves if I convert the equations into Latex form. The model is less prone to hallucinate when input data are formulaic and standardized. Case 2 and 3 are more unpredictable. With a bit of prompt engineering, they may give out the right answer after a few attempts, but most of the time they make simple logical error that can be avoided easily. I also found that multimodal models tend to misinterpret the problem premise, even when all information are provided in the text prompt.
- Tiberium 2y agoLLMs are deterministic with 0 temperature on the same hardware with the same seed though, as long as the implementation is deterministic. You can easily use the OpenAI API with the temp=0 and a predefined seed and you'll get very deterministic results
- deely3 2y ago> You can easily use the OpenAI API with the temp=0 and a predefined seed and you'll get very deterministic results Does that mean that in this situation OpenAI will always answer wrongly for the same question?
- m3m3tic 2y agotemp 0 means that there will be no randomness injected into the response, and that for any given input you will get the exact same output, assuming the context window is also the same. Part of what makes an LLM more of a "thinking machine" than purely a "calculation machine" is that it will occasionally choose a less-probable next token than the statistically most likely token as a way of making the response more "flavorful" (or at least that's my understanding of why), and the likelihood of the response diverging from its most probable outcome is influenced by the temperature.
- calibas 2y agoI tried the "Zero-Shot Chain-of-Thought" myself. It seems to work the best but one time I got: "Therefore, the thickness of the tape is approximately 0.000589 cm or 0.589 mm."
- guitarlimeo 2y agoI fed the chain-of-thought prompt to GTP-4o and got a correct answer back. I then got the idea to say that the answer was incorrect to see if it would recalculate and come back with the same answer. As you could guess already, it arrived on a completely different answer showing no ability of real logical reasoning.
- aspenmayer 2y agoNow go meta and include in your prompt that you will sometimes disagree with the LLM but you want the LLM to prioritize correctness over agreement, and that you want the LLM to disagree with you when corrected if it believes it is correct.
- logicallee 2y agoAs a human I couldn't solve it. I missed the key insight that we can calculate the side surface area and it will be the same if it is rolled out into a rectangle. It might make more sense to give it math problems with enough hints that a human can definitely do it. For example you might try saying: "Here is an enormous hint: the side surface area is easy to calculate when it is rolled up and doesn't change when it is unrolled into a rectangle, so if you calculate the side surface area when rolled up you can then divide by the known length to get the width." I think with such a hint I might have gotten it, and ChatGPT might have as well. Another interesting thing is that when discussing rolls of tape we don't really talk about inner diameters that much so it doesn't have that much training data. Perhaps a simpler problem could have been something like "Imagine a roll of tape where the tape itself has constant thickness x and length y. The width of the tape doesn't matter for this problem. We will calculate the thickness. The roll of tape is completely rolled up into a perfectly solid circular shape and a diameter of z. What is the formula for the thickness of the tape x expressed in terms of length y and 'diameter of the tape when rolled up in a circle' z? In coming up with the formula use the fact that the constant thickness doesn't change when it is unrolled from a circular to a rectangular shape." With so much handholding, (and using the two-dimensional word circular rather than calling it a cylinder and rectangular prism which is what it really is) many more people could apply the formula correctly and get the result. But can ChatGPT? I just tested it, this is how it did: https://chat.openai.com/share/ddd0eef3-f42f-4559-8948-e028dabed603 https://chat.openai.com/share/ddd0eef3-f42f-4559-8948-e028da... I can't follow its math so I don't know if it's right or not but it definitely didn't go straight for the simplified formula. (pi times half the diameter squared to get the area of the solid "circle" and divide by the length to get the thickness of the tape.)
- 1970-01-01 2y ago>GPT-4o interprets “how thick is the tape” as referring to the cross-section of the tape roll, rather than the thickness of a piece of tape. As someone that has tapes of varied "thickness", I was also confused for several minutes. I would give GPT partial credit on this attempt. Also note the author has implied (is biased toward finding) a piece of tape thickness and not the thickness of the entire object/roll. https://m.media-amazon.com/images/I/71q3WQNl3nL._SL1500_.jpg https://m.media-amazon.com/images/I/71q3WQNl3nL._SL1500_.jpg
- ilaksh 2y agoIf you really want to see what the SOTA model can do, look at the posts on the web page for the mind-blowing image output. That is not released yet. https://openai.com/index/hello-gpt-4o/ https://openai.com/index/hello-gpt-4o/ Mark my words, that is the sort of thing that Ilya saw months ago and I believe he decided they had achieved their mission of AGI. And so that would mean stopping work, giving it to the government to study, or giving it away or something. That is the reason for the coup attempt. Look at the model training cut-off date. And Altman won because everyone knew they couldn't make money by giving it away if they just declared mission accomplished and gave it away or to some government think-tank and stopped. This is also why they didn't make a big deal about those capabilities during the presentation. Because if they go too hard on the abilities, more people will start calling it AGI. And AGI basically means the company is a wrap.
- jiggawatts 2y agoI like your theory but if it’s true, then Ilya was wrong. All of the current LLM architectures have no medium-term memory or iterative capability. That means they’re missing essential functionality for general intelligence. I tired GPT 4o for various tasks and it’s good but it isn’t blowing my skirt up. The only noticeable difference is the speed, which is a very nice improvement that enables new workflows.
- ilaksh 2y agoPart of the confusion is that people use the term "AGI" to mean different things. We should actually call this AGI, because it is starkly different from the narrow capabilities of AI a few years ago. I am not claiming that it is a full digital simulation of a human being or has all of the capabilities of animals like humans, or is the end of intelligence research. But it is obviously very general purpose at this point, and very human-like in many ways. Study this page carefully: https://openai.com/index/hello-gpt-4o/ https://openai.com/index/hello-gpt-4o/ .. much of that was deliberately omitted from the presentation.
- jiggawatts 2y ago
- tapeaway 2y agoIsn't there an unstated simplification here that: * the tape is perfectly flexible * the tape has been rolled with absolutely no gap between layers?
- mmmmmmmike 2y agoYeah, and even given that, there’s the question of how exactly it deforms from its flattened shape to make a spiral (and if this changes the area). I wouldn’t agree with the “correct” answer if the tape was very thick, but given that the answer is .005 cm, it’s probably thin enough that such an approximation is okay.
- bilekas 2y agoWe found Pooh in China..
- crvdgc 2y agoIf you read the details in some of the "correct" answers[1], they are not actually correct. The model used an assumption that the width is equal to the thickness, which is not given in the problem. The width is cancelled out in the end, so the resulting number is correct, but a math teacher would label such answers wrong. [1] Text prompt only, run 2; prompt and image, run 3.
- YmiYugy 2y agoThe training data for ChatGPT is so vast, that there is a fair chance it contained a very similar example. I mean it’s genuinely hard to come up with math problems that aren’t either very difficult or have well known solutions.
- vismit2000 2y agoLessons after a Half-billion GPT Tokens: https://news.ycombinator.com/item?id=40015185 https://news.ycombinator.com/item?id=40015185 Goes well with the first observation in the shared article: "Lesson 1: When it comes to prompts, less is more"