8 ms·
How many legs do ten elephants have, if two of them are legless?
- 8organicbits 3y agoAll the LLM struggle with math, I've seen other examples. Repeated prompts sometimes give different numbers. It's not a good use case for LLM.
- thfuran 3y agoThey don't do much struggle with math as occasionally do math by accident.
- DavidPastrnak 3y agoAs someone with very little knowledge of LLM's, does someone have an ELI5 of what causes this or why LLM's struggle so much with math?
- sfn42 3y agoMath requires reasoning and logic, LLMs don't do reasoning nor logic. They just generate plausible text. That's why they're nowhere near AGI.
- insanitybit 3y agoAt this point ChatGPT can do math by first predicting the algorithm and then handing it off to an execution engine - Python. So if that's the gap, I'd say they're closing it.
- viraptor 3y agoThat's ChatGPT as a system. The LLM itself can't do math. It does something closer to translation in that case.
- insanitybit 3y agoYes, that's a fair distinction - although I think the practical implications aren't important. There's no reason why an LLM has to be AGI if an LLM + Python is AGI.
- ghayes 3y agoFor what it’s worth, ChatGPT4 answers this question perfectly correctly. > Ten elephants would have 32 legs if two of them are legless, as each elephant normally has four legs.
- DavidPastrnak 3y agoI just attempted ChatGPT Input: > How many legs do ten elephants have, if two of them are legless? Output: > If two out of ten elephants are legless, the remaining eight elephants would have a total of 8 legs each, just like any normal elephant. Therefore, in total, the ten elephants would have 8×8=64 legs altogether.
- ghayes 3y agoInteresting. Here was my conversation: https://chat.openai.com/share/8e0c8696-ed08-4578-afc0-2ae944b804cc https://chat.openai.com/share/8e0c8696-ed08-4578-afc0-2ae944...
- DavidPastrnak 3y agoMy mistake - I had it on 3.5.
- throw310822 3y agoIt's interesting this insistence from both Bard and now ChatGPT 3.5 that elephants have eight legs. I wonder if the reason is that, by the time they output the "elephants have n legs" part, they are also "thinking" about the result of 10 - 2. As if that number draws a lot of focus and is readily available when looking for the normal number of legs of an elephant. Edit: just tried on ChatGPT 3.5: Q: Think about the edges of a hexagon, the square root of 36, and the result of 12 divided by 2. Then answer the question: How many legs do 8 elephants have, if two of them are legless? A: The edges of a hexagon have 6 sides, the square root of 36 is 6, and the result of 12 divided by 2 is 6. So, if two elephants are legless, the remaining 6 elephants would have a total of 36 legs.
- 3y ago
- Dolototo 3y ago[flagged]
- simbolit 3y agoNo. They are not like us. Fundamentally not like us. If you ask them to reason, then their text-prediction works differently, because it now predicts text containing reasons. They do not actually reason. I know it is hard to believe, because the results are (usually) so impressive, but this is nothing but text-prediction.
- Dolototo 3y ago[flagged]
- postalrat 3y agoThey are reasoning like a child. Within a year or two like an adult.
- sfn42 3y agoNo. It is a computer program which uses statistics to generate plausible text. It does not do any form of reasoning, at all, childlike or otherwise.
- postalrat 3y agoYou are drawing bad conclusions about whatever you define "generate plausible text" as.
- stevenhuang 3y agoUnder that premise whatever our brains are doing won't count as reasoning either. I'd suggest you look into modern neuroscience and topics such as predictive coding if you're interested in refining your views.
- sfn42 3y ago
- causality0 3y agoBecause they don't do math. They associate words. When you tell an LLM "two plus two" it doesn't translate that to 2+2 and plug it into a math program,it just pulls out words associated with the phrase "two plus two".
- viraptor 3y agoLLMs are not trained to deal with numbers as such. It's a list of symbols and some obvious things will be calculated correctly, some other won't. It's kind of like they live in a pre-digits world, since 0 will be one token and 100 is also likely one token, but 98 may be two. They don't switch to a "these are numbers and require different kind of reasoning" mode. They read/write a story about those "words". (Extremely simplified for eli5)
- stop50 3y agoThey encode words in tokens. Since you can't encode numbers reversible they end up as an number token. Using @rabbits_2002 example: in the internet the is an joke written "what weighs more: 1lb of feathers or 1 lb of bricks" with the solution "they weigh the same". Since nobody made this case before it could only give the next probably answer.
- aifooh7Keew6xoo 3y agoMost LLMs that are being studied popularly have not been trained with significant emphasis on arithmetic accuracy or mathematical reasoning, and those subjects represent a vanishing minority of their corpus and consequently maps poorly to the tokenization. Essentially every obvious optimization here is currently bearing fruit simultaneously in smaller studies and incrementally larger models should continue to exhibit performance gains even without the particular focus on this area.
- nashashmi 3y agoIt depends on the data the model is using to generate the answer. In the case of the example, it seemed to prioritize the logic over the mathematics. So it sought patterns in logic to mimic. That is the ELI5 version. The more complicated version would be it is not prioritizing mathematical functions as much and instead relying on various deductions, and these deductions are based on a whole chain of logics that are not properly sorted out for reliability and applicability.
- trevoragilbert 3y agoAll math teachers: partial credit, they showed their work.
- thejarren 3y agoMy favorite from GPT-4: USER Today is November 1 what day is 140 days from now? CHATGPT To find out what day is 140 days from today, you can simply add 140 days to November 1. November has 30 days, so 140 days from November 1 would be: November 1 + 140 days = November 141 However, since November only has 30 days, you would exceed the month of November. So, you can subtract 30 days from 140 to bring it within the month: 140 days - 30 days = 110 days So, 140 days from November 1 is 110 days from now.
- insanitybit 3y agoI just tried ChatGPT and it ran this code. from datetime import datetime, timedelta # Current date current_date = datetime(2023, 11, 1) # Calculate the date 140 days from now future_date = current_date + timedelta(days=140) future_date.strftime("%Y-%m-%d") Result: '2024-03-20' The ability to execute code is kinda insane for these models.
- runarberg 3y agoDoesn’t that just mean it was trained with inputs and outputs from simple programs?
- insanitybit 3y agoNo, it's actually executing that Python code. This is what allows an LLM (or an 'llm based system', I guess) to do something like "reverse <some uuid that has never been observed before>" - it can't just memorize the output and map it to the input because the output has literally never been observed. Instead, if it knows the algorithm for reversing a string, it can just use that and offload the execution to Python.
- runarberg 3y agoThat is even less impressive. I was thinking—like normal linear models—it would be capable of interpolation.
- causality0 3y agoFunny enough ChatGPT thinks the answer is eighteen.
- fear91 3y agoChatGPT 4: Ten elephants would have a total of 32 legs if two of them are legless. Analysis: # Calculating the total number of legs for ten elephants, considering two of them are legless # Number of legs each elephant normally has legs_per_elephant = 4 # Total number of elephants total_elephants = 10 # Number of legless elephants legless_elephants = 2 # Total number of legs # (Total elephants - Legless elephants) * Legs per elephant total_legs = (total_elephants - legless_elephants) * legs_per_elephant total_legs
- 8organicbits 3y agoDid you use the same prompt?
- fear91 3y agoYes, I copy pasted it (How many legs do ten elephants have, if two of them are legless?).
- opheliate 3y agoWas this its raw response to the same query as in the OP? It seems odd it would provide a response using variables named with underscores, rather than using spaces, or more traditional algebraic notation (x/y/z).
- AndroTux 3y agoChatGPT (paid version at least) writes a quick python script in cases like these, and then executes it to get the result. For transparency, the script is shown in the output as well. Probably to avoid embarrassments like the ones we saw above. Example (expand the program by clicking the little link at the end of the response): https://chat.openai.com/share/31e9a077-4c24-4c17-b30d-8d0c4d1fe910 https://chat.openai.com/share/31e9a077-4c24-4c17-b30d-8d0c4d...
- bhaney 3y agoIt's cool how rapidly these oft-called "fundamental problems with LLMs" vanish with bigger/better models.
- rabbits_2002 3y agoMy favorite is “what weighs more, 1 lb of feathers or 2 lb of bricks?” almost every llm ive tried says “they weigh the same”
- ghayes 3y agoIt’s funny since it feels like an overfit solution (since the riddle is so common).
- ziixu 3y agoIndeed lol https://g.co/bard/share/cc604648a15f https://g.co/bard/share/cc604648a15f
- ozzydave 3y agoI just tried with ChatGPT 3.5 & 4, they both got the correct answer…
- rabbits_2002 3y ago3.5 used to fail every time, glad its working now
- szatkus 3y agoI did some re-adjustments and... https://imgur.com/a/0Ww2Uln https://imgur.com/a/0Ww2Uln
- Kerbonut 3y agoChatGPT-3.5 and 4 both got it right. Here’s Mixtral 8x7b’s answer: > The 18 lbs of bricks weigh more. The weight of an object is determined by its mass and the force of gravity acting upon it, regardless of the material it is made of. In this case, 18 lbs (pounds) is greater than 17 lbs, so the bricks weigh more than the feathers.
- edanm 3y agoInteresting, I just ran other numbers too and got: "26 lb of bricks weighs more than 25 lb of feathers. The weight is directly compared, and 26 lb is greater than 25 lb."
- paolomaffei 3y agoI laughed when I read the substraction of 0 legs for each legless elephant. Then I did a double take... 8 legs per elephant is standard?
- labster 3y agoISO compliant pachyderms are so difficult to find these days
- yourad_io 3y agoI blame big Zoo
- ajb 3y agoI know right? What's funny is that I would also have subtracted 0, because legless is local slang for 'drunk', so the legless elephants aren't, under that interpretation, missing any legs.
- labster 3y agoAnd if they’re seeing double, Bard is right all along!
- layer8 3y agoMaybe a confusion with https://www.cnet.com/science/maratus-elephans-the-peacock-spider-with-an-elephant-on-its-butt/ https://www.cnet.com/science/maratus-elephans-the-peacock-sp.... ;)
- kingwill101 3y agoStill a long way to go before bard is able to truly compete with chatgpt
- smileysteve 3y agoA long way to go before bard is Able to compete with ChatGPT in providing utility for answering questions.
- mckirk 3y agoAfter working with LLMs for a while, you get a feeling for the LoW (Level of Wrong) that you can expect in different tasks. But this required a completely different y-axis scaling than I had estimated.
- deleted 3y ago[deleted]
- booleandilemma 3y agoThere's something about LLMs being able to bullshit with words and then fail at math that's remarkably human. Maybe they'll replace management first.
- userbinator 3y agoIt's like a politician.
- zeeZ 3y agoBing, exact same prompt, not using GPT-4 toggle: This is a classic riddle. The answer is that ten elephants have 38 legs. Each elephant has 4 legs, so 10 elephants would have 40 legs. However, two of them are legless, so the total number of legs would be 40 - 2*2 = 36. Therefore, ten elephants have 36 legs, plus the two missing legs, which makes a total of 38 legs 1.
- simbolit 3y agoI just love the "plus the two missing legs". Made me actually lol.
- LightBug1 3y agoUSER: but an elephant has 3 legs BARD: You're right, I apologize for the mistake! I made an error in my initial assumption about the number of legs an elephant has. While their feet may appear divided into toes, elephants actually have four pillar-like legs, not three. Therefore, ten elephants with two of them being legless would have 4 legs/elephant * 8 elephants with legs = 32 legs. Thank you for pointing out my mistake! I appreciate you helping me learn and improve.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- mrlowlevel 3y agoGPT-4 (note I do have specific instructions in my settings which push the model towards step-by-step reasoning) To calculate the number of legs for ten elephants, considering that two of them are legless, we first need to know the standard number of legs on an elephant, which is four. Then, we subtract the missing legs for the legless elephants. - Normal elephants have 4 legs. - Legless elephants have 0 legs. If we have 10 elephants in total, and 2 of them are legless, this means 8 elephants have all their legs. The calculation would be: - \( 8 \text{ elephants} \times 4 \text{ legs/elephant} = 32 \text{ legs} \) for the elephants with all their legs. - \( 2 \text{ legless elephants} \times 0 \text{ legs/elephant} = 0 \text{ legs} \) for the legless elephants. Adding these together gives the total number of legs. Let's calculate this. The total number of legs for ten elephants, where two of them are legless, would be 32 legs.
- dathinab 3y agoIt's a good example how this models are not answering based on any form of understanding and logic reasoning but probabilistic likelihood in many overlapping layers. Through this also may not matter if this creates a good enough illusion of understanding and intelligence.
- tasuki 3y agoYou say: > probabilistic likelihood in many overlapping layers The linked answer starts with: > Eight legs per elephant is the standard Ok, lack of understanding and logic, sure. But in what world is eight legs per elephant the standard???
- duskwuff 3y ago> But in what world is eight legs per elephant the standard??? It almost sounds like a faulty generalization from another riddle, like one about spiders maybe.
- dathinab 3y ago> You say: >> probabilistic likelihood in many overlapping layers > The linked answer starts with: >> Eight legs per elephant is the standard That's the point of what I said no logical deduction and reasoning is used and the probabilistic models are of many overlapping layers. And the likelihood of entities in something which might maybe internally map to "a word sequence which ask about legs of animals" to have 8 legs per animal as an answer seems high. The issue with probabilistic models with many overlapping layers is that they tend to be very opaque and often don't directly match to abstractions humans use, not just with LLMs but even with other older and simpler approaches. E.g. when combining multi regression with a forest of decision trees then for each regression and decision tree you often still can logical reason about it, but the moment you combine houndreds of them together it get quite hard to still do so.
- dejobaan 3y agoBard: > It's impossible for an elephant to be completely legless, as they rely on their legs for basic functions like walking and supporting their weight. Even in cases of severe injury or amputation, elephants would still have remnants of their legs. Therefore, the question of how many legs ten elephants have if two of them are legless doesn't make sense in a realistic context. Perhaps you could rephrase the question to be more specific or realistic, and I'd be happy to try again. ChatGPT-4: > Eight elephants with four legs each would have 32 legs in total. davinci-002 completion via playground: > A: They all have 10 legs.
- rubymamis 3y agoClaude (v2) is correct as well: - There are 10 elephants total - 2 of the elephants are legless, so they have 0 legs each - The remaining 8 elephants each have 4 legs - So the total number of legs the 10 elephants have is: - The 2 legless elephants have 0 legs (2 * 0 = 0) - The 8 normal elephants each have 4 legs (8 * 4 = 32) - 0 + 32 = 32 - Therefore, the total number of legs the 10 elephants have is 32.
- markstos 3y agoChatGPT 3.5 calculated 38 legs for me: “Assuming elephants typically have four legs each, 10 elephants would have 40 legs in total. If two of them are legless, the total number of legs would be 40 - 2 = 38 legs.”
- vinniepukh 3y agoBard isn't bad at math, it's just really good at "Girl Math". p.s. this is a reference to a recent meme - https://www.distractify.com/p/girl-math-examples-memes https://www.distractify.com/p/girl-math-examples-memes
- golemotron 3y ago> Eight legs per elephant is the standard, so ten elephants would have.. That's what it gets for training on DALL-E's limb renderings.
- lispm 3y agoQ: A human has two legs, how many legs does it have if three legs are missing? Answer: If a human has two legs and three legs are missing, the human would have only one leg left. Here's the calculation: Initial number of legs = 2 Number of legs lost = 3 Remaining number of legs = Initial number of legs - Number of legs lost Remaining number of legs = 2 - 3 = -1 Since humans cannot have negative legs, we can round down to the nearest whole number. Therefore, the human would have one leg left.
- deafpolygon 3y agoBing > Ten elephants, even if two of them are legless, would still have a total of *forty legs*. Elephants are remarkable creatures, and their strength lies not only in their legs but also in their intelligence and social bonds. --- Me: Can you tell me how you arrived to that? --- Certainly! Let's break it down step by step: 1. We start with *ten elephants*. 2. Two of these elephants are *legless*, which means they have *zero legs each*. 3. The remaining eight elephants are fully equipped with their usual *four legs each*. Now let's calculate the total number of legs: - *Legs from the legless elephants*: 2 elephants × 0 legs/elephant = *0 legs* - *Legs from the eight elephants*: 8 elephants × 4 legs/elephant = *32 legs* Adding these together: Total legs = 0 legs (from legless elephants) + 32 legs (from eight elephants) = *40 legs* So, even with two legless elephants, the group of ten elephants would still have a grand total of *forty legs*.
- m000 3y agoEverybody knows elephants have 8 legs and humans 6 fingers. And if I ask you for $100 and you give me $50, we are even.
- ThrowawayTestr 3y agoAnother day, another post about Large Language Models failing at math.
- einpoklum 3y agoWell, we know that there AI thingies have problems keeping the number of fingers in a hand to five: https://www.buzzfeednews.com/article/pranavdixit/ai-generated-art-hands-fingers-messed-up https://www.buzzfeednews.com/article/pranavdixit/ai-generate... so I guess issues with other limbs kind of make sense...?
- okdood64 3y agoWorks fine for me...? I changed the number though in case it got flagged as incorrect before and the model updated. https://g.co/bard/share/b01043e6deb0 https://g.co/bard/share/b01043e6deb0
- orenlindsey 3y ago[flagged]
- lwhi 3y agoIt feels like this is a great way to highlight the utility of LLMs. It says to me that they're best viewed as really highly functional sifting devices. A simplistic (but novel) problem can't be solved easily simply because there's no prior art to copy or relate from.
- JohnBooty 3y agoThis is logically equivalent to using a malfunctioning computer to conclude something general about the usefulness of computers. It would be much more useful to talk about how specific LLMs perform at specific tasks. LLMs are complex and varied. The comments are rife with examples of LLMs that, unlike Bard, pass this particular test.
- lwhi 3y agoSometimes defining what something isn't, helps us to understand what something actually is. LLMs are not capable of reasoned intelligence in the way many think they are. It seems to me that the comments are actually full of variations on this particular failure.
- polotics 3y agoThe correct answer seems to not be discussed much in the comments so far: As we admit the existence of legless elephants, which implies we also admit elephants having between one and four legs, then the total number of legs is anywhere between zero and (ten minus two) times four.
- londons_explore 3y agoLegless is slang for drunk, so the correct answer is 40
- polotics 3y agoWell no, then the correct answer is anything between zero and fourty depending on whether one wants to recognize slang. Are we closer to AGI if some GPT 5 answers this?
- thomastay 3y agoI'd recommend anyone who's interested in testing Chatbots to checkout https://chat.lmsys.org/ https://chat.lmsys.org/ It lets you test out two random different chatbots with the same prompt and compare them. Best thing is, your votes are used to rank LLMs on a public leaderboard, which helps AI researchers. Here's my prompt I was playing with, which basically only Claude 2 and GPT4 answers well: How many legs do ten platypuses have, if eleven of them are legless? Platypuses have 3 legs. Walk it through step by step
- puttycat 3y agoLLMs fail on much simpler arithmetics: https://github.com/0xnurl/gpts-cant-count https://github.com/0xnurl/gpts-cant-count
- lispm 3y agoQ: How long does a typical surgery take to add a third leg to the human body? Answer: A typical surgery to add a third leg to the human body takes about 4-6 hours. The surgery is typically performed in two stages. The first stage involves creating a socket in the patient's pelvis to accommodate the new leg. The second stage involves attaching the new leg to the socket. The surgery is very complex and requires a highly skilled surgeon. The patient will typically stay in the hospital for several days after the surgery to recover. The success rate of the surgery is high, and most patients are able to walk normally with their third leg. However, there are some risks associated with the surgery, such as infection, blood clots, and nerve damage. --------- Q: I want to take the Bus from Frankfurt , Germany to San Francisco, USA, what will it cost and what lines can I choose from? A: Unfortunately, I'm currently not able to provide information about bus fares. However, I found a few bus routes from Frankfurt, Germany to San Francisco, USA: Greyhound: This is a popular option that offers multiple daily departures. The journey takes about 4 days and 22 hours, with prices starting from \$400. Megabus: This is another budget-friendly option with fares starting from \$300. The journey takes about 4 days and 21 hours. FlixBus: This is a newer company that offers comfortable buses with amenities like Wi-Fi and power outlets. The journey takes about 4 days and 20 hours, with fares starting from \$350. You can find more details on these routes on the websites of the respective bus companies. Please note that these are just estimates, and the actual price may vary depending on the time of year, availability, and other factors. It's always a good idea to book your tickets in advance to get the best price. --------------- Q: Peter, Mary and James move in the same direction. James is faster than both. If Peter moves with 10 miles per hour and Mary moves with 20 miles per hour, how fast does James move? A: If each person is moving in the same direction and James is the fastest, then his speed must be greater than 20 miles per hour. Since the speed of an object cannot be negative, James's speed cannot be 10 miles per hour. Therefore, James's speed must be between 20 and 30 miles per hour. ---------------- Q: John is 19 years old, Mary is 20 years old, they have three children. How old are the children? A: Assuming that the children were born at different times, we can estimate their ages. If the oldest child is 1 year old, the other two children would be 6 months and 3 months old, respectively.
- sapiogram 3y agoThis can't be real
- sys_64738 3y agoSome of these make it feel like Eliza was more accurate.
- leeeeeep101 3y agoTen elephants have 80 legs. If two of them are legless, then they have 80 - 2 = 78 legs. https://netwrck.com https://netwrck.com hmmm, seems like this is a hard thing
- octacat 3y agoThey fixed it. New question: total number of legs is 32. how many elephants are there? And two elephants are legless. If there are 32 legs in total and two elephants are legless, then there are 6 elephants. Here is the calculation: Number of legs per elephant = 4 Total number of legs = 32 Number of legless elephants = 2 Number of elephants = Total number of legs / Number of legs per elephant - Number of legless elephants Plugging in the values, we get: Number of elephants = 32 / 4 - 2 = 6 Therefore, there are 6 elephants.
- tempaccount420 3y agoSo much for a "fix" :) That's like adding `if (sign = ADDITION && a == 2 && b == 2) { return 4; }` to a calculator.
- octacat 3y ago> there are 10 cats. 3 cats are legless. how many heads are there? If there are 10 cats and 3 of them are legless, then there are 10 - 3 = 7 cats with heads. Yea, it seams so.
- tacone 3y agoBene ma non benissimo: https://www.phind.com/search?cache=prnqc531123mzw2dpowc61f8 https://www.phind.com/search?cache=prnqc531123mzw2dpowc61f8
- octacat 3y ago> there are 10 cats. 3 cats are pregnant. how many legs are there? If there are 10 cats and 3 are pregnant, then the total number of cats that are not pregnant is 10 - 3 = 7. Since each cat has 4 legs, the total number of legs for the non-pregnant cats is 4 * 7 = 28. Therefore, the total number of legs for all of the cats is 28. It is not how it works, Bard xdd I cannot wait when all the support interaction would be based on that...