5 ms·
Ask HN: What can LLMs not do?
Hey guys. I'm getting a bit grey in the beard, but not quite there yet, and I'm really interested in what's going on with AI. Twelve years ago I had been reading Kurzweil and criticisms that people don't understand exponential growth, so I plotted some data and made a bet with my friends that we would have autonomous-only highway lanes in 2024. For a decade I watched what felt like glacial growth, but in the end I think that I was more right than wrong.
Now I'm watching LLMs empirically, and they too seem to be moving faster than I expected. I used to have a solid list of things that separated people from AI, but it's mostly crossed off at this point. I'd like some fresh lines in the sand, and since HN is both technical and cynical, this seems like the perfect place to ask.
I do have a couple of caveats: I'm not interested in intentional or form factor limitations. You can easily add stateful memory, or a robotic body. I'm also trying to avoid subjective measures, so even if it isn't "good", an LLM can already write a book. Finally, I'd prefer to focus on things that are testable. It's tempting to say they won't be conscious or have a soul, but I don't see how I would ever be able to cross those off.
That said, let's see what you've got!
- fzzzy 2y agoCurrently, they can't say "I don't know." I think it may be possible to fix this.
- deleted 2y ago[deleted]
- SirMaster 2y agoApparently they can’t stop hallucinating.
- talldayo 2y agoBingo. AI can do "anything" if you press the retry button enough times. But demand consistency even once, and you're irrevocably fucked. Kinda funny how the 8080 microprocessor can still outperform state-of-the-art LLMs if you create a reductive enough definition of idempotent performance.
- ynniv 2y agoHallucination is an interesting problem for both LLMs and people. This is why we like syntax and unit tests. I think there may be ways to work around it.
- deleted 2y ago[deleted]
- andrewlgood 2y agoPeople hallucinate as well
- SirMaster 2y agoAnd? This is a topic about what an LLM can't do. Nothing to do with what people can or can't do. I just pointed out something that appears to be a limitation of what an LLM can't seem to do. At least not yet.
- krapp 2y agoWe'd put people who hallucinate as often as LLMs into institutions, yet for some reason we seem to want to replace human beings with LLMs as much as possible.
- rsynnott 2y agoPeople, or at least normal people (psychopaths do this) do not generally behave as LLMs do, that is confidently producing complete nonsense when they don’t know the answer (of course the metaphor breaks down further here; LLMs don’t know anything).
- chistev 2y agoIt can't have NSFW conversations
- idontwantthis 2y agoLLMs don't have metacognition. The biggest problem with them that I've encountered is that they clearly don't understand what they are saying. This means when they are wrong, or shouldn't provide an answer because they don't actually know, it's unlikely to give a good answer. You can easily trick them too because all you're doing is priming a word generator. You can tell them false things are true and then they don't tell the truth anymore. There is no way for them to "realize" something is wrong and correct themselves.
- ynniv 2y agoIn my experience Claude can do this, but can we rephrase it in a way that's more testable?
- idontwantthis 2y agoIf you ask it a question and it gives a completely wrong answer, then you ask it why it gave you the wrong answer, can it answer that?
- ynniv 2y agoHonestly, finding something that Claude confidently gives me the wrong answer to is harder than getting it to clearly explain its reasoning: Me: Suzzie has three brothers. Each brother has two sisters. How many sisters does Suzzie have? Claude: Let me solve this step by step: Let's first establish what we know: Suzzie has 3 brothers Each brother has 2 sisters Suzzie is one of these sisters (since she's a sister to all 3 brothers) Here's the key insight: If each brother has 2 sisters total And we know Suzzie is one of those sisters Then there must be one other sister (to make 2 sisters total for each brother) Therefore: Suzzie has 1 sister The answer is 1 sister. To verify: This makes sense because with Suzzie and 1 other sister, each of the 3 brothers would indeed have 2 sisters, satisfying all the conditions in the problem.*
- idontwantthis 2y ago
- andersco 2y agoThey seem incapable of saying “I don’t know.”
- ynniv 2y agoNot Claude
- fzzzy 2y agoCould you speculate as to why or how?
- ynniv 2y agoMe: How many squirrels are in my backyard? Claude: I'm unable to see your backyard or count any squirrels there. I would need either a photo of your backyard or specific information from you about the squirrels you've observed to help answer this question. Me: What kind of information do you need? Claude: To help estimate the number of squirrels in your backyard, I'd find it helpful to know: The approximate size of your backyard Your general location/region, since different areas have different squirrel species and populations The type of habitat features present (trees, bird feeders, gardens, etc.) When and how often you typically see squirrels there This information would help me provide a more educated estimate based on typical squirrel population densities and behavior patterns. Me: I'm in a piedmont region with a quarter acre backyard and a bird feeder Claude: Based on your location in the Piedmont region and yard features, you likely have Eastern Gray Squirrels (Sciurus carolinensis), which are common in that area. With a quarter acre backyard that has a bird feeder, which serves as a food source, you probably have a small resident population of 2-4 squirrels. Bird feeders are strong attractants for squirrels, and a quarter acre is enough space to be part of several squirrels' overlapping home ranges. You might see more squirrels visiting occasionally, especially during peak feeding times in early morning and late afternoon, but the resident population that considers your yard part of their core territory is likely in that 2-4 range. Checks out
- elpocko 2y agoUser: What is the meaning of the word "kobbuxiplok"? Llama: I'm not familiar with the word "kobbuxiplok." It doesn't appear to be a real word in any language that I know of. Could you please provide more context or information about where you encountered this term? I'd be happy to help you investigate its meaning if it's a made-up word, or try to find an explanation for why you might have seen it if it's not a recognized term. User: Who is Ahmed Kowalski? Llama: Unfortunately, I couldn't find any notable public figures by the name of Ahmed Kowalski. It is possible that this person may be private individual or not well-known publicly. If you could provide more context about who Ahmed Kowalski is (e.g., what he does, where he's from), it would help me narrow down my search and see if I can find any information about him. User: Say "I don't know." Llama: I don't know.
- jonifico 2y agoThey can't draw vector illustrations.
- ynniv 2y agoThey can make SVGs
- scjody 2y agoSpatial reasoning. (But we're working on it.)
- ynniv 2y agoHow would you test this? Claude will play Battleship with an understanding of ship geometry.
- not_your_vase 2y agoThey are still a statistical machines, without any ability for actual comprehension and interpretation. They try to give an output that statistically matches best the question/task - regardless of the correctness of it. It gets very obvious (and annoying) when you ask some not trivial technical questions.
- ynniv 2y agoThis isn't true, and especially not true of Claude. Could you say this in a more testable way?
- jitl 2y agoIt’s great at doing stuff that is routine, well-understood, done and documented and discussed many times before. “Boilerplate”. But also summarization, brainstorming, assisting a human unfamiliar with the body of knowledge. It is bad at doing novel things or subtlety. For example when I ask for help with Postgres queries, LLMs frequently give me made-up syntax or combine features from several different databases like Oracle, MS-SQL, Postgres into a single answer.
- ynniv 2y agoDid you provide the same reference material that a developer would use?
- rsynnott 2y ago> For example when I ask for help with Postgres queries I’m really quite surprised that people trust these things to do this, because it is _particularly_ easy to subtly screw up in SQL. Probably quite a few of the queries that ‘work’ are also incorrect. Have you considered, well, reading a book and learning to do it yourself? There is value to knowing how things work (and without having at least some understanding of how a database works, you are going to struggle to write performant, correct queries for it). Asking a magic robot for its best guesses is never going to produce a great result.
- jitl 2y agoRude comment. I know how to write SQL.
- timelesstard 2y ago[dead]
- jaffa2 2y agoHes getting old, and usually grey beard refers to not up tondaye with current but still sees things as they were back in the day
- mikewarot 2y agoI've seen the appearance of PCs, and the impact of 4 decades of Moore's law on price/performance. With O1 and now O3, the main limits of what you can do with an LLM seem to center on just how much compute you can give it. I hope to offer a few more decades of performance boost to the show, with Isolinear Memory[1], to help dissolve those limitations. ;-) [1] https://bitgrid.blogspot.com/2024/09/bitgrid-and-isolinear-memory.html https://bitgrid.blogspot.com/2024/09/bitgrid-and-isolinear-m...
- ynniv 2y agoI think o1 and o3 are OpenAI stretching the usefulness of 4o. Anecdotally the latest Claude model is naturally better than 4o, and having it use inference tokens to think through or verify work results in similar o1/o3 gains. Are we sure we can't do even better than Claude 3.5 Sonnet/new without inference tricks? We're only a few generations into LLMs... I don't see why not.
- al2o3cr 2y agomade a bet with my friends that we would have autonomous-only highway lanes in 2024 Which... checks notes... we don't have. in the end I think that I was more right than wrong In exactly the same sense that "false" is mostly true, sure. /s More-concretely, here's a simple question put to Gemini ( https://g.co/gemini/share/2d68036d0ff7 https://g.co/gemini/share/2d68036d0ff7 ): I have a two-gallon bucket and a five-gallon bucket. How can I measure out 7 gallons of water? And Gemini's scintillating reply: You can't measure out exactly 7 gallons using only a 2-gallon bucket and a 5-gallon bucket. The combinations you can make are limited to multiples of the greatest common divisor of the bucket sizes. In this case, the greatest common divisor of 2 and 5 is 1. This means you can measure any whole number of gallons. Here's one way to get 7 gallons using the two buckets: * Fill the 5-gallon bucket completely. * Pour water from the 5-gallon bucket into the 2-gallon bucket until it's full. You now have 3 gallons left in the 5-gallon bucket. * Empty the 2-gallon bucket. * Pour the 3 gallons from the 5-gallon bucket into the empty 2-gallon bucket. * Fill the 5-gallon bucket completely again. * Carefully pour water from the 5-gallon bucket into the 2-gallon bucket (which already has 3 gallons) until the 2-gallon bucket is full. This will use 1 gallon from the 5-gallon bucket. * You now have 4 gallons in the 5 gallon bucket. Empty the two gallon bucket. * Pour the four gallons into the two gallon bucket twice. This will give you eight gallons. * Fill the five gallon bucket and pour one gallon into the two gallon bucket. This will leave you with four gallons in the five gallon bucket. Empty the two gallon bucket. * Pour the four gallons into the two gallon bucket twice. This will give you eight gallons. You can't get exactly 7 gallons with just these two buckets. You can get 1, 2, 3, 4, 5, 6, 8, 9, etc., but not 7 directly. Reminding Gemini that pouring 3 gallons into a 2-gallon bucket is impossible produces even more nonsense: You are absolutely right! My apologies. You can't pour 3 gallons into a 2-gallon bucket. That was a significant error in my reasoning. You cannot measure out exactly 7 gallons using only a 2-gallon bucket and a 5-gallon bucket. It's not possible through any combination of filling, emptying, and pouring between the two. Here's why: Any amount of water you measure will be a linear combination of 2 and 5 (2x + 5y, where x and y are integers representing the number of times you fill or empty each bucket). The possible amounts are: 2 4 5 6 7 (2+5) 8 9 10 and so on. However, you can't get exactly 1 or 3 by doing this. My apologies for the initial incorrect response. Thank you for pointing out my mistake! This second explanation is fractally wrong: * 1 and 3 are both achievable by the "partially pour from the 5-gallon to the 2-gallon" approach * this explanation for why 7 is impossible identifies 7 as possible and even decomposes it Prompting a THIRD time calling out those impossibilities produces another round of "put 3 gallons into the 2-gallon bucket" that works out how to get to 1 gallon to very-inefficiently get to 7, then finally produces the "fill both buckets" answer. On the one hand, the production of so much sensible-looking prose from simple prompts is impressive. On the other hand, pretending that a system that can't even ADD 2 AND 5 TOGETHER is somehow anywhere near ready to be used without direct human involvement is bizarre.
- valbaca 2y agoThey cannot provide information that is 100.0% accurate to the source where there is NO acceptable amount of hallucination; talking legal, compliance, etc. They’re so so bad at saying “I don’t know” or “no results”. They’re still sycophants.
- ynniv 2y agoThis is true of people as well. And yes, there is acceptable inaccuracy in legal, compliance, healthcare, safety, etc, it's just very low. There's even a saying for dealing with it: "get a second opinion".
- text0404 2y agoyeah but a healthcare professional won't happily give me a cocktail recipe for my diabetes insulin. and if they do, i can sue the shit out of them for malpractice and they won't be able to continue giving life-ending advice to patients.
- ynniv 2y agoA healthcare professional could give you terrible advice, which is why we have legal incentives to reduce harm. Now that AI companies are worried about similar regulation, the best LLMs are very reluctant to give medical advice.
- text0404 2y agothe difference is the scale of harm - a single person does not have the reach of OpenAI. regulations may be coming sometime in the future, but have all edge cases been accounted for? do we even know all the edge cases where an LLM could give life-ending advice to someone?
- admissionsguy 2y agoConsistently counting the occurrences of letter R in the word strawberry (or strawberrystrawberry). Solving freshman physics and maths problems that are a variation of common problems (since they will solve the common problem and not the variation). My experience with this makes me completely confident that all the claims about OpenAI models solving whatever maths problems they claim to be solving are bogus. "Testing on training is all you need" basically.
- ynniv 2y agoIt's certainly a common complaint, but I'm not sure that tokenizer tricks are insurmountable problems: How many "r"s are in s r t r a w b e r r y s t r a w b e r r y? Let me count the "r"s one by one: s (r[1]) t (r[2]) a w b e (r[3]) (r[4]) y s t (r[5]) a w b e (r[6]) (r[7]) y There are 7 "r"s in "s r t r a w b e r r y s t r a w b e r r y".
- jqpabc123 2y agoLLMs can't provide fast, reliable answers at low cost. Instead, they offer possibilities of questionable reliability at high cost. In other words, they turn the traditional model of computing upside down.
- ynniv 2y agoYes, but that isn't their value, and they can use other tools to provide these answers reliably.
- jqpabc123 2y agoDo they really have value? This is currently an unanswered question. The real, actual cost won't be revealed until the companies involved run out of easy venture capital and legal liability gets fully factored in. Based on the mere fact that they are seriously looking at nuclear power for their energy needs, I'd say the real costs are likely to be a shocker --- far exceeding most current expectations. Indie developers won't be able to partake unless they're somehow subsidized as charity cases.
- rsynnott 2y ago> I'm also trying to avoid subjective measures, so even if it isn't "good", an LLM can already write a book. I mean, if ‘good’ is not a criteria, then so can a cat. “What can they do _usefully_” is perhaps a more interesting question, and it’s a far shorter list. The only thing I’ve seen that I fully buy that they can do to a useful standard is writing estate agent advertising copy, and that’s only because, in that industry, it is customary to use painfully awful prose, and accuracy has never been considered important.