4 ms·
It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is l
by b112 1y ago
It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck.
That doesn't mean it knows the answer. That means it guessed or hallucinated correctly. Guessing isn't knowing.
edit: people seem to be missing my point, so let me rephrase. Of course AIs don't think, but that wasn't what I was getting at. There is a vast difference between knowing something, and guessing.
Guessing, even in humans, is just the human mind statistically and automatically weighing probabilities and suggesting what may be the answer.
This is akin to what a model might do, without any real information. Yet in both cases, there's zero validation that anything is even remotely correct. It's 100% conjecture.
It therefore doesn't know the answer, it guessed it.
When it comes to being correct about a language or API that there's zero info on, it's just pure happenstance that it got it correct. It's important to know the differences, and not say it "knows" the answer. It doesn't. It guessed.
One of the most massive issues with LLMs is we don't get a probability response back. You ask a human "Do you know how this works", and an honest and helpful human might say "No" or "No, but you should try this. It might work".
That's helpful.
Conversely a human pretending it knows and speaking with deep authority when it doesn't is a liar.
LLMs need more of this type of response, which indicates certainty or not. They're useless without this. But of course, an LLM indicating a lack of certainty, means that customers might use it less, or not trust it as much, so... profits first! Speak with certainty on all things!
- lechatonnoir 1y agoThis is such a pointless, tired take. You want to say this guy's experience isn't reproducible? That's one thing, but that's probably not the case unless you're assuming they're pretty stupid themselves. You want to say that it Is reproducible, but that "that doesn't mean AI can think"? Okay, but that's not what the thread was about.
- PeterStuer 1y agoWhat would convince you otherwise? The reason I ask is because you sound like you have made up your mind phylosophically, not based on practical experience.
- rsanheim 1y agoIt's just Pattern matching. Most APIs, and hell, most code is not unique or special. Its all been done a thousands of times before. Thats why an LLM can be helpful on some tool you've written just for yourself and never released anywhere. As to 'knows the answer', I'm don't even know what that means with these tools. All I know is if it is helpful or not.
- danielbln 1y agoAlso, most problems are decomposable into simpler, certainly not novel parts. That intractable unicorn problem I hear so much about is probably composed of very pedestrian sub-problems.
- genewitch 1y agoto what does "unicorn problem" refer to? a specific thing or a general idea?
- danielbln 1y agoGeneral idea, it comes up often in LLM discussion from engineers lamenting that LLMs can never solve any novel problems. This might as well be true but as said, almost any problem can be decomposed into bite sized, non-novel parts.
- CamperBob2 1y ago'Pattern matching' isn't just all you need, it's all there is.
- deleted 1y ago[deleted]
- jumploops 1y ago> It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. The amazing thing about LLMs is that we still don’t know how (or why) they work! Yes, they’re magic mirrors that regurgitate the corpus of human knowledge. But as it turns out, most human knowledge is already regurgitation (see: the patent system). Novelty is rare, and LLMs have an incredible ability to pattern match and see issues in “novel” code, because they’ve seen those same patterns elsewhere. Do they hallucinate? Absolutely. Does that mean they’re useless? Or does that mean some bespoke code doesn’t provide the most obvious interface? Having dealt with humans, the confidence problem isn’t unique to LLMs…
- skydhash 1y ago> The amazing thing about LLMs is that we still don’t know how (or why) they work! You may want to take a course in machine learning and read a few papers.
- dboreham 1y agoSuspect you and the parent poster are thinking on different levels.
- js8 1y agoSorry, but that's reductionism. We don't know how human brain works, and that you won't get there by studying quantum electrodynamics. LLMs are insanely complex systems and their emergent behavior is not explained by the algorithm alone.
- whateverbrah 1y agoThat was sarcasm by the poster, in case you failed to notice.
- semiquaver 1y agoParent is right. We know mechanically how LLMs are trained and used but why they work as well as they do is very much not known.
- 1y ago
- hombre_fatal 1y agoThis doesn't seem like a useful nor accurate way of describing LLMs. When I built my own programming language and used it to build a unique toy reactivity system and then asked the LLM "what can I improve in this file", you're essentially saying it "only" could help me because it learned how it could improve arbitrary code before in other languages and then it generalized those patterns to help me with novel code and my novel reactivity system. "It just saw that before on Stack Overflow" is a bad trivialization of that. It saw what on Stack Overflow? Concrete code examples that it generalized into abstract concepts it could apply to novel applications? Because that's the whole damn point.
- skydhash 1y agoProgramming languages, by their nature of being formal notation, only have a few patterns to follow, all of them listed in the grammar of that language. And then there’s only so much libraries out there. I believe there’s more unique comments and other code explanations out there than unique code patterns. Take something like MDN where there’s a full page of text for every JavaScript, html, css symbol.
- demosthanos 1y agoThis is wrong. I write toy languages and frameworks for fun. These are APIs that simply don't exist outside of my code base, and LLMs are consistently able to: * Read the signatures of the functions. * Use the code correctly. * Answer questions about the behavior of the underlying API by consulting the code. Of course they're just guessing if they go beyond what's in their context window, but don't underestimate context window!
- b112 1y agoSo, you're saying you provided examples of the code and APIs and more, in the context window, and it succeeds? That sounds very much unlike the post I responded to, which claimed "no knowledge". You're also seemingly missing this: "If you're getting answers, it has seen it elsewhere" The context window is 'elsewhere'.
- demosthanos 1y agoIf that's the distinction you're drawing then it's totally meaningless in the context of the question of where the information is going to come from if not Stack Overflow. We're never in a situation where we're using an open source library that has zero information about it: The code is by definition available to be put in the context window. As they say, it sounds like you're technically correct, which is the best kind of correct. You're correct within the extremely artificial parameters that you created for yourself, but not in any real world context that matters when it comes to real people using these tools.
- fnordpiglet 1y agoThe argument is futile as the goal posts move constantly. In one moment the assertion is it’s just megacopy paste, then the next when evidence is shown that it’s able to one shot construct seemingly novel and correct answers from an api spec or grammar never seen before, the goal posts move to “it’s unable to produce results on things it’s never been trained on or in its context” - as if making up a fake language and asking it write code in it and its inability to do so without a grammar is an indication of literally anything. To anyone who has used these tools in anger it’s remarkable given they’re only trained on large corpuses of language and feedback they’re able to produce what they do. I don’t claim they exist outside their weights, that’s absurd. But the entire point of non linear function activations with many layers and parameters is to learn highly complex non linear relationships. The fact they can be trained as much as they are with as much data as they have without overfitting or gradient explosions means the very nature of language contains immense information in its encoding and structure, and the network by definition of how it works and is trained does -not- just return what it was trained on. It’s able to curve fit complex functions that inter relate semantic concepts that are clearly not understood as we understand them, but in some ways it represents an “understanding” that’s sometimes perhaps more complex and nuanced than even we can. Anyway the stochastic parrot euphemism misses the point that parrots are incredibly intelligent animals - which is apt since those who use that phrase are missing the point.
- Workaccount2 1y ago>It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. How would you reconcile this with the fact that SOTA models are only a few TB in size? Trained on exabytes of data, yet only a few TB in the end. Correct answers couldn't be dumb luck either, because otherwise the models would pretty much only hallucinate (the space of wrong answers is many orders of magnitude larger than the space of correct answers), similar to the early proto GPT models.
- efavdb 1y agoCould it be that there is a lot of redundancy in the training data?
- daveguy 1y ago> How would you reconcile this with the fact that SOTA models are only a few TB in size? Trained on exabytes of data, yet only a few TB in the end. This is false. You are off by ~4 orders of magnitude by claiming these models are trained on exabytes of data. It is closer to 500TB of more curated data at most. Contrary to popular belief LLMs are not trained on "all of the data on the internet". I responded to another one of your posts that makes this false claim here: https://news.ycombinator.com/item?id=44283713 https://news.ycombinator.com/item?id=44283713
- gejose 1y agoI'm sorry but this is a gross oversimplification. You can also apply this to the human brain. "<the human brain> cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck."