4 ms·
LLMs aren’t capable of “comprehending” anything. They never “know” what they are outputting, they’re simply really good at being lucky. They are not lucky enoug
by devjab 2y ago
LLMs aren’t capable of “comprehending” anything. They never “know” what they are outputting, they’re simply really good at being lucky. They are not lucky enough to be useful unless you’re already an expert on the topic you’re using them on so that you can spot when they aren’t lucky.
This is part of why many enterprise organisations are banning their usage. It’s one thing to use them to build software poorly, the world is already used to IT not working very often. It’s another thing to produce something that has real world consequences. Our legal department used them in a PoC for contract work, and while they were useful very often they also sometimes got things very wrong. Unlike a slow IT system, this would have business shattering consequences. You can continue training your model as well as reigning it in when it gets unlucky, but ultimately you can never be sure it’s never unlucky, and this means that LLMs are useless for a lot of things. We still use them to make pretty PowerPoint presentations and so on, but again, this is an area where faults are tolerable.
- deleted 2y ago[deleted]
- Eisenstein 2y agoI don't believe that current models have the capability to 'comprehend', I was using the term loosely. However I find that people tend to go to extremes when they want to make the point that language models are not 'intelligent' by minimizing their capability and complexity behind 'it is just math', which I think is unhelpful because it merely acts as a 'though-terminating cliche'.
- devjab 2y agoI think the issue is far more psychological than technical personally. One of the issues we struggle with for our junior developers is that they are far more likely to believe an LLM than what they might find Google programming. I do wonder why we’ve ended up with CS graduates who go to LLMs and Search engines before the official documentation which is often very, very good, but I guess that’s a different discussion. I’m not personally against LLM assistance, I use it for programming and it has in many places replaced my usage of snippets completely. This is probably why I’m not really a fan of the “knowledge” part that LLMs are increasingly tasked to do. Because when you use them for programming you’ll get an accrue insight into how terrible they can be when they get things wrong.
- Eisenstein 2y agoAt this point I think what is happening some people either have a natural inclination or they spend time to learn how to use them productively, while others question their usefulness. People scoff at 'prompt engineering' but I see how some of my peers use LLMs and I think to myself 'how do they expect to get a good answer from that?' It doesn't help that google is now mostly full of SEO nonsense, and technical documentation is impenetrable when you are looking for something specific but don't know enough about the system to know how to look for it.
- jannyfer 2y agoYou and I are a mix of molecules arranged in a particular way that responds to electrical, physical, and chemical inputs. It’s entirely possible than an LLM will do something that can be defined as “comprehending” something.
- numpad0 2y agoDoesn't mean LLM architecture is complex enough. Clearly it's far from the ultimate human invention.
- mistermann 2y ago> It’s entirely possible than an LLM will do something that can be defined as “comprehending” something. Agreed, but not even necessary. Each Human uses an adequately customized meaning of "comprehending" when they "comprehend" this problem space such that their belief is always "true". This is how Humans are able to produce numerous "true" statements that disagree with each other. If you disagree, just ask one of them and they will "inform" you of the "logic" behind their version. Joking aside though: if Humans are unable to comprehend past a certain level of complexity, I wonder what will happen once AI starts going beyond our abilities, both at the individual level and the culturally conditioned ("the" "reality") level. And, when is this going to really start kicking in?
- famouswaffles 2y ago>LLMs aren’t capable of “comprehending” anything. They never “know” what they are outputting, they’re simply really good at being lucky. The mental gymnastics people will go through to discount LLMs is wild. This does not even make any sense. "really good at being lucky". What does that even mean ?
- lcnPylGDnU4H9OF 2y ago> "really good at being lucky". What does that even mean ? They mean good at being lucky the way card counters playing 21 are good at being lucky.
- famouswaffles 2y agoThere is almost nothing 'lucky' about how good those kind of players are.