5 ms·
> LLMs/transformers make mistakes in different ways than humans do Sure but I don't think this is an example of it. If you show people a picture and ask "how m
by jxjnskkzxxhx 1y ago
> LLMs/transformers make mistakes in different ways than humans do
Sure but I don't think this is an example of it. If you show people a picture and ask "how many legs does this dog have?" a lot of people will look at the picture, see that it contains a dog, and say 4 without counting. The rate at which humans behave in this way might differ from the rate at which llms do, but they both do it.
- DeathRay2K 1y agoI don’t think there’s a person alive who wouldn’t carefully and accurately count the number of legs on a dog if you ask them how many legs this dog has. The context is that you wouldn’t ask a person that unless there was a chance the answer is not 4.
- tantalor 1y agoYou deeply overestimate people. The models are like a kindergartner. No, worse than that, a whole classroom of kindergartners. The teacher holds up a picture and says, "and how many legs does the dog have?" and they all shout "FOUR!!" because they are so excited they know the answer. Not a single one will think to look carefully at the picture.
- jxjnskkzxxhx 1y agoIt's hilarious how off you are.
- ekianjo 1y agoYou have never seen the video of the gorilla in the background?
- petesergeant 1y agoThat's a specific example that when you draw a human's attention to something (eg: count the number of ball passes in this video), they hyper-fixate on that, to the exclusion of other things, so it seems like it makes the opposite point that I think you're trying to?
- petesergeant 1y agoExactly this. Humans are primed for novelty and being quizzed about things.
- freeone3000 1y agoOk? But we invented computers to be correct. It’s suddenly ok if they can look at an image and be wrong about it just because humans are too?
- jxjnskkzxxhx 1y agoMy point is that these llms are doing something that our brain also is doing. If you don't find that interesting, I can't help you.
- freeone3000 1y agoWell, they’re getting the same result. I don’t particularly see why that’s useful.
- HeatrayEnjoyer 1y agoAll automation has ever been is an object doing something that a human can do, without needing the human.
- freeone3000 1y agoThe result is still wrong, though! It needs to be right to be useful!
- proc0 1y agoThe analogy should be of an artist that can draw dogs but when you ask them to draw a dog with three legs they completely fail and have no idea how to do it. That likelihood is really low. A trained artist will give you exactly what you ask for, meanwhile GenAI models can produce beautiful renders but fail miserably when asked for certain specific but simple details.
- jxjnskkzxxhx 1y agoNo, the example in the link is asking to count the number of legs in the pic.
- proc0 1y agoOk, sure, but I'm trying to point out the gap in expectation, i.e. it's an expert artist but it cannot fulfill certain specific but simple requests.