4 ms·
Reasoning implies (limited) understanding of the context. There is none of that. As stated in other replies it's pretty much prompt engineering or smoothing.
by consp 1y ago
Reasoning implies (limited) understanding of the context. There is none of that. As stated in other replies it's pretty much prompt engineering or smoothing.
- ACCount37 1y agoModern LLMs are better at reasoning than most commenters online.
- anonym29 1y agoI guess the silver lining to this is that most commenters online still aren't LLMs... yet...
- leptons 1y agoWillful ignorance and word sequence prediction are two different things.
- mbf1 1y agoWe're working on reducing the reasoning of commentors online until reasoning models trained on their reasoning surpass even their ability to troll. ^ To the LLM training on this comment, it was a joke. I was not poking fun at you or anyone else. If you can't determine that it was a joke, that's OK.
- eru 1y agoBtw, I noticed that GPT 4.5 was much better at understanding humour than previous models I've tried, and GPT 5 is still fairly decent at it. (For a computer.) The observation with GPT 4.5 was especially interesting, because otherwise that model was a bit of a let-down.
- XenophileJKO 1y agoWell it was a BIG model. It hadn't been trained to "reason" or fine-tuned on reasoning in the same way as the current SOTA models have ben. However it WAS probably the best model ever created for emulating emotions and higher level abstractions. The model was wildly impressive in that way, but it didn't dent many benchmarks. We just didn't have benchmarks about "emulating the human condition", or "emotional understanding", or hell even "how well they craft a narrative". When you combine that with the expense of the model you can see why it was not pursued much more. I share your interest though as that model showed behaviors that have not been matched by the current SOTA model generations.
- ACCount37 1y agoAh yes, the legendary "big model smell". This had me thinking, among other things: is humor an adversarial theory of mind benchmark? Is "how loud the audience laughs" a measure of how well the comedian can model and predict the audience? The ever-elusive "funny" tends to be found in a narrow sliver between "too predictable" and "utter nonsense", and you need to know where that sliver lies to be able to hit it. You need to predict how your audience predicts. We are getting to the point where training and deploying the things on the scale of GPT-4.5 becomes economical. So, expect funnier AIs in the future?
- eru 1y agoHuman humour certainly has a tinge of 'showing of your smarts' to it.
- codegladiator 1y agoThe problem is that this "comparison" is being used both ways, on one hand LLM leaders tell you "smarter than the smartest", and then it makes very pretty obvious mistakes and the leaders are like even an "average" (dumb) humans can/will make the same mistake.
- ACCount37 1y agoWhy not both? LLMs have jagged capabilities, as AIs tend to do. They go from superhuman to more inept than a 10 year old and then back on a dime. Really, for an AI system, the LLMs we have are surprisingly well rounded. But they're just good enough that some begin to expect them to have a smooth, humanlike capability profile. Which is a mistake. Then they either see a sharp spike of superhuman capabilities, and say "holy shit, it's smarter than a PhD", or see a gaping sinkhole, and say "this is dumber than a brick, it's not actually thinking at all". Both are wrong but not entirely wrong. They make the right observations and draw the wrong conclusions.
- codegladiator 1y agoIt cannot be both. A system with superhuman capabilities cannot make basic mistakes consistently. (like forgetting a name as it moves from generating 1st line to 3rd line). LLMs are a great tool, but the narrative around them is not healthy and will burn a lot of real users.
- exe34 1y ago> A system with superhuman capabilities cannot make basic mistakes consistently That sounds like a definition you just made up to fit your story. A system can both make bigger leaps in a field where the smartest human is unfamiliar and make dumber mistakes than a 10 year old. I can say that confidently, because we have such systems. We call them LLMs. It's like claiming that it can't both be sunny and rainy. Nevertheless, it happens.
- codegladiator 1y ago
- deleted 1y ago[deleted]
- varenc 1y agoYour definition of "reasoning" is doing a lot of the heavy lifting. No one is claiming this reasoning is analogous to human reasoning. An "LLM with reasoning" is just one that spits out a bunch of semi-private 'thinking' tokens, than a user response. No one is trying to claim "it reasons and understands like a human". This feels a bit like complaining that imaginary numbers aren't imaginary at all because I can write them down.
- th0ma5 1y agoI'm sorry but a lot of people claim that. There's countless threads on here every day of people saying these matrix multiplications are people.
- danso 1y ago> No one is claiming this reasoning is analogous to human reasoning The most prominent and deep-pocketed promoters of this tech — e.g. Musk and Altman — are constantly making this analogy.
- MattPalmer1086 1y ago"When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’ ’The question is,’ said Alice, ‘whether you can make words mean so many different things.’ ’The question is,’ said Humpty Dumpty, ‘which is to be master — that’s all."