5 ms·
Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, hones
by andrewflnr 1y ago
Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly.
Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.
- srekhi 1y agoI'm not following this either. You'd think this would be frontier back in 2023
- kenjackson 1y agoFirst, I don't think we will ever get to AGI. Not because we won't see huge advances still, but AGI is a moving ambiguous target that we won't get consensus on. But why does this paper impact your thinking on it? It is about budget and recognizing that different LLMs have different cost structures. It's not really an attempt to improve LLM performance measured absolutely.
- _heimdall 1y agoSo you don't expect AGI to be possible ever? Or is your concern mainly with the wildly different definitions people use for it and that we'll continue moving goal posts rather than agree we got there?
- nutjob2 1y agoThere's no concrete evidence AGI is possible mostly because it has no concrete definition. It's mostly hand waving, hype and credulity, and unproven claims of scalability right now. You can't move the goal posts because they don't exist.
- ashirviskas 1y agoWell, if a human is GI, we just need to make it Artificial. Easy.
- abalashov 1y agoI like to say that it's not AI -- it's just A.
- _heimdall 1y agoGot it, and yeah I agree with you there. I've been frustrated by a different view of it though, many people seem to have a definition and they are often wildly different.
- dahcryn 1y agoeven AI does not have a concrete definition. Doesn't mean there aren't practical definitions depending on the context. In essence, teaching an AI using recources meant for humans, and nothing more, would be considered AGI. That could be a practical definition, without needing much more rigour. There is indeed no evidence we'll get there. But there is also no evidence LLM's should work as well as they do
- ACCount37 1y agoI can totally see "it's not really AGI because it doesn't consistently outperform those three top 0.000001% outlier human experts yet if they work together". It'll be a while until the ability to move the goalposts of "actual intelligence" is exhausted entirely.
- 9dev 1y agoWell right now, my niece of 7 years outperforms all LLM contenders in drawing a Pelican on a bicycle
- kenjackson 1y agoI know this was a joke, but LLMs are quite good at this now. If your niece draws better then she’s a good artist.
- neuronexmachina 1y agoI tried it in Gemini just now, it seems to have done a decent job: https://g.co/gemini/share/b6fef8398c01 https://g.co/gemini/share/b6fef8398c01
- baq 1y agoGiven OpenAI definition I’d expect AGI to be around in a decade or two. I don’t expect skynet, though maybe it’s a more realistic vision outcome that just droids mixing with humans.
- guluarte 1y agoI'm starting to think that there will not be an 'AGI' moment, we will simply slowly build smarter machines over time until we realize there is 'AGI'. It would be like video calls in the '90s everybody wanted them, now everybody hates them, lmao.
- nutjob2 1y agoOr we'll realize that human intelligence and machine intelligence is apple and oranges.
- jibal 1y agoLLMs are not on the road to AGI, but there are plenty of dangers associated with them nonetheless.
- nicce 1y agoJust 2 days ago Gemini 2.5 Pro tried to recommend me tax evasion based on non-existing laws and court decisions. The model was so charming and convincing, that even after I brought all the logic flaws and said that this is plain wrong, I started to doubt myself, because it is so good at pleasing, arguing and using words. And most would have accept the recommendation because the model sold it as less common tactic, while sounding very logical.
- roywiggins 1y ago> even after I brought all the logic flaws and said that this is plain wrong Once you've started to argue with an LLM you're already barking up the wrong tree. Maybe you're right, maybe not, but there's no point in arguing it out with an LLM.
- nicce 1y agoThere are cases when they are actually correct, instead of the human.
- roywiggins 1y agoYes, and there's a substantial chance they'll apologize to you anyway even when they were right. There's no reason to expect them to be more likely to apologize when they're actually right vs actually wrong- their agreeableness is really orthogonal to their correctness.
- nicce 1y agoYes, they over-apologize. But my main reason for using LLMs is seeking out things that I missed myself or my own argumentation was not good. Sometimes they are really good at bringing new perspectives. Whether they are correct or incorrect is not the point - are they giving argument or perspective that is worth inspecting more with my own brains?
- yahoozoo 1y agoThat and LLMs are seemingly plateauing. Earlier this year, it seemed like the big companies were releasing noticeable improvements every other week. People would joke a few weeks is “an eternity” in AI…so what time span are we looking at now?
- andrewflnr 1y agoThat's just the thing. There don't seem to have been any breakthroughs in model performance or architecture, so it seems like we're back to picking up marginal reductions in cost to make any progress.
- muldvarp 1y agoThere have been very large improvements in code generation in the last 6 months. A few weeks without improvement are not necessarily a plateau.
- ACCount37 1y agoWait until it ramps up so much that people will say "it's a plateau, for real this time" when they go 3 days without a +10% capability jump.
- muldvarp 1y agoI mean I wish there were a plateau, without one we're well onto our way into techno-feudalism. I just don't see it.
- ACCount37 1y agoThat's what it is: wishful thinking. A lot of people really, really want AI tech to fail - because they don't like the alternative.
- muldvarp 1y agoYeah, obviously nobody that actually though about the consequences wants a large part of the population to become unemployed. Even if your job is not threatened by automation, it will be threatened by a lot of people looking for new jobs. And the kind of automation brought by LLMs is decidely different than automation in the past which almost always created new (usually better) jobs. LLMs won't do this (at least to extent where it would matter) I think. Most people in ten years will have worse jobs (more physically straining, longer hours, less pay) unless there will be a political intervention.
- yieldcrv 1y agojust because it’s on arxiv doesn’t mean anything arxiv is essentially a blog under an academic format, popular amongst asian and south asian academic communities currently you can launder reputation with it, just like “white papers” in the crypto world allowed for capital for some time this ability will diminish as more people catch on
- dahcryn 1y agoarxiv should really have a big red banner "NOT REVIEWED - DON'T USE AS A SOURCE" or something
- ctoth 1y agoIs a random paper from Fujitsu Research claiming to be the frontier of anything?
- andrewflnr 1y agoNot just this paper, but model working shenanigans also seem to have been a big part of GPT-5, which certainly claims to be frontier work.