5 ms·
Another potential takeaway is that the models all gathering around the same point supports the idea that there is a ceiling to LLM capability.
by doginasuit 2mo ago
Another potential takeaway is that the models all gathering around the same point supports the idea that there is a ceiling to LLM capability.
- AustinDev 2mo agoThey always all gather around the same spot then that spot moves every 6-9 months. I think the clustering is more likely evidence of distillation. I don't personally think distillation is a bad thing. If the LLM providers can distill all of human output into their models for 'free'. I don't think distilling a model from the output of those models is morally wrong.
- michelsedgh 2mo agoWhat an interesting take. One question, do you think stealing from a thief is morally okay? I'm just asking no judgement on my side.
- AussieWog93 2mo agoI'd say it's more "Downloading LimeWire Pro from LimeWire" than actual theft.
- dullcrisp 2mo agoWhy isn’t it more like building a hardware store using lumber you purchased from a competing hardware store? Or founding a school using an education you obtained at a different school?
- ygjb 2mo agoBecause that doesn't satisfy the narrative of American exceptionalism. It's easier to point at something and say it was stolen or copied than it is to compete, especially with the political climate in the US. This isn't an anti-American sentiment. It is an anti-corporate/regulatory capture/embrace and extinguish sentiment (which probably reads the same to many people these days).
- Eldt 2mo agoBecause the companies did it underhandedly without prior consent? Imagine you walked into a hardware store to grab some lumber, didn't pay for it, and the store had to call the cops to swing around your house? That's hardly the typical shopping experience, now is it?
- dullcrisp 2mo agoI think that’s my point: none of those things happened, so on what basis can we claim that it’s like they did. Who has to agree with what you’re doing in order for it not to be considered underhanded?
- Gigachad 2mo agoIf the legal system declares the first thief’s theft not theft then all bets are off.
- mannanj 2mo agoIs it theft if another thief steal's the first thief's theft?
- BeetleB 2mo ago> If the legal system declares the first thief’s theft not theft But they didn't find it. The Big LLM provider accepted guilt and paid a fine. You can argue whether it was a fair amount they paid, but there is no legal precedent that was set. It's still considered theft.
- kennywinker 2mo agoAs i understand it, they accepted guilt for downloading stuff illegally. They didn’t accept guilt for incorporating all of human output into their model without consent.
- TheOtherHobbes 2mo agoCopyright law only considers illegal ownership of a work, so the crime - or tort - was making/acquiring copies without permission or payment. Training from copies has been ruled fair use because it's "transformative" and not simply "derivative." This is obviously debatable, but that's where the debate is at the moment.
- michelsedgh 2mo agoSo basically because they just browsed and used the information that was mostly public on the internet and they didnt copy it, they just learned from it and thats fine. Which makes sense. None of the llms let u copy someones work exactly anyways... makes total sense honestly. So in this case what happens to distilling? Is that also learning or ur trying to get to their actual weights by kind of reverse engineering it? Where would the argument fall there?
- itemize123 2mo agoquestion's phrasing made your judgement obvious
- michelsedgh 2mo agoI seriously wasnt judging i maybe shouldve asked llm to frame it better cause i knew it might sound that way, thats why i added: im not judging, merely asking...
- ofjcihen 2mo agoI gathered that the most recent advances haven’t been in capabilities of the model but more the way that it’s able to be employed (most recently agents).
- deleted 2mo ago[deleted]
- conception 2mo agoIf you talk to the Chinese models, even super smart Qwen 3.8, you can tell they are distilled just from the verbal ticks they have. Gemini, ChatGPT and Claude do not sound alike. The Chinese models 100% sound like one of the 3, usually Claude. American models are load bearing for this LLM generation seam.
- FuckButtons 2mo agoThat’s definitely my impression of deepseek 0731 after a fair bit of use via ds4, it sounds like Claude.
- bossyTeacher 2mo ago> LLM providers can distill all of human output into their models for 'free' Not sure what part of being charged guilty and paying a fine you see as "free".
- rlupi 2mo agoI think it's more likely to be the effect of synchronization of launches, and the fact that models that do not challenge SOTA in some way do not get launched (think Gemini Pro delays), launched quietly or do not get any attention.
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- rllearneratwork 2mo agothe ceiling is to eval's quality
- sscaryterry 2mo agoDon't say that too loud, you may burst the bubble prematurely.
- CMay 2mo agoOr are humans more of a bottleneck than before, because to improve on the most complex problems that demonstrate intelligence you need some way to verify that they are correct. If it's hard for humans to even know if something is correct, wouldn't that slow everything down and simply put limits on the scaling speed of models based on human verification? So instead of relying heavily on human bottlenecks, you focus on agentic task verification since that's the low hanging fruit and verifiable at scale?
- edg5000 2mo agoVery interesting point you make! Before LLMs I had a theory that we cannot make something more intelligent/complex than us. LLMs are certainly more knowledgable, but maybe not more intelligent, arguably. It's possible we're approacing a ceiling indeed. Model capability might be on an asymptote appraching but never quite reaching parity with human intelligence.
- CMay 2mo agoThere are many things that can have some decent level of automatic verification and those were some of the first for LLMs to excel at, like math and programming. Now, physics involves math, but verification would still often require some form of measurement to make sure that the math relates to the real world meaningfully. Many extended kinds of verification can be done by LLMs, but they need to be able to follow instructions reliably and agentic task orchestration may be critical to that verification process. There is no doubt they will surpass us as there is a lot of easy to reason about information that they can verify as incrementally proven by other knowledge. The trick is knowing what can be proven with existing knowledge and what needs human evaluation.
- miki123211 2mo agoI don't think there's a ceiling to LLM capability. I do think that many software dev tasks are just far below that ceiling, and the gains to most dev work won't be that large from now on. Where the new generation of LLMs (Fable, Sol) shines is tasks that are much harder than typical soft eng, yet that still have a verifiable answer, think mathematical proofs or exploits. I think there's still a good amount of low-hanging fruit in those (and similar) areas. The next frontier after that is tasks that don't have automatically-verifiable answers, and may not even have correct and incorrect ones in the strictest sense of the word. Reasonable lawyers might disagree on the question of "which trial strategy do I use given the following set of facts." There are answers that are clearly wrong, but being able to choose between many plausibly-correct ones requires many years of lawyering and seeing many trials play out. I do suspect that most lawyers are far below the ceiling that a hypothetical immortal lawyer that has practiced for an infinite amount of time would have achieved.