5 ms·
The most interesting is the realization that if the LLM's input is only the output of a professional (human), then by definition the LLM cannot mimic the proces
by drbig 7mo ago
The most interesting is the realization that if the LLM's input is only the output of a professional (human), then by definition the LLM cannot mimic the process the (human) professional applied to get from whatever input they had to produce the output.
In other words an LLM can spit out a plausible "output of X", however it cannot encode the process that lead X to transform their inputs into their output.
- Eddy_Viscosity2 7mo agoIs it not possible for the process of input to output be inferred by the llm and therefore applied to new inputs to create appropriate outputs.
- whizzter 7mo agoOnly if the LLM knows the inputs connected to particular outputs, pre-digital era or classified material might not be available, neither informal discussions with other experts. Most importantly, negative but unused signals might not be available if the text does not mention it.
- simianwords 7mo agochallenge: provide a single example where the LLM can only provide the output and not the steps? (in text scenario)
- latexr 7mo agoAn LLM can always output steps, but it doesn’t mean they are true, they are great at making up bullshit. When the “how many ‘r’ in ‘strawberry’” question was all the rage, you could definitely get LLMs to explain the steps of counting, too. It was still wrong.
- simianwords 7mo agocan you provide a single example now with gpt 5.4 thinking that makes up things in steps? lets try to reproduce it.
- latexr 7mo agoI’m pretty sure you can think of one yourself, I’m not going to play this game. Now it’s 5.4 Thinking, before that it was 5.3, before that 5.2, 5.1, 5, before that it was 4… At every stage there’s someone saying “oh, the previous model doesn’t matter, the current one is where it’s at”. And when it’s shown the current model can’t do something, there’s always some other excuse. It’s a profoundly bad faith argument, the very definition of moving the goal posts. I do have a number of examples to give you, but I no longer share those online so they aren’t caught and gamed. Now I share them strictly in person.
- simianwords 7mo agoOk so no example.
- NuclearPM 7mo agoCaught and gamed? What do you mean?
- bigstrat2003 7mo agoHe means that if the problem becomes known, the AI companies will hack in a workaround rather than solving the problem by making the model more intelligent. Given that they have been caught cheating in that way in the past, I can't blame the GP for not sharing his tests.
- weird-eye-issue 7mo agoReplace "LLM" with "student" and read that again. You don't just blindly give students output, you teach them, like what you are supposed to do with an LLM.
- shafyy 7mo agoEnough with this analgoy. It's flawed on so many levels. First and foremost, stop devaluing humanitiy and hyping up AI companies by parroting their party line. Second, LLMs don't learn. They can hold a very limited amount of context, as you know. And every time you need to start over. So fuck no, "teaching" and LLM is nothing like teaching an actual human.
- KeplerBoy 7mo agoIt all went south when we started to call it "learning" instead of "fitting parameters".
- Imustaskforhelp 7mo agoI agree with ya so much. I have seen so many people even in hackernews somehow give human qualities to LLM's. This Grammarly thing seems to be a bastardized form of that not even sparing the dead. I'd say that there was some incentive by the AI companies to muddle up the water here.
- fxtentacle 7mo ago„Fitting“ is still too nice of a word choice, because it implies that it’s easy to identify the best solution. I suggest „randomly adjusting parameters while trying to make things better“ as that accurately reflects the „precision“ that goes into stuffing LLMs with more data.
- bonoboTP 7mo agoIt was called learning already back when the field was called cybernetics and foundational figures like Shannon worked on this kind of stuff. People tried to decipher learning in the nervous system and implement the extracted principles in machines. Such as Hebbian learning, the Perception algorithm etc. This stuff goes back to the 40s/50s/60s, so things must have gone south pretty early then.
- simianwords 7mo agoi don't get what the point of what you are saying is? i can ask it to explain how to solve an integral right now with steps. i can ask it to tell me how to write like a person X right now.
- Planktonne 7mo ago"Explain how to solve" and "write like X" are crucially different tasks. One of them is about going through the steps of a process, and the other is about mimicking the result of a process.
- simianwords 7mo agobut llm can do both. so what's the point? can you give a specific example of what an llm can't do? be specific so we can test it.
- plewd 7mo agolike OP originally said, the LLM doesn't have access to the actual process of the author, only the completed/refined output. Not sure why you need a concrete example to "test", but just think about the fact that the LLM has no idea how a writer brainstorms, re-iterates on their work, or even comes up with the ideas in the first place.
- simianwords 7mo agoi don't buy this logic. if i have studied an author greatly i will be able to recognise patterns and be able to write like them. ex: i read a lot of shakespeare, understand patterns, understand where he came from, his biography and i will be able to write like him. why is it different for an LLM? i again don't get what the point is?
- TimorousBestie 7mo agoThis is the plot of a short story of Borges’ called “Pierre Menard, the Author of Don Quixote.”
- treetalker 7mo agoYou've pinpointed the connection that people fail to make when they seek legal advice (or even information) from LLMs.
- JCharante 7mo agowhat prevents the input from being keystrokes and screen recordings of thousands of lawyers solving cases?
- treetalker 7mo agoThis makes the same error, or a related one. That input is not the lawyer's internal expert process, only the intermediate or (near-) final outcome of it.
- BrtByte 7mo agoLLMs obviously aren't reproducing the internal cognitive process, but they might still capture some of the structural patterns that emerge from it
- noslenwerdna 7mo agoInterestingly, there is some neuroscience research that transformer architecture resembles "cue based retrieval" in the human brain in some important ways. https://www.sciencedirect.com/science/article/pii/S0749596X25000634 https://www.sciencedirect.com/science/article/pii/S0749596X2...