3 ms·
I can't tell whether you are sarcastic? I assume you are, because that makes more sense. It's really easy to get lots and lots of originality. Just crank up
by eru 2y ago
I can't tell whether you are sarcastic?
I assume you are, because that makes more sense.
It's really easy to get lots and lots of originality. Just crank up the randomness. What's harder is to get something that's good and original.
- vidarh 2y agoThat's hard for humans too. Perhaps if one tries to interspers "internal monologue" with the temperature cranked up, with the main output, and prompting the LLM to consider and weave in aspects from the inner monologue when continuing while taking care to make it coherent... (I'm sure it'd be harder than that - but basically inject random internal "thoughts" without letting them control the output)
- throwthrowuknow 2y agoThat’s actually a pretty good idea
- AdieuToLogic 2y agoI was not being sarcastic, but instead witnessing a valuable insight regarding what to expect from LLM's. Ask most commercial LLM services to complete the following sentence: It was the best of times, it was the worst of times And one will likely get the quote from "A Tale of Two Cities"[0]. Ask most commercial LLM services what the completed sentence means, and one will likely get voluminous text which is seemingly correct, perhaps often times is depending on the person reading the response and the service used. But these are statistically derived text constructs entirely dependent upon the training set of the LLM. Train one strictly on Java source code available in Maven Central and the answer will be radically different. > It's really easy to get lots and lots of originality. Just crank up the randomness. And anyone can get "lots and lots of originality" be reading from /dev/urandom. Is that "originality" or simply random tokens inserted into a statistical text generator in order to vary the result? > What's harder is to get something that's good and original. Such is the difference between understanding and statistical text generation. People can do the former, LLM's do the latter. 0 - https://en.wikiquote.org/wiki/A_Tale_of_Two_Cities https://en.wikiquote.org/wiki/A_Tale_of_Two_Cities
- eru 2y agoWell, if you ask most educated English speaking humans to complete 'It was the best of times', they will most likely answer with 'it was the worst of times', too. Btw, here's what I get from ChatGTP 4o: > It was the best of times, it was the worst of times > That's a famous opening line from Charles Dickens' A Tale of Two Cities. It contrasts the extremes of the era, reflecting the novel's themes of duality, revolution, and the complexity of human experience. Dickens was commenting on the contradictions of the time, particularly the French Revolution, where there were both tremendous progress and terrible suffering. What made you bring up this line? > But these are statistically derived text constructs entirely dependent upon the training set of the LLM. Train one strictly on Java source code available in Maven Central and the answer will be radically different. Well, if you give that line to a German who hasn't learned any English, the answer will also differ from what an educated English speaker will give you? What's your point? What's an original insight to you? As far as I can tell, the LLM misses the 'insight' part more than the 'original' part. > Such is the difference between understanding and statistical text generation. People can do the former, LLM's do the latter. I agree that LLMs aren't good at understanding. (Yet?) And even people only sometimes are. As far as I can tell, contemporary LLMs generate their answers 'greedily', ie just from left to right more or less directly with the output from the network. In contrast something like AlphaGo overlays what you can call searching or optimisation processes on top of the outputs from their network. I'm impressed by how well these LLMs already work despite all the limitations. And ML is still getting better rapidly.
- AdieuToLogic 2y ago> Well, if you give that line to a German who hasn't learned any English, the answer will also differ from what an educated English speaker will give you? What's your point? Great observation. A German who does not speak English will likely understand the question is in a non-German language and proceed from there. Perhaps seeking a translation, perhaps replying that they do not speak English and so the question is nonsensical for them. My point is that the German has an understanding of this situation and will communicate accordingly. And, in this example, the LLM trained on Java source code does not have understanding, cannot have understanding, and will emit whatever its training data set directs it to do with the same confidence as any other answer to questions posed to it. Because LLM's are algorithms, quite lovely ones, and algorithms can simulate the effect of understanding, but cannot possess it. Because understanding of this sort is a property of a person. Or maybe a better way to state this is what people think of as understanding is what we know to be understanding, which is by definition the ability to understand perceptions we, as people, experience. > What's an original insight to you? I don't think the following insight is original, but I'll put it out there anyway: Understanding is a property of people for any definition of understanding a person is capable of having. This is due to the fact that understanding exists strictly within the consciousness of the entity defining/possessing it.