> Maybe someone would like to contest that. But that should be done directly. Instead of making pedestrian statements I am not sure what you want from me here.

> Maybe someone would like to contest that. But that should be done directly. Instead of making pedestrian statements I am not sure what you want from me here. Yes, I do contest that "that a lot of LLM brilliance comes from datamining creative thoughts/output from humans and that the brilliant insights go down when there isn’t much in the way of such material". Because I think, the same is the case with humans. Most of the cultural artifacts we produce is crap, a bad copy of natural original. And the "brilliant insights of humans" are achievable by models running on higher temperature. I think the proponents of the theory need to explain by which mechanism the actual loss of information supposedly occurs (on the probability distribution of possible LLM outputs). Is it averaging? Added randomness? Preferential skew? To me, it is rather vague, to the point I don't see how it's different from what humans have done for centuries. Or the opposite, show how those "brilliant insights" from humans manage to survive in the sea of cultural crap otherwise produced in human culture. Perhaps a specific example would help.