4 ms·
This kind of begs the question though in the “there’s nothing new under the sun” sense. Maybe your particular task is novel, but are its component parts and alg
by danenania 2y ago
This kind of begs the question though in the “there’s nothing new under the sun” sense. Maybe your particular task is novel, but are its component parts and algorithms novel? In some cases, perhaps some of them are, but I think that’s more the exception than the rule.
LLMs aren’t just tape recorders. They can mix and match and adapt. What percentage of the code you write do you think is truly original?
- shadowerm 2y agoI just don't know if this is true. The music programming language Csound is a good test. There is not a huge amount of csound code online, but what is online is going to work. The models are just terrible with csound. They can not mix and match at all. They can't abstract at all. An even better test is to have the LLM try to make a piece of music in a certain style in csound. That is when you can see that the model is not abstracting the concepts at all. Zero. none. nada. Of course, it gives the probablistic next token prediction but because none of this is in the training data, the most probable output is basically noise/nonsense. In general, I think we highly underestimate how hard it is to come up with things that are not in the training data.