3 ms·
They can only remix what already exists, not create new things, no?
by flohofwoe 3y ago
They can only remix what already exists, not create new things, no?
- jzombie 3y agoTechnically, anything you could ever write in any [programmed or spoken] language is some sort of remix of existing tokens.
- biofunsf 3y agoI think by most definitions LLMs can definitely create new things. As much as I can create new things by cobbling together open source projects and answers on stack overflow at least.
- Der_Einzige 3y agoThere are papers including one by Yann LeCunn which shows that the overwhelming majority of data found within high dimensional spaces is within decision boundries created by AI models. This implies that the overwhelming majority of things we think as humans are "novel" or "extrapolative" are in fact not novel and are nothing more than fancy interpolation. Basically, we humans cannot tell when something is "new to us" or "new to the universe" and it honestly hardly matters in practical reality. Finally, LLMs can totally "extrapolate" with techniques which encourage it to avoid its traditional training loss functions. A high temperature would do this, but it'll mostly just make your model crazy.