3 ms·
> An LLM can only ever reproduce what it has seen before. Anyone who's played around with these models know that at least some generalization is taking place.
by thomasahle 3y ago
> An LLM can only ever reproduce what it has seen before.
Anyone who's played around with these models know that at least some generalization is taking place.
- tempodox 3y agoSo generalization counts as creativity in your book?
- CamperBob2 3y agoInductive reasoning, or the process of inferring from the specific to the general, is kind of a big deal. You can't do that without a semantic model. You can't do it well without a good semantic model... even if it's not something that you or I would recognize as any kind of semantic model at all.
- welshwelsh 3y agoYes, generalization is creativity. I don't think there's a difference between the two concepts. Humans don't come up with ideas out of nowhere. Open a novel and you will find that even though the overall work is unique, it is composed of tropes from other literature and experiences from the author's life which have been generalized into another context.