4 ms·
That would still lead to nothing new. It's an imprint of its training data, unable to understand anything of goals or optimisation. It's output will always be a
by rf15 3y ago
That would still lead to nothing new. It's an imprint of its training data, unable to understand anything of goals or optimisation. It's output will always be a kind of average across its training data, including wording.
- retrac 3y agoTrue. Since most current LLMs have no memory/state, do not update the model when inferring, and don't have any goals (beyond find the next token) the basic architecture is truly incapable of iteration, sequential reasoning, or "feeding ideas back on themselves". They're probably essential to creativity. Real creativity, anyway. One can imagine many layers of deep learning systems that are controlling other deep learning systems, in some self-modifying feedback arrangement, in some architecture that hasn't been invented yet. There's some talk about "internal narratives" lately with language models, for example. Generate a prompt at the end, with memories, notes to self, to guide the next invocation of the model. Anyone who has played around with a language model even a bit likely appreciates how quickly something like that is going to just wander off into, in effect, a terminal thought loop, or total incoherence.
- deleted 3y ago[deleted]
- jamilton 3y agoThat pre-supposes that it's creators have read everything useful that GPT-4 has read and put it into practice. It's likely it wouldn't have any useful improvements for the overall architecture because it's novel relative to it's training data, but maybe it could find a small bit of optimization, the kind that most code can benefit from.