3 ms·
I don't know what this all means. You'll have to explain your point about fabrication as it relates to state in llms in more detail. But as a matter of technic
by OrderlyTiamat 3y ago
I don't know what this all means. You'll have to explain your point about fabrication as it relates to state in llms in more detail.
But as a matter of technical implementation I can tell you: there is no state. You can't simply say "we don't know if theres state" and be done with it. You individually might be unsure. Check the paper "attention is all you need", which introduced transformers, if you'd like to see for yourself [1].
The state is all in the text, that is why I said it's important that llms can't even distinguish between text you wrote and it wrote- how can you talk about an internal state of the llm if I can create that state by my input of text? That's just input at that point, not state irrespective of input. The same would go for your inserted visual sense, I suppose.
[1] https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762