2 ms·
We don't know if there's a state or not. There probably is some kind of state. Many kinds of generations are impossible without it. >It cannot even distinguish
by famouswaffles 4y ago
We don't know if there's a state or not. There probably is some kind of state. Many kinds of generations are impossible without it.
>It cannot even distinguish between text it generated and text you added unless you tell it that.
Many people cannot distinguish reality from fiction invented by the brain. This doesn't mean much. If i could insert visual sense into you anytime i wanted, you'd struggle to tell it apart at best.
People can't really recreate mental states. It's all post hoc rationalization.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/
and the split brain experiments tell us the brain will happily fabricate those rationalizations to be something entirely unrelated.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/
- OrderlyTiamat 4y agoI don't know what this all means. You'll have to explain your point about fabrication as it relates to state in llms in more detail. But as a matter of technical implementation I can tell you: there is no state. You can't simply say "we don't know if theres state" and be done with it. You individually might be unsure. Check the paper "attention is all you need", which introduced transformers, if you'd like to see for yourself [1]. The state is all in the text, that is why I said it's important that llms can't even distinguish between text you wrote and it wrote- how can you talk about an internal state of the llm if I can create that state by my input of text? That's just input at that point, not state irrespective of input. The same would go for your inserted visual sense, I suppose. [1] https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762