3 ms·
I very much have LLMs go through rule sets all the time? In fact, any prompt to an LLM is in fact, a rule set of some sort. Can you say plausible but I think wh
by conception 1y ago
I very much have LLMs go through rule sets all the time? In fact, any prompt to an LLM is in fact, a rule set of some sort. Can you say plausible but I think what you mean is probable. When you give an LLM rules most of the time the most probable answer is in fact follow them. But when you give it lots and lots of rules and or fill up its context sometimes the most probable thing is not necessarily to follow the rule it’s been given, but some other combination of information that it is outputting.
- gronglo 1y agoMy understanding is that there are no "rules", only relationships between words. I picture it as a vector pointing off into a cloud of related words. You can feed it terms that alter that vector and point it into a different part of the word cloud, but if enough of your other terms outweigh the original "instruction", the vector may get dragged back into a different part of the cloud that "disobeys" the instruction. Maybe an expert can correct me here.
- Terr_ 1y agoThe trick is that the rules it follows aren't the ones people write. The real ones just happen to give similar answers, until one day they don't. The LLM takes a document and returns a "fitting" token that would go next. So "Calculate 2+2" may yield a "4", but the reason it gets there is document-fitting, rather than math.