4 ms·
They do not require instances of exact words in the training set, either. They can cope with e.g. typos or novel portmanteaus of your own making. Can you think
by david-gpu 2y ago
They do not require instances of exact words in the training set, either. They can cope with e.g. typos or novel portmanteaus of your own making.
Can you think of an example of the limitations you are thinking of?
- semking 2y agoSorry, I just saw your question. I'm still confused by the comment system here! To answer your question, I've noticed LLMs overfit the patterns in their training data. The performance was poor when asked to "reason" in ways that diverge from what they "learned". Things less frequently mentioned in the datasets were clearly weighted as less important. And the opposite is also true. I've seen bias being amplified. For some reason, I've seen words such as "elevate" or "landscape" being overused. And obviously, I've seen so many factual errors. Search for niche topics/individuals that were not covered in the training data and you'll see incredible hallucinations.