3 ms·
I'm not sure how crappy small models behaving unreliably is relevant here, when a large SOTA model does presumably not produce the same issue.
by user43928 12d ago
I'm not sure how crappy small models behaving unreliably is relevant here, when a large SOTA model does presumably not produce the same issue.
- intended 12d agoIts relevant because those same issues occur with large models. Also, the "crappy small model", was a model trained for safety tasks, and outperformed the frontier lab safety models.
- user43928 12d agoAnd you tested this, that the presence of the last full stop flips the outcome with a large SOTA model? Small models are notoriously unreliable and prone to hallucination in my experience, so that would not surprise me to be an issue there.