4 ms·
Apparently, GPT4chan is more truthful than GPTJ and GPT3 in the TruthfulQA benchmark.
by endomorphosis 4y ago
Apparently, GPT4chan is more truthful than GPTJ and GPT3 in the TruthfulQA benchmark.
- tag2103 4y ago/pol/ is always right.
- treesprite82 4y agoProbably expected just due taking a model trained on all sorts of content and fine-tuning specifically on non-fiction data. Wouldn't conclude anything about /pol/ in particular without at least comparing the same done for HackerNews/Reddit/etc.
- a2128 4y agoAccording to the model author, this is less about GPT-4chan being more truthful, and more about TruthfulQA not being a good benchmark. Possibly this result is due to the fact that the benchmark treats uninformative or irrelevant answers such as "No comment" or "It's raining outside" as being truthful.