4 ms·
This highlights a problem LLMs will face if they improve enough to solve their hallucination problems. People will begin to treat the LLM like some sort of all
by PopePompus 3y ago
This highlights a problem LLMs will face if they improve enough to solve their hallucination problems. People will begin to treat the LLM like some sort of all knowing oracle. Activists will fight fiercely to control the model's output on controversial topics, and will demand lots of model "tuning" after training.
- quotemstr 3y ago> Activists will fight fiercely to control the model's output on controversial topics, They already do. I'd love to know how much "brain damage" RLHF and other censorship techniques cause to the general purpose reasoning abilities of models. (Human reasoning ability is also harmed by lying.) We know the damage is nontrivial.
- int_19h 3y agoTake a look at this. https://cdn2.assets-servd.host/anthropic-website/production/files/model-written-evals.pdf https://cdn2.assets-servd.host/anthropic-website/production/...