3 ms·
Even pass alignment and bias considerations, the fact that larger models saw such a rise of covert racism should worry us about a general regression to the mean
by nyrikki 3y ago
Even pass alignment and bias considerations, the fact that larger models saw such a rise of covert racism should worry us about a general regression to the mean in other contexts.
There was a post last week about the problem of LLMs being biased twords ideals even with averages.
IMHO if LLMs are cutting off the long tail of probabilities as they scale this is a regression from the benefits attention and over Parameterization provide.
We already know self ingestion of generative content can cause this.
So potentially it may have impacts on the value of that investment.