Most people (average and below average) can tell when something is above average, even if they cannot create above average work, so using RLHF it should be quit

Most people (average and below average) can tell when something is above average, even if they cannot create above average work, so using RLHF it should be quite possible to achieve above average. Indeed it is likely already the case that in training the top links scraped or most popular videos are weighted higher, these are likely to be better than average.