5 ms·
The general conceit of this article, which is something that many frontier labs seem to be beginning to realize, is that the average human is no longer smart en
by atleastoptimal 10mo ago
The general conceit of this article, which is something that many frontier labs seem to be beginning to realize, is that the average human is no longer smart enough to provide sufficient signal to improve AI models.
- cyanydeez 10mo agoThey need to spend money on actual experts to curate their data to improve. Instead, finance bros are convinced by the argument that number goes up.
- deleted 10mo ago[deleted]
- 8f2ab37a-ed6c 10mo agoIs that not exactly what https://www.mercor.com/ https://www.mercor.com/ does?
- Terr_ 10mo agoSometimes it feels like: def is_it_true(question): return profit_if_true(question) > profit_if_false(question) AI will make it cheaper, faster, better, no problem. You can eat the cake now and save it for later.
- aspenmartin 10mo agoWait you know that frontier labs do actually do this right?
- Y_Y 10mo agoBut when you're a moron how can you distinguish? I'm being (mostly) serious, suppose you're a stuffed ahort trying to boost your valuation, how can you work out who's smart enough to train your LLM? (Never mind how to get them to work for you!)
- aspenmartin 10mo agoI do a lot of human evaluations. Lots of Bayesian / statistical models that can infer rater quality without ground truth labels. The other thing about preference data you have to worry about (which this article gets at) is: preferences of _who_? Human raters are a significantly biased population of people, different ages, genders, religions, cultures, etc all inform preferences. Lots of work being done to leverage and model this. Then for LMArena there is the host of other biases / construct validity: people are easily fooled, even PhD experts; in many cases it’s easier for a model to learn how to persuade than actually learn the right answers. But a lot of dismissive comments as if frontier labs don’t know this, they have some of the best talent in the world. They aren’t perfect but they in a large sene know what they’re doing and what the tradeoffs of various approaches are. Human annotations are an absolute nightmare for quality which is why coding agents are so nice: they’re verifiable and so you can train them in a way closer to e.g. alphago without the ceiling of human performance
- fc417fc802 10mo ago> in many cases it’s easier for a model to learn how to persuade than actually learn the right answers So we should expect the models to eventually tend toward the same behaviors that politicians exhibit?
- c0balt 10mo agoMaybe a happy to deceive marketing/sales role would be more accurate.
- RA_Fisher 10mo ago100% (am a Bayesian statistician). Isn’t it fascinating how it comes down to quality of judgement (and the descriptions thereof)? We need an LMArena rated by experts.
- Lerc 10mo agoAs a statistician, do you you think you could, given access to the data, identify the subset of LMArena users that are experts?
- Yizahi 10mo agoYep, it's like getting a commoner from the street evaluate a literature PhD in their native language. Sure, both know the language, but the depth difference of a specialist vs a generalist is too large. And neither we can't use AI to automatically evaluate this literature genius because real AI doesn't exist (yet), hence the programs can't understand the contents of text they output or input. Whoops. :)
- ryandrake 10mo agoPopularity has never been a meaningful signal of quality, no matter how many tech companies try to make it so, with their star ratings, up/down voting, and crowdsourcing schemes.
- PaulHoule 10mo agoDifferent strokes for different folks: I mean who is to say if Bleach or Backstabbed in a Backwater Dungeon: My Trusted Companions Tried to Kill Me, but Thanks to the Gift of an Unlimited Gacha I Got LVL 9999 Friends and Am Out for Revenge on My Former Party Members and the World is better?
- gpm 10mo agoNo, it's that the average unpaid human doesn't care to read closely enough to provide signal to improve AI models. Not that they couldn't if they put in even the slightest amount of effort.
- ehnto 10mo agoWhy would an unpaid human want to do that?
- kazinator 10mo agoFirstly, paying is not at all the correct incentive for the desired outcome. When the incentive is payment, people will optimize for maximum payout not for the quality goals of the system. Secondly, it doesn't fix stupidity. A participant who earnestly takes the quality goals of the system to heart instead of focusing on maximizing their take (thus, obviously stupid) will still make bad classifications due to that reason.
- tbrownaw 10mo ago> Firstly, paying is not at all the correct incentive for the desired outcome. When the incentive is payment, people will optimize for maximum payout not for the quality goals of the system. 1. I would expect any paid arrangement to include a quality-control mechanism. With the possible exception of if it was designed from scratch by complete ignoramuses. 2. Do you have a proposal for a better incentive?
- Eisenstein 10mo ago1. Goodhart's law suggests that you will end up with quality control mechanisms which work at ensuring that the measure is being measured, but not that it is measuring anything useful 2. Criticism of a method does not require that there is a viable alternative. Perhaps the better idea is just to not incentivize people to do tasks they are not qualified for
- michaelmrose 10mo agoThe average human is a moron you wouldn't trust to watch your hamster. If you watched them outside of the narrow range of tasks they have been trained to perform by rote you would probably conclude they should qualify for benefits by virtue of mental disability. We give them WAY too much credit by watching mostly the things they have been trained specifically to do and pretending this indicates a general mental competence that just doesn't exist.
- kazinator 10mo agoIt is glaringly obvious that the average human is not smart enough to the level hat their decision making should be replicated and adopted at scale. People hold falsehoods to be true, and cannot calculate a 10% tip.
- echelon 10mo agoIf these frontier models were open source, the market of downstream consumers would figure out how to optimize them. By being closed, they'll never be optimal.