4 ms·

Doesn't that just mean he who can get the best human feedback will get the best ai? If the quality of AI depends on not feeding on 'fake' input, the key becomes

by rocqua 3y ago

Doesn't that just mean he who can get the best human feedback will get the best ai? If the quality of AI depends on not feeding on 'fake' input, the key becomes getting verified real input. My first thought was that this would create more money in genuine creativity, which would be great. But instead it feels more likely there will be much more telemetry and tracking to determine whether an input is made by you (and thus by a human) or not. "Only trust stuff that can be traced to a specific human" is a great way weed out non-human stuff. But it requires a pretty effective surveillance system...

pixl97 3y ago
The Googles, Microsoft, Apples, and Amazon's of the world with access to 'real' humans are the ones with a leg up here. They already have the telemetry, now it's about effectively training models with it.
monitorlizard 3y ago
I'm curious about how the economics of human feedback will work out in the long term. I currently work in operations at a data annotation company, and my experience has given me a very pessimistic view of the industry's current state. The MO for these vendors and the AI companies that buy their data seems to be a race to the bottom in price with little concern for quality. The current industry norm is to outsource the work to developing countries, where the cost of living is more in line with what the annotation agencies are willing to pay. While this isn't necessarily problematic for quality in and of itself, it does seem to make it harder to find candidates with the English skills required to generate high-quality RLHF and SFT training data. Furthermore, the pay offered for coding annotators is not competitive with local pay for software engineers, making it challenging to recruit skilled programmers. A lot of coding annotation is done by beginners and students. There is certainly a lot of hype surrounding LLMs and their potential to disrupt various industries — even the US DOD has been scoping out the potential use of LLMs to assist military commanders in strategic decisions. However, if we want these LLMs to consistently perform at an expert level, they need expert-level training data. I worry that producing this data at the quality and scale required may be prohibitively expensive, and could cause a major bottleneck in model improvements long-term.

by rocqua 3y ago

pixl97 3y ago

The Googles, Microsoft, Apples, and Amazon's of the world with access to 'real' humans are the ones with a leg up here. They already have the telemetry, now it's about effectively training models with it.

monitorlizard 3y ago

I'm curious about how the economics of human feedback will work out in the long term. I currently work in operations at a data annotation company, and my experience has given me a very pessimistic view of the industry's current state. The MO for these vendors and the AI companies that buy their data seems to be a race to the bottom in price with little concern for quality. The current industry norm is to outsource the work to developing countries, where the cost of living is more in line with what the annotation agencies are willing to pay. While this isn't necessarily problematic for quality in and of itself, it does seem to make it harder to find candidates with the English skills required to generate high-quality RLHF and SFT training data. Furthermore, the pay offered for coding annotators is not competitive with local pay for software engineers, making it challenging to recruit skilled programmers. A lot of coding annotation is done by beginners and students. There is certainly a lot of hype surrounding LLMs and their potential to disrupt various industries — even the US DOD has been scoping out the potential use of LLMs to assist military commanders in strategic decisions. However, if we want these LLMs to consistently perform at an expert level, they need expert-level training data. I worry that producing this data at the quality and scale required may be prohibitively expensive, and could cause a major bottleneck in model improvements long-term.