3 ms·
I worked on a project that did fine tuning and RLHF[1] for a major provider, and you would not believe just how utterly broken a large proportion of the prompts
by vidarh 8mo ago
I worked on a project that did fine tuning and RLHF[1] for a major provider, and you would not believe just how utterly broken a large proportion of the prompts (from real users) were. And the project rules required practically reading tea leaves to divine how to give the best response even to prompts that were not remotely coherent human language.
[1] Reinforcement learning from human feedback; basically participants got two model responses and had to judge them on multiple criteria relative to the prompt
- redman25 8mo agoI feel like the right response for those situations is to start asking questions of the user. It’s what a human would do if they did not understand.
- vidarh 8mo agoI made the argument multiple times that the right answer to many prompts would be a question, and it was allowed under some rare circumstances, but far too few. I suspect in part because the provider also didn't want to create an easy cop out for the people working on the fine-tuning part (a lot of my work was auditing and reviewing output, and there was indeed a lot of really sloppy work, up to and including cut and pasting output from other LLMs - we know, because on more than one occasion I caught people who had managed to include part of Claudes website footer in their answer...)
- winterqt 8mo agoAs in, participants would copy output from one LLM as a question to another?