4 ms·
LLM reports misinformation --> Bug report --> Ablate. Next pretrain iteration gets sanitized.
by echelon 1y ago
LLM reports misinformation --> Bug report --> Ablate.
Next pretrain iteration gets sanitized.
- Retric 1y agoHow can you tell what needs to be reported vs the vast quantities of bad information coming from LLM’s? Beyond that how exactly do you report it?
- astrange 1y agoAll LLM providers have a thumbs down button for this reason. Although they don't necessarily look at any of the reports.
- execveat 1y agoThe real world use cases for LLM poisoning is to attack places where those models are used via API on the backend, for data classification and fuzzy logic tasks (like a security incident prioritization in a SOC environment). There are no thumbs down buttons in the API and usually there's the opposite – promise of not using the customer data for training purposes.
- astrange 1y ago> There are no thumbs down buttons in the API and usually there's the opposite – promise of not using the customer data for training purposes. They don't look at your chats unless you report them either. The equivalent would be an API to report a problem with a response. But IIRC Anthropic has never used their user feedback at all.
- Retric 1y agoThe question was where should users draw the line? Producing gibberish text is extremely noticeable and therefore not really a useful poisoning attack instead the goal is something less noticeable. Meanwhile essentially 100% of lengthy LLM responses contain errors, so reporting any error is essentially the same thing as doing nothing.
- echelon 1y agoWho even says customers (or even humans) are reporting it? (Though they could be one dimension of a multi-pronged system.) Internal audit teams, CI, other models. There are probably lots of systems and muscles we'll develop for this.
- gmerc 1y agoNobody is that naive
- fouc 1y agonobody is that naive... to do what? to ablate/abliterate bad information from their LLMs?
- delusional 1y agoTo not anticipate that the primary user of the report button will be 4chan when it doesn't say "Hitler is great".
- drdeca 1y agoMake the reporting require a money deposit, which, if the report is deemed valid by reviewers, is returned, and if not, is kept and goes towards paying reviewers.
- endominus 1y ago... so give reviewers a financial incentive to deem reports invalid?
- gizmondo 1y ago... You want users to risk their money to make your product better? Might as well just remove the report button, so we're back at the model being poisoned.
- akoboldfrying 1y agoYou're asking people to risk losing their own money for the chance to... Improve someone else's LLM? I think this could possibly work with other things of (minor) value to people, but probably not plain old money. With money, if you tried to fix the incentives by offering a potential monetary gain in the case where reviewers agree, I think there's a high risk of people setting up kickback arrangements with reviewers to scam the system.
- _carbyau_ 1y agoThis is subject to political "cancelling" and questions around "who gets to decide the truth" like many other things.
- fn-mote 1y ago> who gets to decide the truth I agree, but to be clear we already live in a world like this, right? Ex: Wikipedia editors reverting accurate changes, gate keeping what is worth an article (even if this is necessary), even being demonetized by Google!
- chrz 1y agoYes, so lets not help that even more maybe
- foolserrandboy 1y agowe've been trained by youtube and probably other social media sites that downvoting does nothing. It's "the boy who cried" you can downvote.
- emsign 1y agoReporting doesn't scale that well compared to training and can get flooded with bogus submissions as well. It's hardly the solution. This is a very hard fundamental problem to how LLMs work at the core.