2 ms·
You would do statistical analysis of the output. E.g. whether it is preferring a certain gender if gender is variable in the prompt, but everything else is the
by mewpmewp2 2y ago
You would do statistical analysis of the output. E.g. whether it is preferring a certain gender if gender is variable in the prompt, but everything else is the same etc.
- deleted 2y ago[deleted]
- dragonwriter 2y ago> You would do statistical analysis of the output. Assuming reproducible settings are used, that will give you perfect insight into the behavior in the tested circumstances, but unless it lets you reconstruct the entire network, it won't tell you the impact that untested changes even if they are recombinations of tested elements behave. It is not perfect insight into the biases.
- czl 2y agoWhat if the LLM is biased against words that start with e or biased towards sentences having five words? Using your technique how would you discover such biases? If highly paid lawyers know about some such innate LLM quirks might they exploit them to get a biased decision?