3 ms·

> P.S.: For example, imagine having applications for jobs and loans rejected because all the companies' internal LLM tooling is secretly racist against subtle g

by squigz 2y ago

> P.S.: For example, imagine having applications for jobs and loans rejected because all the companies' internal LLM tooling is secretly racist against subtle grammar-traces in your writing or social-media profile. [0] We don't have to imagine such things, really, as that's extremely common with humans. I would argue that fixing such flaws in LLMs is a lot easier than fixing it in humans.

Terr_ 2y ago
Fixing it with careful application of software-in-general is quite promising, but LLMs in particular are a terrible minefield of infinite whack-a-mole. (A mixed metaphor, but the imagery is strangely attractive.) I currently work in the HR-tech space, so suppose someone has a not-too-crazy proposal of using an LLM to reword cover-letters to reduce potential bias in hiring. The issue is that the LLM will impart its own spin(s) on things, even when a human would say two inputs are functionally identical. As a very hypothetical example, suppose one candidate always does stuff like writing out the Latin like Juris Doctor instead of acronyms like JD, and then that causes the model to end up on "extremely qualified at" instead of "very qualified at" The issue of deliberate attempts to corrupt the LLM with prompt-injection or poisonous training data are a whole 'nother can of minefield whack-a-moles. (OK, yeah, too far there.)
- squigz 2y ago
  I don't think I disagree with you in principle, although I think these issues also apply to humans. I think even your particular example isn't a very far-fetched conclusion for a human to arrive at. I just don't think your original comment was entirely fair. IMO, LLMs and related technology will be looked at similarly as the Internet - certainly it has been used for bad, but I think the good far outweighs the bad, and I think we have (and continue to) learn to deal with the issues with it, just as we will with LLMs and AI. (FWIW, I'm not trying to ignore the ways this technology will be abused, or advocate for the crazy capitalistic tendency of shoving LLMs in everything. I just think the potential for good here is huge, and we should be just as aware of that as the issues) (Also FWIW, I appreciate your entirely reasonable comment. There's far too many extreme opinions on this topic from all sides.)

by squigz 2y ago

Terr_ 2y ago

Fixing it with careful application of software-in-general is quite promising, but LLMs in particular are a terrible minefield of infinite whack-a-mole. (A mixed metaphor, but the imagery is strangely attractive.) I currently work in the HR-tech space, so suppose someone has a not-too-crazy proposal of using an LLM to reword cover-letters to reduce potential bias in hiring. The issue is that the LLM will impart its own spin(s) on things, even when a human would say two inputs are functionally identical. As a very hypothetical example, suppose one candidate always does stuff like writing out the Latin like Juris Doctor instead of acronyms like JD, and then that causes the model to end up on "extremely qualified at" instead of "very qualified at" The issue of deliberate attempts to corrupt the LLM with prompt-injection or poisonous training data are a whole 'nother can of minefield whack-a-moles. (OK, yeah, too far there.)

squigz 2y ago
I don't think I disagree with you in principle, although I think these issues also apply to humans. I think even your particular example isn't a very far-fetched conclusion for a human to arrive at. I just don't think your original comment was entirely fair. IMO, LLMs and related technology will be looked at similarly as the Internet - certainly it has been used for bad, but I think the good far outweighs the bad, and I think we have (and continue to) learn to deal with the issues with it, just as we will with LLMs and AI. (FWIW, I'm not trying to ignore the ways this technology will be abused, or advocate for the crazy capitalistic tendency of shoving LLMs in everything. I just think the potential for good here is huge, and we should be just as aware of that as the issues) (Also FWIW, I appreciate your entirely reasonable comment. There's far too many extreme opinions on this topic from all sides.)