4 ms·

I like how they brush off VERY REAL concerns of bias, mis-representation as having "spicy takes" and "a rebellious streak". More power to anyone financing the t

I like how they brush off VERY REAL concerns of bias, mis-representation as having "spicy takes" and "a rebellious streak". More power to anyone financing the training of an LLM but if you're too lazy to red-team it, debias it, and expect downwind people to take care of that, say so! Don't pass it off as a unique feature of your LLM.

fallingknife 3y ago
How do you "debias" an LLM? There is no unbiased standard to test against.
- Invictus0 3y ago
  [flagged]
- geraltofrivia 3y ago
  That's not _completely_ true. There are a bunch of datasets (specific to some cultural/lingual contexts) like CrowsPairs[1], StereoSet[2]. There is a lot of work you can do to make sure that the model's predictions are fair as well [3]. But at the end, yes these datasets don't exist at the scale of training sets of these LLMs. Hence red-teaming and RLHF post convergence. PS: Yes I know CrowsPairs is a dataset with a bunch of flaws. My SO is working, in a team of 10+ linguists and researchers to develop a multi-lingual, generalized version of it which also addresses multiple problems with it. Unpublished work, for now. [1] https://github.com/nyu-mll/crows-pairs/ https://github.com/nyu-mll/crows-pairs/ [2] https://arxiv.org/abs/2004.09456 https://arxiv.org/abs/2004.09456 [3] https://arxiv.org/pdf/2204.09591.pdf https://arxiv.org/pdf/2204.09591.pdf
- blueboo 3y ago
  There are all sorts. You do have to be specific about what you mean by “bias” though. https://arxiv.org/abs/2110.08193 https://arxiv.org/abs/2110.08193 and https://arxiv.org/abs/1804.09301 https://arxiv.org/abs/1804.09301 and https://arxiv.org/abs/2206.04615 https://arxiv.org/abs/2206.04615 …
- dragonwriter 3y ago
  You can debias it relative to any standard you choose, and maybe should if there is one particularly applicable to the intended use case, but the more important thing for models intended for broad, multi-domain use is to document and disclose biases, and ideally any particularly effective techniques for nudging them for particular applications short of fine-tuning/retraining.
TaylorAlexander 3y ago
People are going to tear thing thing apart and help build a solid case for why these guardrails are built in in the first place.
- soulofmischief 3y ago
  Elon is ironically going to be a part of the reason that access to foundational models will be banned in the US in the wake of Biden's recent executive order.
  - TaylorAlexander 3y ago
    I hope that does not happen but I do suspect this will backfire in some way. Hopefully it would ultimately be beneficial, demonstrating why handling these models with care is worthwhile.
  - mvelbaum 3y ago
    That executive order is an affront to intelligence and must be cancelled ASAP by the next president who will hopefully not be a Democrat.
    - wilg 3y ago
      Oh I really hope the next president is a Democrat since we're talking about it! Not a great EO, but Biden has done great so far.
      - lucubratory 3y ago
        I'm not an American and don't like either of your parties, but just wanted to say I respect the enthusiasm in immediately confronting someone violating social norms. It's people like you that keep communities alive.
    - soulofmischief 3y ago
      I hate to be the one to break it to you, but the GOP doesn't give a rat's ass about you or your rights, either. It's lip service from both parties, both stand to gain from regulatory capture.
junofan 3y ago
I dunno, I stopped using ChatGPT. The magic’s gone after the fiftieth lecture. I’ll give Grok a shot. People who don’t want to hear from Grok can just not use it. People who are concerned about what Grok might say to me can rest assured their concern is misplaced.
- geraltofrivia 3y ago
  I may be wrong, and if so, correct me. But do you use ChatGPT as a recreation thing, or as a part of a real world solution? In my current job, we've started offloading a lot of real work to GPT4 (it just works ). Sometimes this is solving some recognition tasks, but in some cases we use it to generate summaries for customers, or interface our helpdesk with the RAG pattern [1]. In this case, bias and unfairness can be a real concern since they will affect the products and the business. Its not about being 'preached to' but about avoiding pissed off customers and decrease of trust from them if our system generates hateful/biased text. I want to emphasize that it's not about "what Grok might say to me". Of course, not. I've got friends who revel in dark humour and I part-take more than most. But if you've spent millions training an LLM, you're not just going to use it as a toy. It is going to solve some usecase. That usecase will probably affect real people. If your LLM is biased, or has little guardrails, those affects might not all be positive. [1] https://research.ibm.com/blog/retrieval-augmented-generation-RAG https://research.ibm.com/blog/retrieval-augmented-generation...
  - martindbp 3y ago
    I think the level of caution should just be a setting.
- LightBug1 3y ago
  I dunno, I'll stop eating state of the art fare, the magic has gone. I'll give McDonald's a shot.
mvelbaum 3y ago
By "debias" you obviously mean "bias in the direction of my particular worldview".
- geraltofrivia 3y ago
  Yes. In my case, my particular worldview involves (i) indifferent to gender markers in text (I don't want my LLM to convert me to a female because I'm asking it to write a cover letter for a hairdresser/secretary/nurse role, for e.g.). I don't want my model to write it in AAVE if my name is a common african-american name. I don't want to get the best results only when I write as a middle-aged white man [1]. If I use the model for some decision making, I want it to be fair for subgroups [2], which is a reasonably objective metric [3]. [1] https://aclanthology.org/P19-1339.pdf https://aclanthology.org/P19-1339.pdf [2] https://arxiv.org/pdf/1906.09208.pdf https://arxiv.org/pdf/1906.09208.pdf [3] https://developers.google.com/machine-learning/glossary/fairness#fairness_metric https://developers.google.com/machine-learning/glossary/fair...
  - calderknight 3y ago
    Will Grok really do any of those things? I would have guessed that RLHF would sort those things out even if it wasn't concerned with debiasing, but just about not making ridiculous mistakes.
    - geraltofrivia 3y ago
      These are what debiasing tasks are concerned with, more often than not. RLHF tuning depends greatly on the H part of it, and that data is probably proprietary. So, I guess time will tell. But if I were to hazard a guess based on the content of the announcement , I would say they couldn’t be bothered or couldn’t accomplish proper debiasing/rlhf tuning and therefore worded it so.
  - caskstrength 3y ago
    Isn't it enough to append "write it as if I'm a middle-aged white man" to your request to obtain the desired output?
martindbp 3y ago
My guess is these hypothetical harms are as imaginary as the supposed collapse of X after letting 80% go. The "harm" seems to be the public (media) outrage at some inappropriate content produced. Most companies cave immediately at any level of pressure, but I have a feeling Musk won't. He's the type to take it to court if needed.

fallingknife 3y ago

How do you "debias" an LLM? There is no unbiased standard to test against.

Invictus0 3y ago
[flagged]
geraltofrivia 3y ago
That's not _completely_ true. There are a bunch of datasets (specific to some cultural/lingual contexts) like CrowsPairs[1], StereoSet[2]. There is a lot of work you can do to make sure that the model's predictions are fair as well [3]. But at the end, yes these datasets don't exist at the scale of training sets of these LLMs. Hence red-teaming and RLHF post convergence. PS: Yes I know CrowsPairs is a dataset with a bunch of flaws. My SO is working, in a team of 10+ linguists and researchers to develop a multi-lingual, generalized version of it which also addresses multiple problems with it. Unpublished work, for now. [1] https://github.com/nyu-mll/crows-pairs/ https://github.com/nyu-mll/crows-pairs/ [2] https://arxiv.org/abs/2004.09456 https://arxiv.org/abs/2004.09456 [3] https://arxiv.org/pdf/2204.09591.pdf https://arxiv.org/pdf/2204.09591.pdf
blueboo 3y ago
There are all sorts. You do have to be specific about what you mean by “bias” though. https://arxiv.org/abs/2110.08193 https://arxiv.org/abs/2110.08193 and https://arxiv.org/abs/1804.09301 https://arxiv.org/abs/1804.09301 and https://arxiv.org/abs/2206.04615 https://arxiv.org/abs/2206.04615 …
dragonwriter 3y ago
You can debias it relative to any standard you choose, and maybe should if there is one particularly applicable to the intended use case, but the more important thing for models intended for broad, multi-domain use is to document and disclose biases, and ideally any particularly effective techniques for nudging them for particular applications short of fine-tuning/retraining.

TaylorAlexander 3y ago

People are going to tear thing thing apart and help build a solid case for why these guardrails are built in in the first place.

soulofmischief 3y ago
Elon is ironically going to be a part of the reason that access to foundational models will be banned in the US in the wake of Biden's recent executive order.
- TaylorAlexander 3y ago
  I hope that does not happen but I do suspect this will backfire in some way. Hopefully it would ultimately be beneficial, demonstrating why handling these models with care is worthwhile.
- mvelbaum 3y ago
  That executive order is an affront to intelligence and must be cancelled ASAP by the next president who will hopefully not be a Democrat.
  - wilg 3y ago
    Oh I really hope the next president is a Democrat since we're talking about it! Not a great EO, but Biden has done great so far.
    - lucubratory 3y ago
      I'm not an American and don't like either of your parties, but just wanted to say I respect the enthusiasm in immediately confronting someone violating social norms. It's people like you that keep communities alive.
  - soulofmischief 3y ago
    I hate to be the one to break it to you, but the GOP doesn't give a rat's ass about you or your rights, either. It's lip service from both parties, both stand to gain from regulatory capture.

junofan 3y ago

I dunno, I stopped using ChatGPT. The magic’s gone after the fiftieth lecture. I’ll give Grok a shot. People who don’t want to hear from Grok can just not use it. People who are concerned about what Grok might say to me can rest assured their concern is misplaced.

geraltofrivia 3y ago
I may be wrong, and if so, correct me. But do you use ChatGPT as a recreation thing, or as a part of a real world solution? In my current job, we've started offloading a lot of real work to GPT4 (it just works ). Sometimes this is solving some recognition tasks, but in some cases we use it to generate summaries for customers, or interface our helpdesk with the RAG pattern [1]. In this case, bias and unfairness can be a real concern since they will affect the products and the business. Its not about being 'preached to' but about avoiding pissed off customers and decrease of trust from them if our system generates hateful/biased text. I want to emphasize that it's not about "what Grok might say to me". Of course, not. I've got friends who revel in dark humour and I part-take more than most. But if you've spent millions training an LLM, you're not just going to use it as a toy. It is going to solve some usecase. That usecase will probably affect real people. If your LLM is biased, or has little guardrails, those affects might not all be positive. [1] https://research.ibm.com/blog/retrieval-augmented-generation-RAG https://research.ibm.com/blog/retrieval-augmented-generation...
- martindbp 3y ago
  I think the level of caution should just be a setting.
LightBug1 3y ago
I dunno, I'll stop eating state of the art fare, the magic has gone. I'll give McDonald's a shot.

mvelbaum 3y ago

By "debias" you obviously mean "bias in the direction of my particular worldview".

geraltofrivia 3y ago
Yes. In my case, my particular worldview involves (i) indifferent to gender markers in text (I don't want my LLM to convert me to a female because I'm asking it to write a cover letter for a hairdresser/secretary/nurse role, for e.g.). I don't want my model to write it in AAVE if my name is a common african-american name. I don't want to get the best results only when I write as a middle-aged white man [1]. If I use the model for some decision making, I want it to be fair for subgroups [2], which is a reasonably objective metric [3]. [1] https://aclanthology.org/P19-1339.pdf https://aclanthology.org/P19-1339.pdf [2] https://arxiv.org/pdf/1906.09208.pdf https://arxiv.org/pdf/1906.09208.pdf [3] https://developers.google.com/machine-learning/glossary/fairness#fairness_metric https://developers.google.com/machine-learning/glossary/fair...
- calderknight 3y ago
  Will Grok really do any of those things? I would have guessed that RLHF would sort those things out even if it wasn't concerned with debiasing, but just about not making ridiculous mistakes.
  - geraltofrivia 3y ago
    These are what debiasing tasks are concerned with, more often than not. RLHF tuning depends greatly on the H part of it, and that data is probably proprietary. So, I guess time will tell. But if I were to hazard a guess based on the content of the announcement , I would say they couldn’t be bothered or couldn’t accomplish proper debiasing/rlhf tuning and therefore worded it so.
- caskstrength 3y ago
  Isn't it enough to append "write it as if I'm a middle-aged white man" to your request to obtain the desired output?

martindbp 3y ago

My guess is these hypothetical harms are as imaginary as the supposed collapse of X after letting 80% go. The "harm" seems to be the public (media) outrage at some inappropriate content produced. Most companies cave immediately at any level of pressure, but I have a feeling Musk won't. He's the type to take it to court if needed.