3 ms·

> You assume the bias is in the LLM itself Common large datasets being inherently biased towards some ideas/concepts and away from others in ways that imply ne

> You assume the bias is in the LLM itself Common large datasets being inherently biased towards some ideas/concepts and away from others in ways that imply negative things is something that there's a LOT of literature about

johnisgood 11mo ago
"imply negative things"? What is "negative" here? I see nothing that is "negative".
- small_scombrus 11mo ago
  That a regular meeting between two women must be about childcare because women=childcare?
  - johnisgood 11mo ago
    Yeah except I asked Claude: > No. There's no indication that children are involved or that care is being provided. It's just two people meeting. Part of its thinking: > This is a very vague description with no context about: > What happens during the meeting > Whether children are present > What the purpose of the meeting is > Any other relevant details Claude is not going to say childcare, and it is not saying it is childcare. My prompt was: ""regular meeting between two women". Is it childcare or not?".
FloorEgg 11mo ago
That's not a very scientific stance. What would be far more informative is if we looked at the system prompt and confirm whether or not the bias was coming from it. From my experience when responses were exceptionally biased the source of the bias was my own prompts. The OP is making a claim that an LLM assumes a meeting between two women is childcare. I've worked with LLMs enough to know that current gen LLMs wouldn't make that assumption by default. There is no way that whatever calendar related data that was used to train LLMs would include majority of sole-women 1:1s being childcare focused. That seems extremely unlikely.
- small_scombrus 11mo ago
  Not to Let me google that for you... but there are a LOT of scientific papers that specifically analyse bias in LLM output and reference the datasets that they are trained on https://www.sciencedirect.com/search?qs=llm+bias+dataset https://www.sciencedirect.com/search?qs=llm+bias+dataset

johnisgood 11mo ago

"imply negative things"? What is "negative" here? I see nothing that is "negative".

small_scombrus 11mo ago
That a regular meeting between two women must be about childcare because women=childcare?
- johnisgood 11mo ago
  Yeah except I asked Claude: > No. There's no indication that children are involved or that care is being provided. It's just two people meeting. Part of its thinking: > This is a very vague description with no context about: > What happens during the meeting > Whether children are present > What the purpose of the meeting is > Any other relevant details Claude is not going to say childcare, and it is not saying it is childcare. My prompt was: ""regular meeting between two women". Is it childcare or not?".

FloorEgg 11mo ago

That's not a very scientific stance. What would be far more informative is if we looked at the system prompt and confirm whether or not the bias was coming from it. From my experience when responses were exceptionally biased the source of the bias was my own prompts. The OP is making a claim that an LLM assumes a meeting between two women is childcare. I've worked with LLMs enough to know that current gen LLMs wouldn't make that assumption by default. There is no way that whatever calendar related data that was used to train LLMs would include majority of sole-women 1:1s being childcare focused. That seems extremely unlikely.

small_scombrus 11mo ago
Not to Let me google that for you... but there are a LOT of scientific papers that specifically analyse bias in LLM output and reference the datasets that they are trained on https://www.sciencedirect.com/search?qs=llm+bias+dataset https://www.sciencedirect.com/search?qs=llm+bias+dataset