6 ms·

This is expected behavior if you understand that the results from any data-based modeling process (machine learning generally) is a concactination of the cumula

• djbusby 2y ago
  it shows us ourselves, and the parts we pretend aren't there.
  • anarchy79 2y ago
    I don't know about pretending, I'm pretty sure most people would think twice before hiring an autistic CEO. On the other hand there is X, so I might be wrong.
    • killingtime74 2y ago
      Linda Yaccarino is autistic? First I've heard of this. https://en.m.wikipedia.org/wiki/Linda_Yaccarino https://en.m.wikipedia.org/wiki/Linda_Yaccarino
    • bigstrat2003 2y ago
      Most people would think twice before hiring an autistic CEO. Very few people would admit "I don't think it's a very good idea to hire an autistic CEO". That's the pretense GP was speaking of.
    • tbrownaw 2y ago
      > On the other hand there is X, so I might be wrong. I don't think companies "hire" their owners, exactly.
      • skeledrew 2y ago
        The CEO of a company isn't always the owner. Which is why some can also be fired.
• sandworm101 2y ago
  >> there are no giant databases for ethical hiring Setting aside ethics, there are so many bright line anti-discrimination rules that I find it hard to believe that an AI could possibly account for them, not without lots of hand-holding. One often forgotten law states that you cannot discriminate against veterans. That is a hard thing for an AI to grasp. Phrases like "served four years in X" is confusing, so too all the military names/units/ranks. But if your AI is even slightly downvoting veterans... good luck in court. What makes that particular law so dangerous is there is no sliding slope. Either someone is a vet or not: a binary choice. So much of the are they/aren't they testing is dead simple. It will be detected and actioned against very quickly.
  • steveBK123 2y ago
    What's kind of funny/telling about the current state of AI is that.. if it really worked as incredibly as all the pumps claim, couldn't you simply train it on all the relevant legal codes by jurisdiction? But not really, its mostly just predicting the next token.
    • _heimdall 2y ago
      More likely than not it would be stuck in a rat nest of contradicting codes and rules. The US Supreme Court ruling with regards to Colorado leaving Trump off the ballot was a complete farse. Their explanation was conveluted and contradictory, and they decided to include answers to questions that weren't directly part of the case. What is an LLM supposed to do with that, and how can an LLM trained on our laws be expected to make use of that when courts can, and sometimes do, go against the rules as written?
      • steveBK123 2y ago
        So you're saying humans can understand how to follow the law better than AI?
        • darepublic 2y ago
• steveBK123 2y ago
  "Weapons of Math Destruction" covered some of these problems, good book. Essentially automating existing biases, and in a way its even more insidious because companies can point to the black box and say "how could it be biased, its a computer!". Then you have companies trying to overcorrect the other way like Google with their AI images fiasco.
  • electrodank 2y ago
    I don’t personally agree with the term “overcorrecting” because they aren’t correcting anything. The output is already correct according to the input (humans behaving as they are). It is not biased. What they are doing is attempting to bias it, and it’s leading to false outputs as a result. Having said that, these false outputs have a strange element of correctness to them in a weird roundabout uncanny valley way: we know the input has been tampered with, and is biased, because the output is obviously wrong. So the algorithm works as intended. If people are discriminatory or racist or sexist, it is not correct to attempt to hide it. The worst possible human behaviours should be a part of a well-formed Turing test. A machine that can reason with an extremist is far more useful than one that an extremist can identify as such.
    • steveBK123 2y ago
      It really was just trading one bias (the existing world as it stands) for another bias (the preferred biases of SF tech lefties) so that was kind of funny in its own way. It would have been one thing if it just randomly assigned gender/race, but it had certain one-way biases (modifying men to women and/or white to non-white) but not the opposite direction... and then being oddly defiant in its responses when people asked for specific demographic outputs. Obviously a lot of this was done by users for the gotcha screen grabs, but in a real world product users may realistically may want specific demographic outputs for example if you are using images for marketing and have specific targeting intent or to match the demographics of your area / business /etc. Stock image websites allow you to search including demographic terms for this reason.
      • AstralStorm 2y ago
• saghm 2y ago
  This is all correct, but it doesn't change make it any less of a real issue because adding an AI intermediate step in the biased process only makes things worse. It's already hard enough to to try to prove or disprove bias in a the current system without companies being able "outsource" the bias to an AI tool and claim ignorance of it. The reason research like this can still be useful is that of the people who write labor laws (and most of the people who vote for them) aren't necessarily going to "understand that the results from any data-based modeling process is a concactination of the cumulative input data topologies and nothing else"; an academic study that makes a specific claim about what results would be expected from using ChatGPT to filter resumes helps people understand without needing domain knowledge.
  • aprilthird2021 2y ago
    Bingo. When suits tell us they plan to replace us with LLMs, that means they also plan to absolve themselves of any guilt for their mistakes, so we should know about the mistakes they make.
• noqc 2y ago
  >It is mathematically impossible to create a “aligned” artificial intelligence towards human goals if humans do not provide demonstration data that is ethical in nature —- which we currently do not incentivize the creation of. That is not what mathematically impossible means.
• mc32 2y ago
  Asking out of ignorance, on average, compared to people without disabilities, do disabled people produce more or less than their non disabled peers, academically speaking? If they are then, an objective system would pick a disabled person, else it should pick a non disabled person, other things being equal or better. As a matter of law and ethics the answer could be different in either case, but objectively, is the system outputting the better answer? Of course laws and ethics may direct us otherwise. Hawking was disabled, but you’d be hard pressed to find a better astrophysicist, for example.
• skeledrew 2y ago
  For the life of me, I don't understand how this almost always misses people: that AI only has data from humanity to learn from, and so every result/action it provides/takes reflects the state of humanity. Even "hallucinations" in some way are likely triggered by content that is for example broken by web sources interspersing unrelated bits such as ads. Or maybe it's a convenient ignorance.
  • tbrownaw 2y ago
    > Even "hallucinations" in some way are likely triggered by content that is for example broken by web sources interspersing unrelated bits such as ads. Or maybe it's a convenient ignorance. They could be something like a compression artifact?
    • skeledrew 2y ago
      Maybe, but I think any obvious software-related side-effects would be accounted for.
  • aprilthird2021 2y ago
    I don't think people are stupid or ignorant. We control the data we train LLMs on. We can, knowing what we know about human biases, introduce and generate data that can contradict these. But we can only do that if we know the biases the LLMs replicate and the contexts where they do.
    • skeledrew 2y ago
      Actually there isn't much "control", beyond the awareness that X model is trained on a dataset scraped from Y, and basic cleaning/sanitizing. There's so much data in use that it'd take decades for a human team to curate or generate in a way that meaningfully balances the datasets. And so models are also used in curation and generation, which themselves are blackboxes...
      • aprilthird2021 2y ago
• Aerroon 2y ago
  >It is mathematically impossible to create a “aligned” artificial intelligence towards human goals if humans do not provide demonstration data that is ethical in nature —- which we currently do not incentivize the creation of. Which also implies that humans aren't aligned with "human goals" in the first place.
• doe_eyes 2y ago
  Hold up. To the best of our knowledge, ChatGPT isn't trained on the behavior of HR departments - or really, it isn't trained on a whole lot of real-world behavioral data at all. It's trained on books, Wikipedia, Reddit, and so on. Even if your assertion that "hiring departments are well known for discriminating" is true, the ChatGPT bias is independent of that and is coming from casual human behavior on social media, not corporate malevolence.
  • Arn_Thor 2y ago
    We really have no idea what the training data is, or how the black box of training integrated that data. Perhaps a subreddit or other forum with hiring managers encouraging each others’ biases ended up weighing heavily. The problem is we don’t know. But whatever the input, the output is less useful, that much is clear
• surfingdino 2y ago
  The problem with ethics is that everyone has their own. Our definition of ethical behaviour also changes over time and between social and cultural groups. It's one of good arguments against training LLMs on past historical data or just giving them all the data we can find and hoping they will come up with the answers we will like.