8 ms·
AI Safety is important, but the unsafety isn't from the superintelligrnt AI, it's from dumb and cruel people hiding behind it as an excuse for their misbehavior
by lupire 4y ago
AI Safety is important, but the unsafety isn't from the superintelligrnt AI, it's from dumb and cruel people hiding behind it as an excuse for their misbehavior.
- PaulHoule 4y agoIt's like automotive safety, safety driving a forklift, etc.
- MonkeyMalarky 4y agoWeapons of Math Destruction is good book on the topic. Using ML to do tasks like evaluate employee performance (and trigger firings), issue loans, insurance etc. is affecting peoples lives in the real world today.
- dekhn 4y agoInsurance has always been a statistically driven process and ML is just an advanced form of stats, so it makes perfect sense that the industry should use this technology.
- twobitshifter 4y agoI know of models that use things like typing speed and birth month to make loan decisions. Feed the AI inputs like race and it will make racist decisions.
- MonkeyMalarky 4y agoFeed a model zipcode information and you've got a pretty decent proxy for race too.
- dekhn 4y agoI am not aware of models that use birth month to make loan decisions, but if that's a useful predictor, why not include it? I'm not really sure exactly what you mean by "feed AI inputs like race and it will make racist decisions", as that's an absurdly general claim that doesn't even really say anything useful. If you mean the model reflects the underlying stats that correlate with race, class, gender, then sure, yes, I agree the predictor will do that. To say that it's racist is an entirely subjective claim. THe point of insurance isn't to avoid social ills, but to statistically minimize the impact of risk on individuals (while making a tidy profit for the insurer).
- PuppyTailWags 4y agoAn insurance model that uses data on financial stability in a society where a specific race of people have been, through bigotry alone, forced into a less stable position by default, will be bound to perpetuate the original bigoted patterns of behavior. It's not inaccurate to describe the model as therefore racist because it's modeling itself off a risk assessment where certain groups have historically been forced into the fringe and are therefore inherently risky. I'm sure if black wall street wasn't razed to the ground by a racist white mob (among several other attempts to gain stability, wealth, etc. by black people being destroyed by racist white anger) then maybe the model wouldn't need to "reflect the underlying stats that correlate with race...". But those underlying stats, and those correlations, didn't just happen in a vacuum, hewn out of the aether like a magical and consequentless thing.
- dekhn 4y agoThe points you are making are entirely political and apply generally, not just to insurance. Insurance is a business and it's not the job of the insurer to correct long-standing social ills.
- PuppyTailWags 4y agoWhether or not my points apply generally does not mean it does not apply to insurance. In fact, it very likely means insurance is a subset of generally, and therefore insurance also has to deal with racism. You cannot extricate models of society from their ills by pretending the ills aren't relevant.
- mrguyorama 4y agoIt's not political to point out bad data will make bad stats.
- dekhn 4y agoIs it bad data if it reflects the underlying observed frequencies regardless of their social origins? I don't think so.
- otabdeveloper4 4y ago> and birth month Holy cow, so literally a horoscope? Lol.
- terminal_d 4y agoI'll have you know that crypto astrologers exist.
- mattmcknight 4y agoI thought it was one of the most deceptive books I ever read! It contradicts itself enough that it is often obviously wrong even without conceptual knowledge of the field.
- WaffleIronMaker 4y agoCan you elaborate on this? I found it to be a very insightful read?
- thatguy0900 4y agoLike the post on here a while ago about ai algorithms setting prices for landlords. Most human have a limit to what they will personally do to someone, but for whatever reason if they get told to do something worse and they can blame it on that thing instead people are willing to be monsters. So it only takes a few sociopaths making the software to make a whole industry even worse.
- UIUC_06 4y agoYou don't even need AI to get this behavior. Merely having something in writing to point to is enough. How often have you talked to someone at an institution about some egregious wrong, and they defend it as "that's our policy?"
- MonkeyMalarky 4y agoOr even just fulfilling a role or persona, like in the Stanford prison experiment.
- adamsmith143 4y ago"Nuclear Weapon Safety is important but the unsafety isn't from the Bombs but from dumb and cruel people hiding behind them as an excuse for their misbehavior."
- deleted 4y ago[deleted]
- willbes 4y agoA book called Automating Inequality by Virginia Eubanks touches on how for many years now, we've hid unfair policies behind computer complexity. It was really eye opening to me how what seems to be a poorly implemented system is actually working as intended in order to make life difficult for certain people.
- marcinzm 4y agoInequality has existed long before computers and is a social problem not a technical problem. You don't solve social problem with technical solutions since people are clever enough to simply find the next tool to enforce the social status quo using.
- willbes 4y agoYou are 100% right. The book just laid out exactly what you are saying. Computers just made the enforcement easier and with less opportunities to break out of it - for example, a sympathetic public servant no longer has the power to make exceptions since "the computer won't allow it."
- wingspar 4y agoNot familiar with the book, but hasn’t ‘sympathetic public servant with power to make exceptions” - major source of inequity? Giving/not giving exceptions for reasons of bias or through bribes?
- willbes 4y agoIt depends on the intent of the rules being enforced. An example given was a state wanted to spend less on low income benefits. They setup an overly complex computer system that decides eligibility and requires participants in the program to fill out multi page forms 100% accurately on a regular basis or their benefits are suspended immediately. There was no way to appeal this decision, only to re-apply. Employees of the state agency no longer had any insight into why a form was rejected, and could not speed up the process to resubmit it because the participant had long term medical issues and needed potentially lifesaving prescriptions filled. As a result, many people who would otherwise be eligible for benefits had them denied. This lead to a marked decrease in spending on social services and was considered a win by the state.
- deleted 4y ago[deleted]
- rsynnott 4y agoAlso, credulous people trusting the magic box too much. Towards the beginning of the pandemic there was a lot of this sort of stuff, say: https://www.nature.com/articles/s42256-021-00338-7 https://www.nature.com/articles/s42256-021-00338-7