5 ms·

I'm not sure how you think my list would bifurcate into multiple lists. > It's obviously caused by studying the people who present, and then demographics chang

I'm not sure how you think my list would bifurcate into multiple lists. > It's obviously caused by studying the people who present, and then demographics changing. Nobody made a decision here with any ill intent. Both the hospital and the insurance company share the patient's interest in them getting better and are already looking for better data. Not a single example I gave was the result of ill intent. That's literally the point of my list. There's a difference between systemic bias and intentional bias. These are examples of systemic bias. > A higher chance of any offense, or of being a burden to society? I provided the link which answers your question. In this case, it predicted a higher likelihood of future crime for a kid who attempted to steal someone's bike and scooter, than for an adult who had shoplifted, been previously convicted of armed robbery, and served 5 years in prison already. > Kids are more likely to commit smaller crimes, and those are a gateway to larger crimes that then put you away until you're a hardened repeat felon. Your logic seems to acknowledge that one of the large drivers of crime is being "put away" in prison. In this case, the algorithm predicted a higher likelihood of committing future crimes for a kid than for someone who had already served 5 years in prison for armed robbery. Even worse is that the algorithm was being used to determine the severity of their respective punishments. So, the act of predicting a higher likelihood of future crime for the kid becomes a self-fulfilling prophecy, giving her a harsher sentence, which in turn is more likely to drive her toward future crime. This is systemic bias in action.

Rule35 6y ago
> I'm not sure how you think my list would bifurcate into multiple lists. Because the two things I pulled from your list aren't systematic. One was local to a specific health system and about a specific condition, and the other about kids from a certain area. Neither was just about black people in general. Your list is predominantly racial and I thought that was the point. This medical thing is really about any group not being recognized as different enough from the whole, women get this a lot, but even men do in areas where women are the primary sufferers of something. (Breast-cancer for instance.) Also, to distinguish things that are intentional (redlining) vs absolutely unforeseen but still problematic. A few absolutely undeniable things, and they exist, would make the list more meaningful and perhaps give more weight to the rest which are a bit ambiguous. > There's a difference between systemic bias and intentional bias. These are examples of systemic bias. No, they're an example of non-systemic bias. The aren't broadly across any system, or symptomatic of a general attitude. Systematic bias doesn't mean that the majority gets more attention, it means that the entire system is biased against something, and you aren't showing that. > Your logic seems to acknowledge that one of the large drivers of crime is being "put away" in prison. It is a separate topic, but yes. Being put away unjustly, and in the wrong place or severity for your crime hurts. But not serving a sentence for crime is just as bad, if not worse. I'm not against jailing criminals, just doing it where it doesn't serve us or them. > In this case, the algorithm predicted a higher likelihood of committing future crimes for a kid than for someone who had already served 5 years in prison for armed robbery. Could be. Without knowing the severity of the crime predicted though it's kind of meaningless. > Even worse is that the algorithm was being used to determine the severity of their respective punishments. Well, without knowing the prediction you can't know if a stiffer sentence would have encouraged or deterred future crime. If they were predicted to be killers, a hard but fair sentence at the first crimes could turn them around. If they're predicted to steal a few cars then even a year in prison is probably going to make them worse. > So, the act of predicting a higher likelihood of future crime for the kid becomes a self-fulfilling prophecy, giving her a harsher sentence, which in turn is more likely to drive her toward future crime. This is systemic bias in action. It would be, if it were system wide. It was a specific population though, and there's still no indicator that it said black kids specifically other than because in those neighborhoods they were the ones at risk. fwiw, stealing scooters from children doesn't seem like the path to good behavior. Systematic risk is AIs for this purpose in regard to anyone, not that they targeted black people more in some cases. That's essentially random.
- JangoSteve 6y ago
  > Your list is predominantly racial and I thought that was the point. Ah, no, that wasn't the point. They were just examples of systemic bias. The fact that so many are racial is just a reflection of the unfortunate state we find ourselves in. > A few absolutely undeniable things, and they exist, would make the list more meaningful and perhaps give more weight to the rest which are a bit ambiguous. I honestly don't know how any of the examples posted are deniable, they're all well founded with research and evidence. But you're right that adding more to the list can only help. > Systematic bias doesn't mean that the majority gets more attention, it means that the entire system is biased against something, and you aren't showing that. That's not what systemic bias means. "Systemic bias, also called institutional bias, and related to structural bias, is the inherent tendency of a process to support particular outcomes." https://en.wikipedia.org/wiki/Systemic_bias https://en.wikipedia.org/wiki/Systemic_bias Also, examples are evidence which support a theory (of systemic bias in this case). Any one data point is rarely intended to be absolute proof of that theory, much less a description of its scope in its entirety. All of these examples do indeed provide evidence toward the existence of systemic bias within systems without fully describing or proving the extent of the entire scope of systemic bias within those systems. > But not serving a sentence for crime is just as bad, if not worse. The article I linked explains that this is not what the algorithm was used for. It was not used to decide whether or not to sentence them, it was used to help decide the severity of their sentence. > If they were predicted to be killers, a hard but fair sentence at the first crimes could turn them around. If they're predicted to steal a few cars then even a year in prison is probably going to make them worse. You seem to be agreeing that this was in fact a badly biased algorithm, since it was recommending a stiffer punishment for a kid who attempted to steal a bike than for a repeat felon, previously convicted of armed robbery, who shoplifted. > It would be, if it were system wide. It was a specific population though, and there's still no indicator that it said black kids specifically other than because in those neighborhoods they were the ones at risk. fwiw, stealing scooters from children doesn't seem like the path to good behavior. If something affects a population, that is system-wide by definition, since the definition of "system" is pretty broad; see the description above. Regardless of whether or not you think stealing scooters is a good prediction, the fact here is that it rated the likelihood of future crime as higher than for someone who was already a career criminal. > Systematic risk is AIs for this purpose in regard to anyone, not that they targeted black people more in some cases. That's essentially random. If only. Unfortunately, this is seen over and over again, more so than would happen by randomness. That's the point of the list.

Rule35 6y ago

> I'm not sure how you think my list would bifurcate into multiple lists. Because the two things I pulled from your list aren't systematic. One was local to a specific health system and about a specific condition, and the other about kids from a certain area. Neither was just about black people in general. Your list is predominantly racial and I thought that was the point. This medical thing is really about any group not being recognized as different enough from the whole, women get this a lot, but even men do in areas where women are the primary sufferers of something. (Breast-cancer for instance.) Also, to distinguish things that are intentional (redlining) vs absolutely unforeseen but still problematic. A few absolutely undeniable things, and they exist, would make the list more meaningful and perhaps give more weight to the rest which are a bit ambiguous. > There's a difference between systemic bias and intentional bias. These are examples of systemic bias. No, they're an example of non-systemic bias. The aren't broadly across any system, or symptomatic of a general attitude. Systematic bias doesn't mean that the majority gets more attention, it means that the entire system is biased against something, and you aren't showing that. > Your logic seems to acknowledge that one of the large drivers of crime is being "put away" in prison. It is a separate topic, but yes. Being put away unjustly, and in the wrong place or severity for your crime hurts. But not serving a sentence for crime is just as bad, if not worse. I'm not against jailing criminals, just doing it where it doesn't serve us or them. > In this case, the algorithm predicted a higher likelihood of committing future crimes for a kid than for someone who had already served 5 years in prison for armed robbery. Could be. Without knowing the severity of the crime predicted though it's kind of meaningless. > Even worse is that the algorithm was being used to determine the severity of their respective punishments. Well, without knowing the prediction you can't know if a stiffer sentence would have encouraged or deterred future crime. If they were predicted to be killers, a hard but fair sentence at the first crimes could turn them around. If they're predicted to steal a few cars then even a year in prison is probably going to make them worse. > So, the act of predicting a higher likelihood of future crime for the kid becomes a self-fulfilling prophecy, giving her a harsher sentence, which in turn is more likely to drive her toward future crime. This is systemic bias in action. It would be, if it were system wide. It was a specific population though, and there's still no indicator that it said black kids specifically other than because in those neighborhoods they were the ones at risk. fwiw, stealing scooters from children doesn't seem like the path to good behavior. Systematic risk is AIs for this purpose in regard to anyone, not that they targeted black people more in some cases. That's essentially random.

JangoSteve 6y ago
> Your list is predominantly racial and I thought that was the point. Ah, no, that wasn't the point. They were just examples of systemic bias. The fact that so many are racial is just a reflection of the unfortunate state we find ourselves in. > A few absolutely undeniable things, and they exist, would make the list more meaningful and perhaps give more weight to the rest which are a bit ambiguous. I honestly don't know how any of the examples posted are deniable, they're all well founded with research and evidence. But you're right that adding more to the list can only help. > Systematic bias doesn't mean that the majority gets more attention, it means that the entire system is biased against something, and you aren't showing that. That's not what systemic bias means. "Systemic bias, also called institutional bias, and related to structural bias, is the inherent tendency of a process to support particular outcomes." https://en.wikipedia.org/wiki/Systemic_bias https://en.wikipedia.org/wiki/Systemic_bias Also, examples are evidence which support a theory (of systemic bias in this case). Any one data point is rarely intended to be absolute proof of that theory, much less a description of its scope in its entirety. All of these examples do indeed provide evidence toward the existence of systemic bias within systems without fully describing or proving the extent of the entire scope of systemic bias within those systems. > But not serving a sentence for crime is just as bad, if not worse. The article I linked explains that this is not what the algorithm was used for. It was not used to decide whether or not to sentence them, it was used to help decide the severity of their sentence. > If they were predicted to be killers, a hard but fair sentence at the first crimes could turn them around. If they're predicted to steal a few cars then even a year in prison is probably going to make them worse. You seem to be agreeing that this was in fact a badly biased algorithm, since it was recommending a stiffer punishment for a kid who attempted to steal a bike than for a repeat felon, previously convicted of armed robbery, who shoplifted. > It would be, if it were system wide. It was a specific population though, and there's still no indicator that it said black kids specifically other than because in those neighborhoods they were the ones at risk. fwiw, stealing scooters from children doesn't seem like the path to good behavior. If something affects a population, that is system-wide by definition, since the definition of "system" is pretty broad; see the description above. Regardless of whether or not you think stealing scooters is a good prediction, the fact here is that it rated the likelihood of future crime as higher than for someone who was already a career criminal. > Systematic risk is AIs for this purpose in regard to anyone, not that they targeted black people more in some cases. That's essentially random. If only. Unfortunately, this is seen over and over again, more so than would happen by randomness. That's the point of the list.