4 ms·

Even if you haven't had any contact with the police, they can still put you on the list if they suspect you committed a crime. The manual they follow[0] uses a

Even if you haven't had any contact with the police, they can still put you on the list if they suspect you committed a crime. The manual they follow[0] uses a point-based system, and "suspicions of an offense" counts for half the points of actually committing the offense: > • Half of the applicable point values are awarded for suspicions of an offense listed in 1-4. [0] https://s3.documentcloud.org/documents/20412738/ilp_manual012918.pdf https://s3.documentcloud.org/documents/20412738/ilp_manual01...

heavyset_go 5y ago
Looking at page 13, they talk about their harassment list for at-risk youth, and mention these as 'identifiable risk factors': - Hanging around in public - Being a victim of personal crime - Coming from broken homes Page 72 show that they use records of children who experience physical abuse, sexual abuse, neglect, or witness domestic violence to put them on lists. Page 73 shows the "At-Risk Youth Spreadsheet" which is both terrifying and strikingly banal.
- jtbayly 5y ago
  … and well known. For example, we have known that broken homes lead to all sorts of problems a much greater proportion of the time for ages. And one of the things that becomes more likely is crime. Not that that makes a computer algorithm’s predictions true. Just saying that it’s not terrifying to point out that there really are such things that increase risk that we are well aware of.
  - easterncalculus 5y ago
    This in my opinion is a point that brings a lot of well-needed nuance to this issue. All or at least many of these facts or assumptions are all relatively reasonable or well-studied, that past criminal records and traumatic upbringings can lead to crime, but when put together under the dubious question of is the person a criminal can be greater than the sum of their parts. A lot of people come from broken homes or experience trauma and never commit a(nother) crime. The fact that these assumptions are standard practice has only legitimized taking a black-box algorithm at its word. It's not all just bias, but this kind of system certainly can incentivize it when making seemingly authoritative predictions.
    - vkou 5y ago
      The nuance doesn't matter, though. If you tell a cop that person A is 1% likely to be guilty of a crime, and person B is 3% likely to be guilty of a crime, they are going to focus their efforts on person B - despite both people being more likely than not, to be innocent. When being harassed by police is itself sufficient to turn a person into a criminal, or ruin their life, you end up in a situation where everyone is doing what is 'reasonable', but leading to outcomes that are both unfair, and even negative overall, to society.
      - easterncalculus 5y ago
        > When being harassed by police is itself sufficient to turn a person into a criminal, or ruin their life, you end up in a situation where everyone is doing what is 'reasonable', but leading to outcomes that are both unfair, and even negative overall, to society. I agree. I just think that when critics read articles like the one in this post, they might walk away with the idea that all of these assumptions are fine and that critics of these systems are being alarmist. They might see these metrics as reasonable considering they reflect trends in actual criminals, but not understand that they are being used to predict criminal behavior, which like you mentioned with the two almost definitely innocent people is not the same thing.
  - 0134340 5y ago
    >Just saying that it’s not terrifying If you think China's social credit system isn't terrifying, then I guess I wouldn't be surprised if you didn't find this terrifying.
    - jtbayly 5y ago
      I said it’s not terrifying to point those facts out. I do think trying to use them in a prediction system is terrifying.
  - heavyset_go 5y ago
    This article about the program[1] directly addresses your assertion, specifically by one of the professors whose research was used to justify the program: > The Sheriff’s Office says its program is based on research. It points to a 2015 study that found young people who had experienced multiple childhood traumas were at a higher risk of becoming serious, violent criminals than those who hadn’t. > But David Kennedy, a renowned criminologist and professor at the John Jay College of Criminal Justice whose research is referenced in Pasco’s manual, said the associations between childhood trauma and criminal behavior are “extremely weak.” He said using them to make predictions about individuals “flies in the face of the science.” > The methodology used by the Sheriff’s Office, he added, was likely to generate a large pool of children, the vast majority of whom would never get in serious trouble. > “There’s nothing — absolutely nothing — that can be fed into even the most sophisticated algorithm or risk-assessment tool based on information available when someone is a child that can say this person is going to be a criminal later on, much less a serious prolific criminal,” he said. [1] https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/school-data/ https://projects.tampabay.com/projects/2020/investigations/p...
    - jtbayly 5y ago
      I think making predictions from these things is absurd (and terrifying).

heavyset_go 5y ago

Looking at page 13, they talk about their harassment list for at-risk youth, and mention these as 'identifiable risk factors': - Hanging around in public - Being a victim of personal crime - Coming from broken homes Page 72 show that they use records of children who experience physical abuse, sexual abuse, neglect, or witness domestic violence to put them on lists. Page 73 shows the "At-Risk Youth Spreadsheet" which is both terrifying and strikingly banal.

jtbayly 5y ago
… and well known. For example, we have known that broken homes lead to all sorts of problems a much greater proportion of the time for ages. And one of the things that becomes more likely is crime. Not that that makes a computer algorithm’s predictions true. Just saying that it’s not terrifying to point out that there really are such things that increase risk that we are well aware of.
- easterncalculus 5y ago
  This in my opinion is a point that brings a lot of well-needed nuance to this issue. All or at least many of these facts or assumptions are all relatively reasonable or well-studied, that past criminal records and traumatic upbringings can lead to crime, but when put together under the dubious question of is the person a criminal can be greater than the sum of their parts. A lot of people come from broken homes or experience trauma and never commit a(nother) crime. The fact that these assumptions are standard practice has only legitimized taking a black-box algorithm at its word. It's not all just bias, but this kind of system certainly can incentivize it when making seemingly authoritative predictions.
  - vkou 5y ago
    The nuance doesn't matter, though. If you tell a cop that person A is 1% likely to be guilty of a crime, and person B is 3% likely to be guilty of a crime, they are going to focus their efforts on person B - despite both people being more likely than not, to be innocent. When being harassed by police is itself sufficient to turn a person into a criminal, or ruin their life, you end up in a situation where everyone is doing what is 'reasonable', but leading to outcomes that are both unfair, and even negative overall, to society.
    - easterncalculus 5y ago
      > When being harassed by police is itself sufficient to turn a person into a criminal, or ruin their life, you end up in a situation where everyone is doing what is 'reasonable', but leading to outcomes that are both unfair, and even negative overall, to society. I agree. I just think that when critics read articles like the one in this post, they might walk away with the idea that all of these assumptions are fine and that critics of these systems are being alarmist. They might see these metrics as reasonable considering they reflect trends in actual criminals, but not understand that they are being used to predict criminal behavior, which like you mentioned with the two almost definitely innocent people is not the same thing.
- 0134340 5y ago
  >Just saying that it’s not terrifying If you think China's social credit system isn't terrifying, then I guess I wouldn't be surprised if you didn't find this terrifying.
  - jtbayly 5y ago
    I said it’s not terrifying to point those facts out. I do think trying to use them in a prediction system is terrifying.
- heavyset_go 5y ago
  This article about the program[1] directly addresses your assertion, specifically by one of the professors whose research was used to justify the program: > The Sheriff’s Office says its program is based on research. It points to a 2015 study that found young people who had experienced multiple childhood traumas were at a higher risk of becoming serious, violent criminals than those who hadn’t. > But David Kennedy, a renowned criminologist and professor at the John Jay College of Criminal Justice whose research is referenced in Pasco’s manual, said the associations between childhood trauma and criminal behavior are “extremely weak.” He said using them to make predictions about individuals “flies in the face of the science.” > The methodology used by the Sheriff’s Office, he added, was likely to generate a large pool of children, the vast majority of whom would never get in serious trouble. > “There’s nothing — absolutely nothing — that can be fed into even the most sophisticated algorithm or risk-assessment tool based on information available when someone is a child that can say this person is going to be a criminal later on, much less a serious prolific criminal,” he said. [1] https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/school-data/ https://projects.tampabay.com/projects/2020/investigations/p...
  - jtbayly 5y ago
    I think making predictions from these things is absurd (and terrifying).