3 ms·

I'm not upset about what they measured. I'm upset about what they claim they learned from it and about the kinds of causation-related lemmata they (erroneously)

I'm not upset about what they measured. I'm upset about what they claim they learned from it and about the kinds of causation-related lemmata they (erroneously) assert in the report.

dpark 9y ago
Firstly, you're critiquing a study based on a journalist's write-up about it. Secondly, you're essentially making an epistemological argument. You could apply your core argument to almost anything and come up with the assertion that the research is invalid. "Researchers tracked use-of-force incidents" -> "Researchers tracked reports of use-of-force incidents, which is factually rather different. "Researchers observed bosons at 125GeV" -> "Researchers observed detections of bosons at 125GeV, which is factually rather different." This is merely a pedantic argument that things are not truly knowable. But whether the Higgs boson exists and whether body cameras reduce police violence are not unknowable questions. Even if we cannot directly measure, the indirect measures are meaningful to the questions. Now, if there's something else that the researchers should measure, then that would be useful criticism.
- spurcell93 9y ago
  I agree... at some point, when analyzing a topic, we need to prune the decision tree leading to some tentative conclusions, even if some subtrees might not be 100% verifiably invalid. Our science is imperfect, reporting is imperfect, and very few (if any) complete-information scenarios exist.
- jamesw6811 9y ago
  There is a pretty big difference in your two examples. Models of bosons behave in a predictable way. I can think of several likely reasons that reports of use-of-force incidents would not correlate directly with actual use-of-force. As a result, in a study or a journalistic write-up of said study, I wouldn't jump to the shorthand of "tracked use-of-force incidents" as opposed to "reports of use-of-force incidents". Just because something is the best way of measuring a quantity we currently have doesn't mean it is good enough to equate the measurement with the quantity in shorthand.
  - dpark 9y ago
    I think the examples track pretty well. You could say that body cameras somehow cause a rise and reporting that exactly offsets the reduction in violence, but that’s not even wild conjecture. It’s worse. It’s inventing factors to make the data fit, factors that are by definition not measurable. Whatever we measure, it’s possible to invent a factor that makes the measurement completely wrong. Scientists could similarly argue that there is another particle that only shows up when we measure that happens to look exactly like the Higgs boson in every way we can measure. But then everyone would call bullshit. Again, if there are better/other things to measure, that’s legitimate criticism. Simply inventing factors that make the valid measurements null and declaring it impossible to actually measure isn’t.
    - BugsJustFindMe 9y ago
      Measure what you can measure. Claim what you can claim. Don't claim what you can't claim based on what you measured. We're not talking about counting photons here. We're talking about human selective self-reporting under different circumstances. Understand the chasm between those things.
- BugsJustFindMe 9y ago
  > Firstly, you're critiquing a study based on a journalist's write-up about it. I'm critiquing a political abuse of a bogus study and also the bogus study itself, based on the study's language and based on the political website referenced by the journalist's write-up, by actually reading them. You can see them too at http://bwc.thelab.dc.gov/ http://bwc.thelab.dc.gov/ and http://bwc.thelab.dc.gov/static/files/supplementary-materials.pdf http://bwc.thelab.dc.gov/static/files/supplementary-material... > This is merely a pedantic argument that things are not truly knowable. Bullshit. I'm telling you that correlation doesn't imply causation and rather than saying "oh yeah, I heard that once somewhere, it was probably important" your response is "but physics jargon". They are measuring one thing and making, and these next words are key, an unreasonable leap onto something very different. In your boson world nobody ever doesn't report something that happened. In the real world that is very clearly false. Even a mediocre scientist accounts for blindingly obvious confounding factors. A better scientist accounts for things that don't even seem like they could possibly be confounding factors just because they are differences. Neither of those appear to have happened here. They assume that reporting rates are predictable for different circumstances just by guessing, which is not a reliable assumption. And on top of all of that, this report is likely going to be used as a giant war banner in the police union's fight to reduce the use of body cameras. And it just slides right over the fact that cameras actually do see police committing crimes and abuses for which they then don't get significantly punished and which would never have been uncovered otherwise. It arbitrarily privileges "reported uses of force" as the target of attention rather than "witnessed crimes and other violations". Why? The report isn't titled "Evaluating the Effects of Police Body-Worn Cameras on Uses of Force: A Randomized Controlled Trial". It's called "Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial" full stop. And yet they are clearly ignoring huge categories of observed malfeasance in order to tell a particular dismissive story.
  - dpark 9y ago
    > I'm critiquing a political abuse of a bogus study and also the bogus study itself... It's disappointing but not surprising that you accuse them of "political abuse". Your own criticism seems clearly political because you start with the premise that it's literally impossible that they are correct. I actually agree with you that the results here feel suspect because they run so counter to my expectations, but I find it sad that this is how you put forward your agenda. > Bullshit. I'm telling you that correlation doesn't imply causation and rather than saying "oh yeah, I heard that once somewhere, it was probably important" your response is "but physics jargon". "Correlation is not causation" is a great response to erroneous conclusions drawn from studies constructed to establish correlation. It's a really poor response to a study designed to establish causation. If you believe that it is in general impossible to establish causation, then you are indeed just making an epistemological argument. If you have a problem with the construction of the study and believe it doesn't establish causation, because of X, Y, and Z, then provide that criticism. Don't assert that it's wrong because measurement is impossible. Even if we accept that claim as true, it's still a useless criticism because it means the best we can do is our indirect measurements and we are left with no option except to optimize for those. > They are measuring one thing and making, and these next words are key, an unreasonable leap onto something very different. It's not an unreasonable leap. A priori, many people would agree that a reduction in police use of force would result in a reduction of reports of use of force by both police and civilians. Previous studies indicated the same a posteriori. Clearly many people find the leap very reasonable. I believe the only reason you find it unreasonable is that you don't like the conclusion it leads to in this study. > In your boson world nobody ever doesn't report something that happened. In the real world that is very clearly false. Not sure what you're talking about. Erroneous measurements happen all the time in the scientific world. It was really hard for scientists to prove the existence of the Higgs boson because it's so hard to even detect. > They assume that reporting rates are predictable for different circumstances just by guessing, which is not a reliable assumption. Where does this claim come from? Did I miss something where the authors claimed that reporting would definitely be 100% consistent? And where does the support for your claim that reported rates are essentially constant regardless of actual rates come from? > And on top of all of that, this report is likely going to be used as a giant war banner in the police union's fight to reduce the use of body cameras.... And here you go with the actual justification for your criticism. You don't have a scientific criticism. You have a political one. You want to discredit the study because you don't like the conclusion. Maybe this study is flawed, but you don't establish that by asserting that it's impossible to even measure or by claiming (without any evidence of your own) that there's no link between actual use of force and reports of use of force. > It arbitrarily privileges "reported uses of force" as the target of attention rather than "witnessed crimes and other violations". Exactly how do you imagine that you would track "witnessed crimes and other violations" in absence of reports of those things?

I'm not upset about what they measured. I'm upset about what they claim they learned from it and about the kinds of causation-related lemmata they (erroneously) assert in the report.

dpark 9y ago

Firstly, you're critiquing a study based on a journalist's write-up about it. Secondly, you're essentially making an epistemological argument. You could apply your core argument to almost anything and come up with the assertion that the research is invalid. "Researchers tracked use-of-force incidents" -> "Researchers tracked reports of use-of-force incidents, which is factually rather different. "Researchers observed bosons at 125GeV" -> "Researchers observed detections of bosons at 125GeV, which is factually rather different." This is merely a pedantic argument that things are not truly knowable. But whether the Higgs boson exists and whether body cameras reduce police violence are not unknowable questions. Even if we cannot directly measure, the indirect measures are meaningful to the questions. Now, if there's something else that the researchers should measure, then that would be useful criticism.

spurcell93 9y ago
I agree... at some point, when analyzing a topic, we need to prune the decision tree leading to some tentative conclusions, even if some subtrees might not be 100% verifiably invalid. Our science is imperfect, reporting is imperfect, and very few (if any) complete-information scenarios exist.
jamesw6811 9y ago
There is a pretty big difference in your two examples. Models of bosons behave in a predictable way. I can think of several likely reasons that reports of use-of-force incidents would not correlate directly with actual use-of-force. As a result, in a study or a journalistic write-up of said study, I wouldn't jump to the shorthand of "tracked use-of-force incidents" as opposed to "reports of use-of-force incidents". Just because something is the best way of measuring a quantity we currently have doesn't mean it is good enough to equate the measurement with the quantity in shorthand.
- dpark 9y ago
  I think the examples track pretty well. You could say that body cameras somehow cause a rise and reporting that exactly offsets the reduction in violence, but that’s not even wild conjecture. It’s worse. It’s inventing factors to make the data fit, factors that are by definition not measurable. Whatever we measure, it’s possible to invent a factor that makes the measurement completely wrong. Scientists could similarly argue that there is another particle that only shows up when we measure that happens to look exactly like the Higgs boson in every way we can measure. But then everyone would call bullshit. Again, if there are better/other things to measure, that’s legitimate criticism. Simply inventing factors that make the valid measurements null and declaring it impossible to actually measure isn’t.
  - BugsJustFindMe 9y ago
    Measure what you can measure. Claim what you can claim. Don't claim what you can't claim based on what you measured. We're not talking about counting photons here. We're talking about human selective self-reporting under different circumstances. Understand the chasm between those things.
BugsJustFindMe 9y ago
> Firstly, you're critiquing a study based on a journalist's write-up about it. I'm critiquing a political abuse of a bogus study and also the bogus study itself, based on the study's language and based on the political website referenced by the journalist's write-up, by actually reading them. You can see them too at http://bwc.thelab.dc.gov/ http://bwc.thelab.dc.gov/ and http://bwc.thelab.dc.gov/static/files/supplementary-materials.pdf http://bwc.thelab.dc.gov/static/files/supplementary-material... > This is merely a pedantic argument that things are not truly knowable. Bullshit. I'm telling you that correlation doesn't imply causation and rather than saying "oh yeah, I heard that once somewhere, it was probably important" your response is "but physics jargon". They are measuring one thing and making, and these next words are key, an unreasonable leap onto something very different. In your boson world nobody ever doesn't report something that happened. In the real world that is very clearly false. Even a mediocre scientist accounts for blindingly obvious confounding factors. A better scientist accounts for things that don't even seem like they could possibly be confounding factors just because they are differences. Neither of those appear to have happened here. They assume that reporting rates are predictable for different circumstances just by guessing, which is not a reliable assumption. And on top of all of that, this report is likely going to be used as a giant war banner in the police union's fight to reduce the use of body cameras. And it just slides right over the fact that cameras actually do see police committing crimes and abuses for which they then don't get significantly punished and which would never have been uncovered otherwise. It arbitrarily privileges "reported uses of force" as the target of attention rather than "witnessed crimes and other violations". Why? The report isn't titled "Evaluating the Effects of Police Body-Worn Cameras on Uses of Force: A Randomized Controlled Trial". It's called "Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial" full stop. And yet they are clearly ignoring huge categories of observed malfeasance in order to tell a particular dismissive story.
- dpark 9y ago
  > I'm critiquing a political abuse of a bogus study and also the bogus study itself... It's disappointing but not surprising that you accuse them of "political abuse". Your own criticism seems clearly political because you start with the premise that it's literally impossible that they are correct. I actually agree with you that the results here feel suspect because they run so counter to my expectations, but I find it sad that this is how you put forward your agenda. > Bullshit. I'm telling you that correlation doesn't imply causation and rather than saying "oh yeah, I heard that once somewhere, it was probably important" your response is "but physics jargon". "Correlation is not causation" is a great response to erroneous conclusions drawn from studies constructed to establish correlation. It's a really poor response to a study designed to establish causation. If you believe that it is in general impossible to establish causation, then you are indeed just making an epistemological argument. If you have a problem with the construction of the study and believe it doesn't establish causation, because of X, Y, and Z, then provide that criticism. Don't assert that it's wrong because measurement is impossible. Even if we accept that claim as true, it's still a useless criticism because it means the best we can do is our indirect measurements and we are left with no option except to optimize for those. > They are measuring one thing and making, and these next words are key, an unreasonable leap onto something very different. It's not an unreasonable leap. A priori, many people would agree that a reduction in police use of force would result in a reduction of reports of use of force by both police and civilians. Previous studies indicated the same a posteriori. Clearly many people find the leap very reasonable. I believe the only reason you find it unreasonable is that you don't like the conclusion it leads to in this study. > In your boson world nobody ever doesn't report something that happened. In the real world that is very clearly false. Not sure what you're talking about. Erroneous measurements happen all the time in the scientific world. It was really hard for scientists to prove the existence of the Higgs boson because it's so hard to even detect. > They assume that reporting rates are predictable for different circumstances just by guessing, which is not a reliable assumption. Where does this claim come from? Did I miss something where the authors claimed that reporting would definitely be 100% consistent? And where does the support for your claim that reported rates are essentially constant regardless of actual rates come from? > And on top of all of that, this report is likely going to be used as a giant war banner in the police union's fight to reduce the use of body cameras.... And here you go with the actual justification for your criticism. You don't have a scientific criticism. You have a political one. You want to discredit the study because you don't like the conclusion. Maybe this study is flawed, but you don't establish that by asserting that it's impossible to even measure or by claiming (without any evidence of your own) that there's no link between actual use of force and reports of use of force. > It arbitrarily privileges "reported uses of force" as the target of attention rather than "witnessed crimes and other violations". Exactly how do you imagine that you would track "witnessed crimes and other violations" in absence of reports of those things?