5 ms·

At the start of the pandemic, an academic at a well-known university asked me to review a paper of his, kinda sorta to build a relationship and to eventually pu

by chaps 5y ago

At the start of the pandemic, an academic at a well-known university asked me to review a paper of his, kinda sorta to build a relationship and to eventually publish something of my own based on his paper. The claim of the paper was basically that white (non-hispanic) communities were significantly more affected by COVID when compared against other races in similar income brackets. Lots of statistics, analysis against census data, etc to support the argument and such. The source of the data from his paper was from an "open data" portal of COVID deaths, published by the city's coroner's office. Thing is, the data published by the coroner's office didn't include a lot of fields that I'd already received from FOIA. Fields missing like the name of the person who died. Since the paper's conclusions didn't pass a gut check, I dug in and found that just glancing through the names of those marked down as "white", a majority clearly had names with Hispanic origins. I shared with the researcher that his findings were probably really, really wrong and I was responded with something like, "even if 50% are marked wrong, the findings are still accurate". Some name comparison [1] and spot checking later.. and it ended up being about 80% of hispanic-origin names were marked as "white". The researcher eventually conceded that the paper was wrong, but it was definitely of an eye opening experience into how the academic world works. I lost a lot of faith in the academic world after that. [1] https://www.census.gov/data/developers/data-sets/surnames.html https://www.census.gov/data/developers/data-sets/surnames.ht...

drugstorecowboy 5y ago
I am definitely not a scientist, but speculating on race because of names you find to be too hispanic to be white feels like spectacularly bad science to me. This is not my field though so perhaps that is normal an accepted.
- chaps 5y ago
  The point of name-based analysis wasn't for publication or even "science", but to instead show that the analysis done by the researcher was very likely wrong in order to prevent its publication. There is no way in hell I would ever publish anything that relied on name-based analysis as an accurate measure, that would be deeply unethical as far as I'm concerned. I did ask around with these exact concerns in-mind, which led to this paper by the census in their evaluation of this exact problem (the Passel-Word Spanish Surname List was what I compared against): https://www.census.gov/library/working-papers/1993/demo/POP-twps0004.html https://www.census.gov/library/working-papers/1993/demo/POP-... So, I get it, and I had the same concerns as you, but I like to believe I followed proper due diligence in making sure my approach was reasonable. Especially against the risk of the paper being published (consider how the paper could be used in racist narratives).
  - drugstorecowboy 5y ago
    That makes sense, thank you for the clarification. I apologize, I should not have worded my former comment so strongly.
    - chaps 5y ago
      No worries!
- s1artibartfast 5y ago
  There are some large datasets for name-race correlations that can and have been used in real science. It is statistical, so not an accurate prediction for an individual, but has some merit for large groups. For example, someone with the name José Santiago Hernández very is unlikely to be Germanic-white versus Hispanic-white.
bachmeier 5y ago
> I lost a lot of faith in the academic world after that. I would argue instead that you have demonstrated the importance of peer review (a cornerstone of academic research). The problem is not really academic research, but that we do too little of both academic research and peer review.
- chaps 5y ago
  So what you're saying is that the current state of the academic world isn't working as it's theoretically designed to? Frankly, I'm not sure how that's supposed to boost my faith.
  - bachmeier 5y ago
    If you have a very profitable product but the manager decides not to produce much of it, the problem isn't the product, it's that there's too little of the product to help the company's bottom line. If there was more peer review of the type you described and there were more studies, we wouldn't be inflating the importance of unreliable results because we'd have ten other studies pointing out that it's BS.
- erdos4d 5y ago
  I dunno, I've met a lot of profs who take the attitude "if you want your name on crap, be my guest" when it comes to reviewing papers. They might check for glaring errors, but rarely does anyone work through the results for correctness.
  - bachmeier 5y ago
    I agree but that's a sign we need more peer review. IMO it's unethical to engage in fake peer review like that. I recently encountered the opposite - a reviewer that wrote a fake negative review of my paper without reading it. Killed the paper at a good journal by making claims that are easily verified to be false.
    - nicce 5y ago
fluoridation 5y ago
Surname is correlated with descent (which is distinct from race, as people understand that word), while in all likelihood the races that were written down for the patients corresponded to the color of their skin, as well as possibly other features. That would suggest three things: First, as you found, there was no correlation between deaths and descent. Presumably there weren't statistically anomalously fewer Gomezes than Smiths. This doesn't eliminate the fact that people wrote down fewer dead as Hispanic than as white. So: Second, there actually were fewer dead Hispanics than dead whites, understanding "Hispanic" as "a person who looks Hispanic" and "white" as "a person who looks white". Third, or alternatively there is no statistical difference between dead Hispanics and dead whites, but somehow fewer dead Hispanics ended up at the morgues to be written down as dead.
- chaps 5y ago
  The field in question was filled out by the families of the deceased. You're making some really wild conclusions here.
  - fluoridation 5y ago
    Ah, I didn't know that. I assumed it was filled by the coroner. Never mind, then. That would certainly explain the anomaly away.

by chaps 5y ago

drugstorecowboy 5y ago

I am definitely not a scientist, but speculating on race because of names you find to be too hispanic to be white feels like spectacularly bad science to me. This is not my field though so perhaps that is normal an accepted.

chaps 5y ago
The point of name-based analysis wasn't for publication or even "science", but to instead show that the analysis done by the researcher was very likely wrong in order to prevent its publication. There is no way in hell I would ever publish anything that relied on name-based analysis as an accurate measure, that would be deeply unethical as far as I'm concerned. I did ask around with these exact concerns in-mind, which led to this paper by the census in their evaluation of this exact problem (the Passel-Word Spanish Surname List was what I compared against): https://www.census.gov/library/working-papers/1993/demo/POP-twps0004.html https://www.census.gov/library/working-papers/1993/demo/POP-... So, I get it, and I had the same concerns as you, but I like to believe I followed proper due diligence in making sure my approach was reasonable. Especially against the risk of the paper being published (consider how the paper could be used in racist narratives).
- drugstorecowboy 5y ago
  That makes sense, thank you for the clarification. I apologize, I should not have worded my former comment so strongly.
  - chaps 5y ago
    No worries!
s1artibartfast 5y ago
There are some large datasets for name-race correlations that can and have been used in real science. It is statistical, so not an accurate prediction for an individual, but has some merit for large groups. For example, someone with the name José Santiago Hernández very is unlikely to be Germanic-white versus Hispanic-white.

bachmeier 5y ago

> I lost a lot of faith in the academic world after that. I would argue instead that you have demonstrated the importance of peer review (a cornerstone of academic research). The problem is not really academic research, but that we do too little of both academic research and peer review.

chaps 5y ago
So what you're saying is that the current state of the academic world isn't working as it's theoretically designed to? Frankly, I'm not sure how that's supposed to boost my faith.
- bachmeier 5y ago
  If you have a very profitable product but the manager decides not to produce much of it, the problem isn't the product, it's that there's too little of the product to help the company's bottom line. If there was more peer review of the type you described and there were more studies, we wouldn't be inflating the importance of unreliable results because we'd have ten other studies pointing out that it's BS.
erdos4d 5y ago
I dunno, I've met a lot of profs who take the attitude "if you want your name on crap, be my guest" when it comes to reviewing papers. They might check for glaring errors, but rarely does anyone work through the results for correctness.
- bachmeier 5y ago
  I agree but that's a sign we need more peer review. IMO it's unethical to engage in fake peer review like that. I recently encountered the opposite - a reviewer that wrote a fake negative review of my paper without reading it. Killed the paper at a good journal by making claims that are easily verified to be false.
  - nicce 5y ago

fluoridation 5y ago

Surname is correlated with descent (which is distinct from race, as people understand that word), while in all likelihood the races that were written down for the patients corresponded to the color of their skin, as well as possibly other features. That would suggest three things: First, as you found, there was no correlation between deaths and descent. Presumably there weren't statistically anomalously fewer Gomezes than Smiths. This doesn't eliminate the fact that people wrote down fewer dead as Hispanic than as white. So: Second, there actually were fewer dead Hispanics than dead whites, understanding "Hispanic" as "a person who looks Hispanic" and "white" as "a person who looks white". Third, or alternatively there is no statistical difference between dead Hispanics and dead whites, but somehow fewer dead Hispanics ended up at the morgues to be written down as dead.

chaps 5y ago
The field in question was filled out by the families of the deceased. You're making some really wild conclusions here.
- fluoridation 5y ago
  Ah, I didn't know that. I assumed it was filled by the coroner. Never mind, then. That would certainly explain the anomaly away.