6 ms·
No funding for uncomfortable results
- asciident 8y agoI see no evidence that the problem is that people were unwilling to fund uncomfortable results, but that they didn't see the point of funding a calculation that as the article notes, could be done on the back of the envelope, and without any proposed solution. Basically, what would the funding add to this statistic?
- mirimir 8y agoWhat's significant is this: > Over 20 journals turned down her paper on the Weld study ... And this: > A decade ago, funding sources refused to fund re-identification experiments unless there was a promise that results would likely show that no risk existed or that all problems could be solved by some promising new theoretical technology under development. And she didn't just do a "back-of-the-envelope calculation". She re-identified an actual public dataset, including the then governor of Massachusetts :)
- new299 8y agoIn order to publish in a scientific journal the work needs to be novel. Based on this article, the work doesn’t seem novel... I’d like to see the rejections, my guess is they mostly said “this is not novel or unexpected”. To make the work novel you might suggest a new approach to anonymizing data for example...
- mcguire 8y agoNot novel or unexpected, or "would disrupt everyone's research and make it impossible to publish data"?
- extra88 8y agoI think it was only trivial in hindsight. Scientific publishing is rife with articles the reveal a problem without offering a solution; to expect one seems outright weird.
- mirimir 8y ago> my guess is they mostly said “this is not novel or unexpected”. Well, given how widely vulnerable de-identification was being implemented, that seems unlikely.
- asciident 8y agoI agree that 20 rejections is significant. But I think it's a bit naive to think that 20 journals (about 60 peers reviewers) were fools who rejected because they were uncomfortable with the results. Are we really to believe these scientific experts were bumbling, "gee, this shows that it's easy to deidentify someone! better reject since I'm feeling uncomfortable now." And yes, showing this with a real dataset is nice, since it validates the estimation. But it still doesn't explain why it's worthy to fund. Funding is very competitive, and generally goes towards prospective research that can lead to innovation in the field. The article suggests that funding should go towards already-finished descriptive dataset analytics confirming back-of-envelope estimations, with no clear plan to find a solution. It's also possible that this article is not expressing the nuance behind the actual work and reasons behind the rejections, so I'm not judging the researcher's work, but rather judging the claims in this article.
- Illniyar 8y agoIsn't there a preliminary step before peer reviewers review an article for publication (otherwise they'll be flooded). My impression from the post was that it stopped at some point in the marketing/editorial pipeline - I.E. it wasn't adequate enough (for business reasons) to send to peer review. Though even so, your comment still stands (since I'm assuming whomever is in an editorial position has some kind of science background, at least at some of those journals).
- dash2 8y ago> But I think it's a bit naive to think that 20 journals (about 60 peers reviewers) were fools who rejected because they were uncomfortable with the results. As a social scientist, I'm not so sure...
- ssdd 8y agoFiltering research grant application based on personal/organizational beliefs is serious thing that can alter understanding of science. Highlighting these problems in application review process is important for public to understand to not to trust research blindly without accessing for any hidden bias.
- sgt101 8y agoI thought this was famous work that stopped a lot of anonymized data being released and was part of the reason that the programme around differential privacy kicked off?
- extra88 8y agoEventually people caught on but not in 1997. I think Dr. Sweeney spent years in the wilderness. I wonder if her work got more attention after the deanonymized Netflix data in 2006 got headlines.
- gumby 8y agoIt became famous because she was clever enough to get the governor's data, which meant her work became interesting to the general press. The specialist press remained uninterested for reasons discussed in the article and elsewhere in this discussion.
- new299 8y agoThe statement in the original article linked is more interesting: “In the few experiments a decade ago where publication would have been possible, academic journals refused to do so for reasons having nothing to do with the scientific quality of the work. Computer-science publications refused to publish re-identification experiments unless the paper also included a technological solution, notwithstanding assertions that publishing these experiments would inspire technological innovation to address the real-world problem. Health-policy publications refused to publish re-identification experiments related to health data from fear that reaction might make data sharing more difficult, despite assertions that because technology was fostering unprecedented levels of data-sharing, it was timely to scientifically re-examine data-sharing practices. Even my Weld example and related demographic analyses, despite making significant contributions to privacy regulations worldwide, were refused publication by more than 20 academic publications at the time.” Personally if feels like the work was an important journalistic effort, but its scientific merit is less clear.
- dash2 8y agoThat's not how I would read it. Instead, the article fell between the gaps between two different disciplines. The computer science guys wanted a tech fix - reasonably enough, that's kind of what computer science is about. The health policy guys' reaction is harder to understand: they didn't want to publish it because they feared the consequences of publicity, rather than because of flaws in the methods. That's almost never a legitimate reason to reject a paper.
- new299 8y agoThat’s what the author said, but it seems suspect to me. I’d like to read the rejections and understand if this is just the authors interpretation. But looking at what was done, it doesn’t seem novel. It’s just combining two datasets, and while journalistically important, doesn’t seem to be sufficiently novel for publication in most journals.
- naasking 8y ago> doesn’t seem to be sufficiently novel for publication in most journals. The question is why you think novelty is relevant. Replication studies aren't novel but are critically important to real science. Novelty is not and should not be mandatory.
- deleted 8y ago[deleted]
- bjourne 8y agoIn the 80:s, when I was a child, I was part of study about something called DAMP - Deficits in Attention Motor control and Perception). The DAMP diagnosis was roughly similar to ADHD, and kind of a precursor to it, invented by psychologists in the university in the city I grew up on. Back then, this Swedish university was very influential in the area of child psychology. Several renowned text books are written by researches on this university and so on. Luckily for me, I was part of the control group and not among the kids they thought suffered from DAMP. Anyway, Gillberg the professor who led the study had promised all participating families total anonymity. But in the mid nineties, other researchers started to doubt the veracity of Gillberg's results. They thought they had found flaws in his methodology and wanted access to the raw material to double-check his results. Gillberg refused, realizing that anonymizing the data sufficiently so as to not leak any identifying details would be impossible. Eventually, the other researchers went to court and got a court order demanding that Gillberg give up his data. At which point he destroyed all the raw data thereby completely removing any scientific underpinnings the DAMP diagnosis had. There is a lot more to be said about this controversy, but one thing is clear and that is that you shouldn't promise anonymity if you can't keep it.
- user982 8y ago> …one thing is clear and that is that you shouldn't promise anonymity if you can't keep it. Although it sounds like he did keep it, at significant cost.
- meowfly 8y agoOne thing I don't quite understand is why aren't the wealthiest citizens lobbying for privacy legislation? I'm sure there are plenty of wealthy people who have a lot to lose having their medical or financial information leaked. This seems like the kind of bipartisan cross-class issue nearly everyone can get behind.
- guitarbill 8y agoBecause they can pay to either insulate themselves (private clinic, fake names, etc), or pay to have such things silenced/changed/denied would be my guess? Especially if their businesses somehow benefit from not having privacy legislations...
- Terretta 8y agoThey don’t realize.