3 ms·

Thanks for taking the time to reply. I agree that those use cases are definitely valid. And I would also say that research is a technologically underdeveloped

Thanks for taking the time to reply. I agree that those use cases are definitely valid. And I would also say that research is a technologically underdeveloped space. So any innovation here is great and will definitely help people! The only worry I have (from past experience) is that easy tools have a differential impact: That they facilitate bad research more than they facilitate sound research. My perception as a scientist is that we primarily need to address actively harmful science (as in: spurious results, p-hacking). Fostering more science, on the other hand, is definitely a second or even third priority.

sleepingsoul 5y ago
Yep, I agree. I would like to point out though that one of our aims with this is to buffer against that bad research you mention. Along with our extraction of citation statements, we have a deep learning model that classifies each statement as "supporting", "contrasting", or "mentioning" of the claims made in the original paper. Indeed there is a lot of nuance here, and we've tried to address this in a number of ways (including a manual flag process for review where someone believes it is incorrectly labeled), and we're constantly improving in this area. In general we would expect that research that is not sound will receive more contrasting citations rather than supporting citations because their findings won't be as well supported by future research. I should say explicitly that a "contrasting" citation is not a bad thing per se, especially given the nature of scientific discovery. But a study that was published through p-hacking whose findings are not reproducible should be easier to discern through subsequent contrasting citations. In today's world, all we might know is that it was cited a bunch, with the nuance buried underneath. - Ashish

3 ms·

Thanks for taking the time to reply. I agree that those use cases are definitely valid. And I would also say that research is a technologically underdeveloped

by uniqueuid 5y ago

sleepingsoul 5y ago
Yep, I agree. I would like to point out though that one of our aims with this is to buffer against that bad research you mention. Along with our extraction of citation statements, we have a deep learning model that classifies each statement as "supporting", "contrasting", or "mentioning" of the claims made in the original paper. Indeed there is a lot of nuance here, and we've tried to address this in a number of ways (including a manual flag process for review where someone believes it is incorrectly labeled), and we're constantly improving in this area. In general we would expect that research that is not sound will receive more contrasting citations rather than supporting citations because their findings won't be as well supported by future research. I should say explicitly that a "contrasting" citation is not a bad thing per se, especially given the nature of scientific discovery. But a study that was published through p-hacking whose findings are not reproducible should be easier to discern through subsequent contrasting citations. In today's world, all we might know is that it was cited a bunch, with the nuance buried underneath. - Ashish