3 ms·

>For example, if a collaborator sends data for analysis where the sample size was too small for the effect, or worse, asked to find statistical differences betw

>For example, if a collaborator sends data for analysis where the sample size was too small for the effect, or worse, asked to find statistical differences between 1-2 samples per group, you can't fix that with any amount of algorithmic mumbo-jumbo. Au contraire: a few years back my colleagues came to me with an shRNA screen with no replication. I said you can't do this, without an estimate of variability this is meaningless (NOT the first, or last, time i gave this lecture). The reply was, analyze it anyway. I made up some bullshit. The reviewers did nothing. We published in Nature Genetics.

cgiles 7y ago
Ha. I have a lot of those papers too, but none that are BS in such a high tier, congrats! (or "congrats") At this point I am cynical enough to believe people include analysts on their papers as an insurance policy to have the "licensed" person to give the seal of approval, and take the blame if necessary, for their shitty data. Which they will never listen to concerns about before publication. Just today I am working on a qPCR array (yes people still use those) with one "housekeeping" gene replicated 4 times for ddCt normalization. 80 miRs go up, none go down, out of 400. No one sees any problem with this, the data is the data. Them: Oh, and can you, cgiles, also tell us about the genes regulated by these miRs? Me: Not really in any principled way, especially when the underlying data is incredibly fishy, and even if I did, we would be saying all the target genes go down. Don't you think people will find that odd? Do you think perhaps there could be a problem with your "housekeeping" gene? Them: Who cares? Just run the algorithm. Thank you for this tale. I'm glad I'm not the only one in this situation.
- astazangasta 7y ago
  This sort of stuff, the general lack of interest in method in the drive to publish, has really killed a lot of my faith in the idea of a scientific method. It can't work, we can't produce good, useful knowledge if even the best scientists at top flight institutions just regard it as a shell game where the facsimile of a result is as good as a real result, as long as you can sneak it past publishers, who are playing the same game with the public.
User23 7y ago
This makes me sad. Also it makes me not believe anything "scientists say." So is it a good thing or a bad thing that nobody cares what reasonably educated non-experts like myself think?
- cgiles 7y ago
  It is just like anything else, for example politics. Many politicians have strong convictions and care about the electorate and outcomes (I have several politician friends). But when push comes to shove, keeping their job and food on the table for their families comes first. So ethical compromises are made. Similarly, for tech people and privacy. I would wager there are dozens if not hundreds here on HN who have raised concerns with management about the privacy implications of something they're being requested to build. Management says, build it anyway. Programmer reluctantly complies because they like their job. WRT science it is obvious that it works over the long term. But in the short term it can be quite nasty. So I would say if you are a member of the public, you can safely believe something scientists have said for a long time without controversy, like the helical structure of DNA, but new or controversial findings should be taken with a large grain of salt.

cgiles 7y ago

Ha. I have a lot of those papers too, but none that are BS in such a high tier, congrats! (or "congrats") At this point I am cynical enough to believe people include analysts on their papers as an insurance policy to have the "licensed" person to give the seal of approval, and take the blame if necessary, for their shitty data. Which they will never listen to concerns about before publication. Just today I am working on a qPCR array (yes people still use those) with one "housekeeping" gene replicated 4 times for ddCt normalization. 80 miRs go up, none go down, out of 400. No one sees any problem with this, the data is the data. Them: Oh, and can you, cgiles, also tell us about the genes regulated by these miRs? Me: Not really in any principled way, especially when the underlying data is incredibly fishy, and even if I did, we would be saying all the target genes go down. Don't you think people will find that odd? Do you think perhaps there could be a problem with your "housekeeping" gene? Them: Who cares? Just run the algorithm. Thank you for this tale. I'm glad I'm not the only one in this situation.

astazangasta 7y ago
This sort of stuff, the general lack of interest in method in the drive to publish, has really killed a lot of my faith in the idea of a scientific method. It can't work, we can't produce good, useful knowledge if even the best scientists at top flight institutions just regard it as a shell game where the facsimile of a result is as good as a real result, as long as you can sneak it past publishers, who are playing the same game with the public.

User23 7y ago

This makes me sad. Also it makes me not believe anything "scientists say." So is it a good thing or a bad thing that nobody cares what reasonably educated non-experts like myself think?

cgiles 7y ago
It is just like anything else, for example politics. Many politicians have strong convictions and care about the electorate and outcomes (I have several politician friends). But when push comes to shove, keeping their job and food on the table for their families comes first. So ethical compromises are made. Similarly, for tech people and privacy. I would wager there are dozens if not hundreds here on HN who have raised concerns with management about the privacy implications of something they're being requested to build. Management says, build it anyway. Programmer reluctantly complies because they like their job. WRT science it is obvious that it works over the long term. But in the short term it can be quite nasty. So I would say if you are a member of the public, you can safely believe something scientists have said for a long time without controversy, like the helical structure of DNA, but new or controversial findings should be taken with a large grain of salt.