10 ms·

Having spent over 10 years in a university and been a professor, the problem isn’t attention seeking behavior but a lack of accountability. For example, you ca

Having spent over 10 years in a university and been a professor, the problem isn’t attention seeking behavior but a lack of accountability. For example, you can literally make up any data you want in a grant proposal and so long as it sounds right no one can or will double check it. The foundation of academia is rotting, but maybe it’s always been like this

anonporridge 4y ago
> but maybe it’s always been like this The older I get, the more I believe this is the truth. For most institutions we've been taught to hold in high regard.
- marcus_holmes 4y ago
  This. I'm actually optimistic (I know, Unpopular Opinion) about the future for a bunch of reasons, but one of them is the increased transparency that we now have on our institutions. It used to be that the media controlled how much of what went on that we saw, and the media was part of an establishment that valued the status quo and "stability" so didn't report on some things. Now, the blinders are off and we're seeing what was always there. For Science, this means that the institutions that were set up by a bunch of rich men who could afford to spend their time satisfying their curiosity, are now visibly falling apart. Because we can see this, we can change it to something better. The pressure to kill off the journal system is growing. The various crises that the article mentions, etc. This is all good, in the long term.
  - anonporridge 4y ago
    I strongly agree. But I'm also increasingly worried that we're not very far from losing this. It already seems like social media sites like Twitter, Facebook, and Reddit are increasingly controlled by US propaganda and corporate interests. It's hard to know how far gone what we think is true freedom of speech and democratization of voice already is.
- derbOac 4y ago
  I don't know. I've been around academics for awhile now and really tried to get a good answer to this and I still don't know. If anything I don't think it's always been like this. There's a variety of indicators suggesting that something changed in the mid90s to about 2000 with academics (grant success rates, publications, etc). When I talk to much more senior colleagues, I also get the sense something changed even if they describe it in different terms. Was everything better before? No, I think many things are better now, but it feels like the fundamentals of the system are worse. To me it feels like being in a house that's been remodeled with a new roof and HVAC system, but where the ground is sinking and foundation is in the process of collapsing.
wrp 4y ago
I have read a lot about the history of science in Britain. Even before the creation of the Royal Society, people doing science generally all knew each other and communicated extensively, so I think there was a high degree of accountability. Even in the 19th century, scientific sub-communities were small. I suspect that the problem of accountability arose in the 20th century with the expansion of institutional science.
godelski 4y ago
> lack of accountability. I think this also applies at all levels. I'd argue that it is the big reason people feel very frustrated with reviewing, and especially in hyped areas (e.g. ML). There's plenty of incentives to reject papers (low acceptance rates mean "higher quality" publications, advantage yourself by rejecting or being overly critical of your competitors, you get no clout for reviewing and no one will get upset if you take all of 5 minutes to review), but very few incentives (I can't even name one) to accept papers. It is fairly easy to dismiss papers as not novel because we all build off the shoulders of giants and things are substantially more obvious post hoc. Metareviewers and Area Chairs will 99/100 times stand with reviewers even if they are in the wrong and can be proven so (I had a reviewer give me a strong reject claiming I should compare to another work, which was actually our main comparitor and we compared to in 5+ tables and 5+ graphs). I can't see these issues being resolved until we all agree that there needs to be some incentive to write high quality reviews. The worst of this is that the people it hurts the most is the grad students and junior researchers (prevents graduating and career advancement). I'm not saying we have to accept papers, but I am saying we need to ensure that we are providing high quality reviews. Rejections suck, but rejections that don't provide valuable feedback are worse. If the publication system is a noisy signal then we need to fix it AND recognize it as such. There's been plenty of studies to show that this process is highly noisy but we're all acting like publications are all that matters. This is all before we even talk about advantages linked to connections even in double blind reviews, collusion rings, or citation hacking. I feel we can't even get the first step right.
- enviclash 4y ago
  We all get papers rejected. Nobel prizes get also papers rejected. And rejection is one of the very few tools we have to stop the crap from flowing in. This is not to say that all rejected papers are crap. Sorry to hear about your bad time with rejections, this is a universal thing.
  - godelski 4y ago
    I don't think this response really is getting to what I'm complaining about. Rejects suck, but often they are deserving. I'm not saying that we need to throw out the peer review system. I even specifically said I don't think we need to accept more papers. But I do think we need a mechanism to ensure that reviews are good and high quality. Especially the reject ones. If your paper is being rejected without feedback that can convey what needs to be fixed to become a good paper, then this wasn't a peer review. I've had plenty of good reviews and plenty of bad reviews. A reject doesn't sting nearly as much when I think "maybe they have a point." But if a reviewer tells me to compare to something I'm already comparing to and is in many graphs and tables, I'm entirely unconvinced that they even read the paper and I think we can all agree that that type of reviewer is not benefiting anyone. They are the system failing. You may disagree with the number of bad reviewers there are (and that's okay) but I highly doubt you would actually defend them. I'm just saying we need a system where we either encourage these people to change the _quality_ of their reviews or stop reviewing (I am _not_ arguing that we need to change their scores).
chiefalchemist 4y ago
While I hate to parse words, by definition, fabrication of data it's not science. It's fraud.
- DiggyJohnson 4y ago
  That's just restating the problem in a single, flashy sentence.
  - chiefalchemist 4y ago
    Not really. The difference is, it's not giving credit where credit is *not* due. We have the same problem with the misuse of the words leader and journalist. Allowing violators to claim a respected title *that they don't deserve* only enables the problem. Put another way: if you pet barked, do you still call it a cat? The point is, a scientist does science. Fraud is not science. If you do fraud you no longer deserve the right to be called a scientist. Words matter.
club_tropical 4y ago
It has not. And "a lack of accountability" is a band-aid on the real problem: bad gatekeeping. People getting into science, not for the search for truth, but in search of respectability, green card, money, or whatever else. Trying to whip them into real scientists through transparency and accountability is like trying to achieve security in your home by flinging the gates and doors wide open but slapping cameras and motion detectors everywhere. Either they win, or you get fatigued.
- AussieWog93 4y ago
  >People getting into science, not for the search for truth I got into science in order to discover truth and develop new things. Left after less than a year, got a job as a SWE then eventually started my own business selling used video games of all things. Have never met a group of people less interested in the truth, or a system that goes to such extreme effort to actively inhibit discovery, than what I experienced in academia. A friend who works for a private non-profit "researching" cures for cancer has reached the same conclusion there too. Getting money from rich old people/the government takes precedence over science and it's just sad.
  - club_tropical 4y ago
    Mirrors my experience. The good ones leave and the vultures remain.
    - Foobar8568 4y ago
      Hi to any charities.
  - fsckboy 4y ago
    > Getting money from rich old people that's sort of shabby treatment of a group of people who step forward as willing to give money. You're singling them out because virtually nobody else is stepping forward. But you want the money (that's called greed, btw) so I guess you'd rather have money disbursed to you... how? the proper functioning of an economy is essentially defined as allocating resources in a way that leads to the best outcomes for the most people. Look around the world and describe a better system than the one the US has had. I agree it's headed in the wrong direction, but it's not at all clear how to fix it.
chaosbolt 4y ago
The biggest newspapers are writing stories with no proof or facts, and no one cares anymore. No one cares, when you can watch 5 movies in a day and feel all sorts of emotions people used to only feel a few times a year, watch porn, play videogames, read about anything you want, you have little time or neurochemistry left to be mad at scientist, journalists or politicians lying to you.
- colejohnson66 4y ago
  Let’s not pretend that yellow journalism is a recent thing. It’s been around since before the automobile. The difference is that we have internet. So it’s easier to fact check, but it’s also easier for lies to spread more widely before being corrected.
  - prox 4y ago
    Not only that, but the power of suggestion is corrupting by itself. “Is colejohnson66 working hard enough?” is sufficient to put a doubt in someone’s mind. I mean you probably do, but you have to defend yourself after such a uttering. Some media use this and it’s disgusting.
Ztynovovk 4y ago
You have any data/research to support your experience?
wisty 4y ago
As far as I can tell, there's way more accountability in science now, just like there was more accountability in a Soviet factory than an American one in the Cold War. It just doesn't always achieve its intended results.

anonporridge 4y ago

> but maybe it’s always been like this The older I get, the more I believe this is the truth. For most institutions we've been taught to hold in high regard.

marcus_holmes 4y ago
This. I'm actually optimistic (I know, Unpopular Opinion) about the future for a bunch of reasons, but one of them is the increased transparency that we now have on our institutions. It used to be that the media controlled how much of what went on that we saw, and the media was part of an establishment that valued the status quo and "stability" so didn't report on some things. Now, the blinders are off and we're seeing what was always there. For Science, this means that the institutions that were set up by a bunch of rich men who could afford to spend their time satisfying their curiosity, are now visibly falling apart. Because we can see this, we can change it to something better. The pressure to kill off the journal system is growing. The various crises that the article mentions, etc. This is all good, in the long term.
- anonporridge 4y ago
  I strongly agree. But I'm also increasingly worried that we're not very far from losing this. It already seems like social media sites like Twitter, Facebook, and Reddit are increasingly controlled by US propaganda and corporate interests. It's hard to know how far gone what we think is true freedom of speech and democratization of voice already is.
derbOac 4y ago
I don't know. I've been around academics for awhile now and really tried to get a good answer to this and I still don't know. If anything I don't think it's always been like this. There's a variety of indicators suggesting that something changed in the mid90s to about 2000 with academics (grant success rates, publications, etc). When I talk to much more senior colleagues, I also get the sense something changed even if they describe it in different terms. Was everything better before? No, I think many things are better now, but it feels like the fundamentals of the system are worse. To me it feels like being in a house that's been remodeled with a new roof and HVAC system, but where the ground is sinking and foundation is in the process of collapsing.

wrp 4y ago

I have read a lot about the history of science in Britain. Even before the creation of the Royal Society, people doing science generally all knew each other and communicated extensively, so I think there was a high degree of accountability. Even in the 19th century, scientific sub-communities were small. I suspect that the problem of accountability arose in the 20th century with the expansion of institutional science.

godelski 4y ago

> lack of accountability. I think this also applies at all levels. I'd argue that it is the big reason people feel very frustrated with reviewing, and especially in hyped areas (e.g. ML). There's plenty of incentives to reject papers (low acceptance rates mean "higher quality" publications, advantage yourself by rejecting or being overly critical of your competitors, you get no clout for reviewing and no one will get upset if you take all of 5 minutes to review), but very few incentives (I can't even name one) to accept papers. It is fairly easy to dismiss papers as not novel because we all build off the shoulders of giants and things are substantially more obvious post hoc. Metareviewers and Area Chairs will 99/100 times stand with reviewers even if they are in the wrong and can be proven so (I had a reviewer give me a strong reject claiming I should compare to another work, which was actually our main comparitor and we compared to in 5+ tables and 5+ graphs). I can't see these issues being resolved until we all agree that there needs to be some incentive to write high quality reviews. The worst of this is that the people it hurts the most is the grad students and junior researchers (prevents graduating and career advancement). I'm not saying we have to accept papers, but I am saying we need to ensure that we are providing high quality reviews. Rejections suck, but rejections that don't provide valuable feedback are worse. If the publication system is a noisy signal then we need to fix it AND recognize it as such. There's been plenty of studies to show that this process is highly noisy but we're all acting like publications are all that matters. This is all before we even talk about advantages linked to connections even in double blind reviews, collusion rings, or citation hacking. I feel we can't even get the first step right.

enviclash 4y ago
We all get papers rejected. Nobel prizes get also papers rejected. And rejection is one of the very few tools we have to stop the crap from flowing in. This is not to say that all rejected papers are crap. Sorry to hear about your bad time with rejections, this is a universal thing.
- godelski 4y ago
  I don't think this response really is getting to what I'm complaining about. Rejects suck, but often they are deserving. I'm not saying that we need to throw out the peer review system. I even specifically said I don't think we need to accept more papers. But I do think we need a mechanism to ensure that reviews are good and high quality. Especially the reject ones. If your paper is being rejected without feedback that can convey what needs to be fixed to become a good paper, then this wasn't a peer review. I've had plenty of good reviews and plenty of bad reviews. A reject doesn't sting nearly as much when I think "maybe they have a point." But if a reviewer tells me to compare to something I'm already comparing to and is in many graphs and tables, I'm entirely unconvinced that they even read the paper and I think we can all agree that that type of reviewer is not benefiting anyone. They are the system failing. You may disagree with the number of bad reviewers there are (and that's okay) but I highly doubt you would actually defend them. I'm just saying we need a system where we either encourage these people to change the _quality_ of their reviews or stop reviewing (I am _not_ arguing that we need to change their scores).

chiefalchemist 4y ago

While I hate to parse words, by definition, fabrication of data it's not science. It's fraud.

DiggyJohnson 4y ago
That's just restating the problem in a single, flashy sentence.
- chiefalchemist 4y ago
  Not really. The difference is, it's not giving credit where credit is *not* due. We have the same problem with the misuse of the words leader and journalist. Allowing violators to claim a respected title *that they don't deserve* only enables the problem. Put another way: if you pet barked, do you still call it a cat? The point is, a scientist does science. Fraud is not science. If you do fraud you no longer deserve the right to be called a scientist. Words matter.

club_tropical 4y ago

It has not. And "a lack of accountability" is a band-aid on the real problem: bad gatekeeping. People getting into science, not for the search for truth, but in search of respectability, green card, money, or whatever else. Trying to whip them into real scientists through transparency and accountability is like trying to achieve security in your home by flinging the gates and doors wide open but slapping cameras and motion detectors everywhere. Either they win, or you get fatigued.

AussieWog93 4y ago
>People getting into science, not for the search for truth I got into science in order to discover truth and develop new things. Left after less than a year, got a job as a SWE then eventually started my own business selling used video games of all things. Have never met a group of people less interested in the truth, or a system that goes to such extreme effort to actively inhibit discovery, than what I experienced in academia. A friend who works for a private non-profit "researching" cures for cancer has reached the same conclusion there too. Getting money from rich old people/the government takes precedence over science and it's just sad.
- club_tropical 4y ago
  Mirrors my experience. The good ones leave and the vultures remain.
  - Foobar8568 4y ago
    Hi to any charities.
- fsckboy 4y ago
  > Getting money from rich old people that's sort of shabby treatment of a group of people who step forward as willing to give money. You're singling them out because virtually nobody else is stepping forward. But you want the money (that's called greed, btw) so I guess you'd rather have money disbursed to you... how? the proper functioning of an economy is essentially defined as allocating resources in a way that leads to the best outcomes for the most people. Look around the world and describe a better system than the one the US has had. I agree it's headed in the wrong direction, but it's not at all clear how to fix it.

chaosbolt 4y ago

The biggest newspapers are writing stories with no proof or facts, and no one cares anymore. No one cares, when you can watch 5 movies in a day and feel all sorts of emotions people used to only feel a few times a year, watch porn, play videogames, read about anything you want, you have little time or neurochemistry left to be mad at scientist, journalists or politicians lying to you.

colejohnson66 4y ago
Let’s not pretend that yellow journalism is a recent thing. It’s been around since before the automobile. The difference is that we have internet. So it’s easier to fact check, but it’s also easier for lies to spread more widely before being corrected.
- prox 4y ago
  Not only that, but the power of suggestion is corrupting by itself. “Is colejohnson66 working hard enough?” is sufficient to put a doubt in someone’s mind. I mean you probably do, but you have to defend yourself after such a uttering. Some media use this and it’s disgusting.

Ztynovovk 4y ago

You have any data/research to support your experience?

wisty 4y ago

As far as I can tell, there's way more accountability in science now, just like there was more accountability in a Soviet factory than an American one in the Cold War. It just doesn't always achieve its intended results.