5 ms·

I lost faith in academia when I saw so many fudge data and copy pasta. This wasn’t even secret, it was quietly acknowledged. I once inherited code that “worked

I lost faith in academia when I saw so many fudge data and copy pasta. This wasn’t even secret, it was quietly acknowledged. I once inherited code that “worked” but operated on uninitialized values and out of bounds data unintentionally. I also couldn’t get it to complete a run without crashing. Even when I fixed bugs, I couldn’t reproduce the results. My own mentor told me to look the other way and move on as if this was known and totally usual. It was also exceedingly hard to reproduce results from papers and authors seldom responded. Few would open data and even fewer would detail enough about their methods in the paper. I’m sure others saw exceptional levels of fraud and other unethical behavior. I don’t, for one second, doubt that Claudine could have forged her way to the top. In fact, I think it could be the norm.

rayiner 3y ago
Nobody ever read or utilized President Gay’s “research” anyway, or that of the overwhelming majority of social “scientists.” What’s the point of adhering to some charade about analytical rigor? It’s not like bridges are falling down as a result.
- throwaway167 3y ago
  It's still sadly fashionable in engineering circles to bash social science without reason. Please list objections objectively * . '* Was Kant a social scientist? Perhaps. Odd trivia: his work reversed the notion of subjectivism and objectivism. But what's in a word?
  - goalieca 3y ago
    [flagged]
    - novemp 3y ago
      [flagged]
    - throwaway167 3y ago
      Critical theory? I feel an analysis of power structures - what they are, how they come about - is a pretty useful arrow to the bow of the study of a society (as in, any collection of people, not exclusive to a nation, etc). But I draw an analogy to the 1980s when 'trickle down' policies were called "the economic argument" - indeed an economic argument but hardly closed-book economic proof and there's plenty of opposing economic argument. It was a political agenda painted under an economic slogan to fool many (and even fool many into promoting). Likewise Critical Theory: a useful tool but only one of many and to paint it as the analysis a political agenda (again，fooling many into not only believing but promoting). Modern (can't say Neo, some terms get camped aggressively) takes on critical theory can be a bit removed from Marx, or can be close, but to knee jerk against it completely just coz one doesn't like some of the users feels like denial for the sake of it, and in the end means missing out just like those that stick to one theory to rule them all miss out on the othrs.
      - orangepurple 3y ago
        [flagged]
        dang 3y ago
        You can't do this here. Since we've warned you multiple times already, I've banned the account. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - rayiner 3y ago
    Good reasons: 1) “Social science” isn’t science. 2) “Social science” doesn’t enable you to build bridges, computers, weapons, power plants, or anything useful. The situation at Harvard is strong evidence of the above. Do you think the other folks in President Gay’s department are much more rigorous about their work? But it doesn’t matter. Nobody relies on that research. Gay’s predecessors (Faust, Bacow), may not have plagiarized their papers and books, but nobody read those either. If folks don’t have an axe to grind about President Gay over her comments about Israel, nobody would have even paid attention to this story. But people noticed when, e.g., Boeing engineers were lax on the 737 MAX!
    - NikolaNovak 3y ago
      I may somewhat agree with your point but method of getting there makes me doubt yourself and myself :-) 1. "Social science isn't science" is a claim, but there's no proof or detail or evidence or logic put forward to back thay claim. 2. Social science doesn't build computers or bridges,sure. But that's not definition of science (maybe engineering?). If for example social science could reliably predict social group dynamics and behaviours that would make it a science. Or if it could help a group of construction workers reliably work better together to build that bridge that also counts. 3. All sciences sadly have at times both real whopsie and repeatability crisis. If we want to make a (fairly worn out) claim that social science is lacking, we really need to out forward a considered claim.
    - skissane 3y ago
      > 1) “Social science” isn’t science. Have you ever read anything by Eric Kaufmann? He's a political scientist (like Claudine Gay), but one whose conclusions I think you'd find have a lot in common with your own thinking (unlike Claudine Gay).
      - rayiner 3y ago
        I wouldn't call Eric Kauffman a "scientist," and I don't think Kauffman would call himself one either. His bio page says: > I am a Professor of Politics from Vancouver, British Columbia, I was born in Hong Kong and spent eight early years of my life in Tokyo. In winter, my favourite pastimes are (ice) hockey and skiing. In summer, cooling off in the ocean or by the lake. I am principally interested in cultural politics: ethnicity, national identity, left-wing ideology and religion. Lots of people study things, without properly being called scientists. Historians, archeologists, etc.
- zug_zug 3y ago
  Are you seriously arguing that fabricating data should be allowed for social sciences? Because otherwise it might just look like you are you just going off topic to grind some personal axe.
  - rayiner 3y ago
    I’m arguing it’s pointless to criticize that because the “data” and results aren’t the point anyway.
- tptacek 3y ago
  Gay should be fired. Meanwhile: almost every comment you make on HN is a social science argument of one sort or the other. You're a social science nerd. It's fine if you think social science at Harvard is being done badly; that's a colorable argument. But I don't think you of all people can put yourself above the whole endeavor of social science.
  - rayiner 3y ago
    I think firing Gay would be indulging in a fiction that any of this matters. You’re correct to identify the similarities, but my point would be that none of us are doing science. My arguments in social issues are rooted in culture and tradition. Pointing to data and observation doesn’t turn those arguments into scientific ones. Culture and tradition also relies on experiential knowledge—that doesn’t make it a science. Indeed, my problem with the social sciences is that they intrude on the province of culture, tradition, and religion. It fills the vacuum of authority resulting from the secularization of society. But it’s not any better, it has no greater claim to authority because it’s not science. You’re better off asking your grandma how to raise kids rather than the Harvard faculty.
    - tptacek 3y ago
      Gay isn't the President of Harvard because she's widely viewed as one of the most effective social scientists of her generation; if anything, her hiring puzzled social scientists, because of her scant track record. So calling out her whole academic discipline is a non sequitur. By the letter of your arguments, she's close to the ideal social scientist for the role: one who does very little social science. Culture, tradition, and religion are all objects of study by social scientists. People do social science one way or the other. You can't have a world without it; you can only have good social science or bad social science. Again: that Harvard produces bad social science is hardly an outré argument. Just make that argument.
      - rayiner 3y ago
        You’re begging the question. Just because people who study the same objects as culture, tradition, and religion, call their field a “science” does not mean we have to accept their terminology. I would call their field something more akin to theology, with the use of numbers being a sort of ritualistic exercise. Gay’s presidency is consistent with that—it simply abandons the pretense that those trappings actually matter to the field.
CrazyStat 3y ago
A friend of mine did her PhD in a prestigious lab in basic medical sciences. A postdoc in the lab was quite clearly fabricating data, but the PI refused to believe it. The postdoc got a tenure track position at a top ten department, from where he was quietly let go a couple years later when they figured out what was going on. The real tragedy is that, as the PI of my friend's lab was about to retire, the postdoc took decades of work--cell lines, antibodies, etc.--with him to start his new lab, and appears to have just thrown it all away when he lost his job. My own experience in academia was not nearly so bad, but I was pressured by my advisor to exclude some less-flattering results from a paper we wrote. I wanted to leave them in because I thought they illustrated some interesting trade-offs between our work and other approaches, but I wasn't brave enough to stand up to someone with the power to seriously fuck my life up.
cxr 3y ago
Not entirely the same, but at each step of the semiconductor manufacturing process, the process step is run on a pool of available tools from the fleet. (Think photolithography machines to make the pattern, etch or deposition tools, and SEMs &c for inline measurement.) These things being hardware for controlling physical and chemical processes, they're of course fickle, so even though two tools should be expected to produce the same results for the same step running the same configuration ("recipe") they unsurprisingly don't always do that. So in a given fleet maybe only 3/4 of them will be cleared to run a given recipe, and all the tools in this pool are supposed to be running the same version of the recipe. Discovering that several tools have not been running the same recipe and are producing noticeably diverging results going back weeks or months is an every-other-week sort of discovery and raising the alarm was always met with the same response—cover it up; don't make waves.
chaxor 3y ago
This tends to be more of the norm for the ones at 'good' institutions, which is aligned with the common knowledge / heavily replicated studies that 'top performers (best students, etc) are the most likely to cheat. There can still be good academic works (and the alternative in industry has even worse incentives, such as intentionally hiding the ability to replicate) but unfortunately the structure as of now rewards bad behavior.
sevagh 3y ago
Your comment reminds me of a mini rant I put in a blog post of mine about my irritations in academia. Maybe a tad over-emotional but I stand by the lack of openness as an issue. >Regarding headbang.py and beat tracking, I tried to find some "open" beat tracking datasets used in ISMIR and MIREX, and actually couldn't! I even emailed the task captain. Ludicrous. Hidden algorithms with hidden code using hidden datasets peer reviewed by the same hidden cabal that peer-review each others' beat tracking papers. Good job, you solved beat tracking . Dear Sevag, I am sorry to tell you that I don't have/know those datasets. I was the captain on the downbeat estimation task. For the beat estimation, you can email <redacted>. Maybe he can help you. Unfortunately, even as task captains, we don't have access to the datasets used for the evaluation. I am not sure if <redacted> will be able to give these to you but he might be able to guide you toward other datasets. Good luck, <redacted> Hello <redacted>, I was able to find the SMC12 dataset (hosted by SMC), but I'm having a harder time finding any copies of the MCK 06 and MAZ 09 datasets available for download. Are those available?
skissane 3y ago
> I once inherited code that “worked” but operated on uninitialized values and out of bounds data unintentionally. I also couldn’t get it to complete a run without crashing. Even when I fixed bugs, I couldn’t reproduce the results. My own mentor told me to look the other way and move on as if this was known and totally usual. 20+ years ago I worked as a research assistant, on a software engineering research project (at an Australian university). A defence industry consultant had built his own requirements database, in Microsoft Access, which (apparently) was being used on some military project. There was also an Italian academic who had written some tool for parsing English requirements into logical statements and feeding them into a theorem prover. The tool only ran on Linux, and he wouldn't give us the source code, only a binary. One programmer was working on porting the Microsoft Access database to VB6, whereas my part was building an RPC system using Perl 5 (I used some CPAN module, might have been RPC::Simple) which the VB6 code on Windows could use to call the tool on Linux. Somehow we got it all to work, for a highly unrealistic contrived demo; and then the funding ran out and I left the project. Some months later, it had stopped working, they couldn't work out how to fix it, and they wanted to hire me back to fix it-but I declined. I always had the feeling the project was designed to maximise research funding (international collaboration got one type of grant, collaboration with the military got another) rather than deliver something with real world value, or even serious value as basic research. But it was my first ever programming job, I hadn't even finished my bachelor's degree, so I took what I could get. I'd moved on to something more obviously valuable (working on the university's course catalog database), and I didn't want to go back to that project. I never found out if they got the code to work in the end or not.

rayiner 3y ago

Nobody ever read or utilized President Gay’s “research” anyway, or that of the overwhelming majority of social “scientists.” What’s the point of adhering to some charade about analytical rigor? It’s not like bridges are falling down as a result.

throwaway167 3y ago
It's still sadly fashionable in engineering circles to bash social science without reason. Please list objections objectively * . '* Was Kant a social scientist? Perhaps. Odd trivia: his work reversed the notion of subjectivism and objectivism. But what's in a word?
- goalieca 3y ago
  [flagged]
  - novemp 3y ago
    [flagged]
  - throwaway167 3y ago
    Critical theory? I feel an analysis of power structures - what they are, how they come about - is a pretty useful arrow to the bow of the study of a society (as in, any collection of people, not exclusive to a nation, etc). But I draw an analogy to the 1980s when 'trickle down' policies were called "the economic argument" - indeed an economic argument but hardly closed-book economic proof and there's plenty of opposing economic argument. It was a political agenda painted under an economic slogan to fool many (and even fool many into promoting). Likewise Critical Theory: a useful tool but only one of many and to paint it as the analysis a political agenda (again，fooling many into not only believing but promoting). Modern (can't say Neo, some terms get camped aggressively) takes on critical theory can be a bit removed from Marx, or can be close, but to knee jerk against it completely just coz one doesn't like some of the users feels like denial for the sake of it, and in the end means missing out just like those that stick to one theory to rule them all miss out on the othrs.
    - orangepurple 3y ago
      [flagged]
      - dang 3y ago
        You can't do this here. Since we've warned you multiple times already, I've banned the account. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- rayiner 3y ago
  Good reasons: 1) “Social science” isn’t science. 2) “Social science” doesn’t enable you to build bridges, computers, weapons, power plants, or anything useful. The situation at Harvard is strong evidence of the above. Do you think the other folks in President Gay’s department are much more rigorous about their work? But it doesn’t matter. Nobody relies on that research. Gay’s predecessors (Faust, Bacow), may not have plagiarized their papers and books, but nobody read those either. If folks don’t have an axe to grind about President Gay over her comments about Israel, nobody would have even paid attention to this story. But people noticed when, e.g., Boeing engineers were lax on the 737 MAX!
  - NikolaNovak 3y ago
    I may somewhat agree with your point but method of getting there makes me doubt yourself and myself :-) 1. "Social science isn't science" is a claim, but there's no proof or detail or evidence or logic put forward to back thay claim. 2. Social science doesn't build computers or bridges,sure. But that's not definition of science (maybe engineering?). If for example social science could reliably predict social group dynamics and behaviours that would make it a science. Or if it could help a group of construction workers reliably work better together to build that bridge that also counts. 3. All sciences sadly have at times both real whopsie and repeatability crisis. If we want to make a (fairly worn out) claim that social science is lacking, we really need to out forward a considered claim.
  - skissane 3y ago
    > 1) “Social science” isn’t science. Have you ever read anything by Eric Kaufmann? He's a political scientist (like Claudine Gay), but one whose conclusions I think you'd find have a lot in common with your own thinking (unlike Claudine Gay).
    - rayiner 3y ago
      I wouldn't call Eric Kauffman a "scientist," and I don't think Kauffman would call himself one either. His bio page says: > I am a Professor of Politics from Vancouver, British Columbia, I was born in Hong Kong and spent eight early years of my life in Tokyo. In winter, my favourite pastimes are (ice) hockey and skiing. In summer, cooling off in the ocean or by the lake. I am principally interested in cultural politics: ethnicity, national identity, left-wing ideology and religion. Lots of people study things, without properly being called scientists. Historians, archeologists, etc.
zug_zug 3y ago
Are you seriously arguing that fabricating data should be allowed for social sciences? Because otherwise it might just look like you are you just going off topic to grind some personal axe.
- rayiner 3y ago
  I’m arguing it’s pointless to criticize that because the “data” and results aren’t the point anyway.
tptacek 3y ago
Gay should be fired. Meanwhile: almost every comment you make on HN is a social science argument of one sort or the other. You're a social science nerd. It's fine if you think social science at Harvard is being done badly; that's a colorable argument. But I don't think you of all people can put yourself above the whole endeavor of social science.
- rayiner 3y ago
  I think firing Gay would be indulging in a fiction that any of this matters. You’re correct to identify the similarities, but my point would be that none of us are doing science. My arguments in social issues are rooted in culture and tradition. Pointing to data and observation doesn’t turn those arguments into scientific ones. Culture and tradition also relies on experiential knowledge—that doesn’t make it a science. Indeed, my problem with the social sciences is that they intrude on the province of culture, tradition, and religion. It fills the vacuum of authority resulting from the secularization of society. But it’s not any better, it has no greater claim to authority because it’s not science. You’re better off asking your grandma how to raise kids rather than the Harvard faculty.
  - tptacek 3y ago
    Gay isn't the President of Harvard because she's widely viewed as one of the most effective social scientists of her generation; if anything, her hiring puzzled social scientists, because of her scant track record. So calling out her whole academic discipline is a non sequitur. By the letter of your arguments, she's close to the ideal social scientist for the role: one who does very little social science. Culture, tradition, and religion are all objects of study by social scientists. People do social science one way or the other. You can't have a world without it; you can only have good social science or bad social science. Again: that Harvard produces bad social science is hardly an outré argument. Just make that argument.
    - rayiner 3y ago
      You’re begging the question. Just because people who study the same objects as culture, tradition, and religion, call their field a “science” does not mean we have to accept their terminology. I would call their field something more akin to theology, with the use of numbers being a sort of ritualistic exercise. Gay’s presidency is consistent with that—it simply abandons the pretense that those trappings actually matter to the field.

CrazyStat 3y ago

A friend of mine did her PhD in a prestigious lab in basic medical sciences. A postdoc in the lab was quite clearly fabricating data, but the PI refused to believe it. The postdoc got a tenure track position at a top ten department, from where he was quietly let go a couple years later when they figured out what was going on. The real tragedy is that, as the PI of my friend's lab was about to retire, the postdoc took decades of work--cell lines, antibodies, etc.--with him to start his new lab, and appears to have just thrown it all away when he lost his job. My own experience in academia was not nearly so bad, but I was pressured by my advisor to exclude some less-flattering results from a paper we wrote. I wanted to leave them in because I thought they illustrated some interesting trade-offs between our work and other approaches, but I wasn't brave enough to stand up to someone with the power to seriously fuck my life up.

cxr 3y ago

Not entirely the same, but at each step of the semiconductor manufacturing process, the process step is run on a pool of available tools from the fleet. (Think photolithography machines to make the pattern, etch or deposition tools, and SEMs &c for inline measurement.) These things being hardware for controlling physical and chemical processes, they're of course fickle, so even though two tools should be expected to produce the same results for the same step running the same configuration ("recipe") they unsurprisingly don't always do that. So in a given fleet maybe only 3/4 of them will be cleared to run a given recipe, and all the tools in this pool are supposed to be running the same version of the recipe. Discovering that several tools have not been running the same recipe and are producing noticeably diverging results going back weeks or months is an every-other-week sort of discovery and raising the alarm was always met with the same response—cover it up; don't make waves.

chaxor 3y ago

This tends to be more of the norm for the ones at 'good' institutions, which is aligned with the common knowledge / heavily replicated studies that 'top performers (best students, etc) are the most likely to cheat. There can still be good academic works (and the alternative in industry has even worse incentives, such as intentionally hiding the ability to replicate) but unfortunately the structure as of now rewards bad behavior.

sevagh 3y ago

Your comment reminds me of a mini rant I put in a blog post of mine about my irritations in academia. Maybe a tad over-emotional but I stand by the lack of openness as an issue. >Regarding headbang.py and beat tracking, I tried to find some "open" beat tracking datasets used in ISMIR and MIREX, and actually couldn't! I even emailed the task captain. Ludicrous. Hidden algorithms with hidden code using hidden datasets peer reviewed by the same hidden cabal that peer-review each others' beat tracking papers. Good job, you solved beat tracking . Dear Sevag, I am sorry to tell you that I don't have/know those datasets. I was the captain on the downbeat estimation task. For the beat estimation, you can email <redacted>. Maybe he can help you. Unfortunately, even as task captains, we don't have access to the datasets used for the evaluation. I am not sure if <redacted> will be able to give these to you but he might be able to guide you toward other datasets. Good luck, <redacted> Hello <redacted>, I was able to find the SMC12 dataset (hosted by SMC), but I'm having a harder time finding any copies of the MCK 06 and MAZ 09 datasets available for download. Are those available?

skissane 3y ago

> I once inherited code that “worked” but operated on uninitialized values and out of bounds data unintentionally. I also couldn’t get it to complete a run without crashing. Even when I fixed bugs, I couldn’t reproduce the results. My own mentor told me to look the other way and move on as if this was known and totally usual. 20+ years ago I worked as a research assistant, on a software engineering research project (at an Australian university). A defence industry consultant had built his own requirements database, in Microsoft Access, which (apparently) was being used on some military project. There was also an Italian academic who had written some tool for parsing English requirements into logical statements and feeding them into a theorem prover. The tool only ran on Linux, and he wouldn't give us the source code, only a binary. One programmer was working on porting the Microsoft Access database to VB6, whereas my part was building an RPC system using Perl 5 (I used some CPAN module, might have been RPC::Simple) which the VB6 code on Windows could use to call the tool on Linux. Somehow we got it all to work, for a highly unrealistic contrived demo; and then the funding ran out and I left the project. Some months later, it had stopped working, they couldn't work out how to fix it, and they wanted to hire me back to fix it-but I declined. I always had the feeling the project was designed to maximise research funding (international collaboration got one type of grant, collaboration with the military got another) rather than deliver something with real world value, or even serious value as basic research. But it was my first ever programming job, I hadn't even finished my bachelor's degree, so I took what I could get. I'd moved on to something more obviously valuable (working on the university's course catalog database), and I didn't want to go back to that project. I never found out if they got the code to work in the end or not.