11 ms·

The PhD Metagame: Don't try to reform science – not yet

148 points by jxmorris12 2y ago

gunian 2y ago
[dead]
eikenberry 2y ago
Maybe nit-picking but the article started out problematic for me with their placement of "maybe?" after the scientific method in their definitions. Without the scientific method as the basis, science is meaningless. Both the ways they define the "idealized concept" are 100% applicable to religions and other lines of thought. It is the methodology of science that differentiates science from all the other ways of doing those things.
- accurrent 2y ago
  100% I wish more people paid attention to the _methodology_. A lot of people define science as the search for "truth", where it really it is the methodology that makes it the key differentiator from religion. The process of hypothesizing then subjecting a theory to rigorous, rational, evidence-based scrutiny is THE key differentiator.
  - deleted 2y ago
    [deleted]
  - floweronthehill 2y ago
    Also the fact that a theory is refutable. That a theory is not _the_ truth, but the best explanation so far and if something proves it completely or partially wrong then we move on to the new, better theory.
- skybrian 2y ago
  Yes, methodology is important, but it’s unclear that the way scientists really work has much to do with the naive description of the scientific method that’s taught in school.
- somenameforme 2y ago
  It's not the methodology in and of itself. The key factor that makes science science and not just an arbitrary belief is falsifiability. No matter how you come up with an idea, if it's falsifiable then it is worth pondering because if it's wrong then it can be proven wrong. It's only when we enter into the domain of unfalsifiable things that we enter more into systems of belief than systems that can challenged or tested. So for instance most of social science is not scientific, because the concepts are generally entirely unfalsifiable. The "Journal of Personality and Social Psychology" is one of the leading journals in its domain - and also one of the most frequently cited by the media due to its oft catchy headlines. It's also one of the best examples of the replication crisis where only about 20% of papers published in that journal are able to be replicated. Does that mean that 80% of the papers in it are thus fake and false? Nope. Because the entire domain is just completely unfalsifiable, so the complete inability to replicate the overwhelming majority of what that journal claims has done little to change its premier place in social psychology. It's just entertainment with some standard deviations attached. So looping back to religion, the issue isn't the methodology. It's the lack of falsifiability. You simply cannot disprove the concept of e.g. a spirit because it's inherently immeasurable, untestable, and unknowable. Yet the lack of falsifiability does not mean false. For instance the exact same is true of consciousness. If I claim you're a philosophical zombie [1], you can't prove I'm wrong (or right), because the entire notion of consciousness is unfalsifiable. [1] - https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
  - AStonesThrow 2y ago
    > the concept of e.g. a spirit because it's inherently immeasurable, untestable, and unknowable Do you mean by artifical means or sensing machines or something? Because spirits are a metaphysical but very human concept, and I answer that they can be perceived, discerned, described, and known [perhaps not fully or objectively]. Faith and reason are both in operation for science and religion. There is a complex history of falsified doctrines, miracles, apparitions, and communities for mainstream Christianity, as I am sure all others. You can't dissect a Eucharistic host or throw it into a Mass Spectrometer and find Jesus molecules, [unless you're Carlo Acutis] but if one billion people can tell the difference, who are we to judge? It's better known than Prozac, and possibly more effective, what do you think?
Legend2440 2y ago
> BERT was nothing short of a revolution in the field when it happened. You could cleanly draw a line pre-BERT and post-BERT. After it came out, something absurd like 95% of papers used it. It was so good, nobody could ignore it. >Naively, you’d think such a revolutionary paper would be met with open arms. But when it was given the best paper award (at NAACL 2019), the postdocs I talked to universally grumbled about it. Why? It wasn’t interesting, they bemoaned. “It just scaled some stuff up.” I still hear people making this complaint, despite the extraordinary success of scaling over the last few years. It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repulsive.
- deleted 2y ago
  [deleted]
- kelipso 2y ago
  Ha, I remember when the scaling stuff started taking over in computer vision and how it annoyed us grad students. Suddenly companies (obviously, gross places where research and intellectual curiosity goes to die) were able to produce results that were much better than universities and they did it in a way that was inaccessible to us. Of course, not to mention in such a boring way. We didn’t have nearly the compute resources or any way of getting them. Now it’s slightly better I think.
- aleph_minus_one 2y ago
  > It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repulsive. "Winning" in the sense of "perhaps suitable to build practical applications that make a lot of money" - perhaps (even though I'd claim that whether this is true is still an open question). On the other hand, "winning" in the sense of "getting a deeper understanding why the method works and having a model that can be analyzed deeply very well", then I would clearly say that scaling is not the winning method. Scientific research is about truth-seeking, so many researchers are in particular interested in the second interpretation.
  - 2y ago
jal278 2y ago
The idealized (Science 1) / realpolitik (Science 2) dichotomy is both real and at first depressing. I also did a PhD in machine learning, and became quite disillusioned after seeing how the sausage was made, and how different the process is from how I had imagined it. At the same time -- engaging in 'game change' within Science 2 (perhaps not as a PhD, but after you have some security), is I think one of science's highest moral callings. The aim is not necessarily to inch Science 2 towards an impossible Science 1, but to help science to take itself more seriously (it really is a messy social process & there are ways that social process can work better or worse towards the public good -- itself a scientific question) -- and contribute towards science 2 becoming a better (and ideally better-at-self-improving) science 2.
- accurrent 2y ago
  I really think a lot of it falls on the funding agencies. While communication is key, there need to be some real questions answered. If a professor has 100 researchers under him and is churning out 2 papers per day how much is he really an expert? If his cousin is getting a PhD under him without doing the legwork shouldn't he be fired? Should we keep funding the same person excessively? It doesnt help that professors often rely on immigrant labor meaning that they can have a real choke hold over the lives of their staff. I really think there should be upper limits of what a professor can get away with, they need to be answerable to how tax payer money is being burned.
  - nickff 2y ago
    I think you’re pointing out a very important point; it seems like everyone pretends that you can ‘scale up’ scientific research, when it’s more like a service (that doesn’t scale). One problem might be that the people allocating the funding also stand to gain from pretending that they can “supervise” infinite amounts of research with no diminution of quality.
roenxi 2y ago
It is worth reflecting on group dynamics - every group has a small core of members who set the standards for everyone else. They can reform the group. No-one else can. Sometimes an outsider can manage through sheer force of personality. Groups display some sad behaviours where most of the members know that what they are doing but the leadership is committed to something silly so they go along anyway. Sometimes it gets bad enough that they quit the group, but that is all they can do since most never have a chance at reforming it. This turns up formally in corporations (the executive team have the power to reform) but that is actually just a mirror of a natural social dynamic. Consider religion for the more natural form of this. It gives a good indication of how the ratio of insider to outsiders pans out in practice. Catholicism is a very well established one, and how many people have the moral and practical authority to reform their doctrine? Not many.
- derektank 2y ago
  Yes, I think the story of Catholicism actually points to the idea that ignoring or even defying the norms of your community as part of an effort to form an alternative community is often more effective than trying to stick it out long enough to earn the clout to reform it. Luther's 95 theses were distributed in 1517 but the Catholic Church did not seriously reform their use until the Council of Trent in 1563, when their sale was banned. By that point a fifth of Europe was Protestant.
  - gostsamo 2y ago
    There were a few reform movements before that with a bit more mixed results. See the cathari and Jan Hus.
  - ashoeafoot 2y ago
    You made a important point for academic reform their. Regularly allowing for fast, unbeaucrati domation of groups and auto dissolving already formed hierarchical structures would help. The reactor internals cant be allowed to stratify.
- AStonesThrow 2y ago
  Perhaps Science and Religion are undergoing some sort of schism. Looking back several hundred years, the universities that were forming priests were also educating the elite and turning out polymaths. Perhaps there is a blossoming of specialization in these latter days, where kids don't need church or a noble family to put them on track for some really heady science work. In fact university systems are practically religions unto themselves, with perceptible and distinctive campus cultures, Greek fraternity/sorority and what have you clubs to join, there is certainly a churchy blueprint being followed in education. It seems to me that religion and medicine and architecture and other various sciences were tightly intertwined. It's why the reactionary Christians are rejecting Science qua Science and Scientism, because if science is not subject to religious ethics or morals, then science has power that church doesn't. I don't know about Asia and the rest of the world, but seeing as the Catholic Church set up systems such as universities and hospitals, it's altogether unsurprising that religion is woven into the DNA of those disciplines even as they secularize and syncretize.
deleted 2y ago
[deleted]
kelseydh 2y ago
Something rubs me the wrong way about this article. I struggle to articulate it, but it would be how success in science now requires political ability. It's rare to find people who are both good scientists and good politicians.
- kazinator 2y ago
  Is that something wrong with the article, or is it something that the article is right about, which is wrong in academia?
  - kelseydh 2y ago
    Likely the latter. But the article expresses no interest or desire in changing it.
    - kazinator 2y ago
      Basically it covers only the topic of what you should not do if you're a new researcher, without getting into what you should do and when. So it skews towards cynicism.
- BlackFly 2y ago
  It is the stated assumption that competition for limited resources will lead to the most qualified person getting the position. This is true only in the most tautoligical sense. The biggest sin of science 2 is that all the survivors believe that they were the most qualified to practice science 1 as well.
- rosstex 2y ago
  As someone who left their PhD, it rubs me with how accurate it is lol
- stereolambda 2y ago
  True, but I also see the wisdom. As a PhD student, chances are you are overworked, in the clutches of the institution, dependent on the higher ups, and being mentally crushed by it all. (Source: know, have known people in such programs.) Trying to reform as well, while being a nobody, has lots of costs and not much expected value. I would focus on staying sane psychologically and socially, and not feeling I'm wasting my years as much as possible. Though if you can participate in some movement of your colleagues with potential for a positive change, do that. Just don't overburden yourself alone. I kind of think academia needs to be reformed and unwound from the outside. But I am more and more convinced by the news that the 'outside' might be unable to do it in ways that aren't appalingly dumb. So my faint hope would be that the much maligned "gen Z work ethic" will eventually force some change, and the old academia barons just die out.
kazinator 2y ago
> While lone wolves can go off and do Science 1 on their own, if you’re reading this, that’s probably not you. At the moment, yes it is; but when HackerNews buggers off, things will be back to normal.
- bruce511 2y ago
  In my day to day I write code libraries. Some of that code is boring, but some is making something very simple out of something very complicated. Since I do this commercially, not for science, and since I don't agree with software patents (on principle), I don't "describe" the method anywhere. Now it's possible that in that code is some novel thing. Perhaps am insight, perhaps an algorithm or technique. (I don't really think there is, such is my modesty, but its possible.) Obviously from a "science" point of view, my work is meaningless. And perhaps that speaks to the point of academic science versus the "real world". Their goal is to "write stuff down" - not "do stuff" as much as "record stuff". In the context of the above quote, I believe we might be doing scientific things (even novel things) but if we're not actively sharing that knowledge it's not "science". Now of course lots of us -do- document things in blogs etc. But this informal writing on the internet is adding to a haystack, in which there might be gems, but how can they ever be found?
  - AStonesThrow 2y ago
    > Now of course lots of us -do- document things in blogs etc. But this informal writing on the internet is adding to a haystack, in which there might be gems, but how can they ever be found? Give Copilot and the others some time to chew through GitHub, I guess. Perhaps one day, an LLM will write the successor to Chrome.
    - kazinator 2y ago
      I suspect we may have hit peak LLM. Or if not yet, then just about. The AI that is capable of writing Chrome will be something other than an LLM.
      - 2y ago
foofoofoobar123 2y ago
[dead]
kelseydh 2y ago
Max Planck argued that change takes time because good ideas need enough staying power to outlive their detractors: > A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it … An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth. https://www.benwhite.com/misc/good-ideas-need-to-outlive-the-old-guard/ https://www.benwhite.com/misc/good-ideas-need-to-outlive-the...
- godelski 2y ago
  Why wait when we could just not be dicks?
  - blitzar 2y ago
    Why risk it when you can be a dick and assure your success?
    - godelski 2y ago
      If you are using a first order approximation, then yes, this is the correct strategy. But when you consider the elements of time or consider that there are other players in the field (incorporating either will do, both is better), then this strategy becomes far from optimal. In fact, it will lead you down the wrong direction, making your own life harder. The thing is, bullshit compounds. Builds up slowly, but we all know that's how you boil a frog. Humans have always advanced through the formation of coalitions. To optimize your own success you have to simultaneously optimize the success of others.
  - tgv 2y ago
    Because you don’t know which side is the “dicks”. Evidence is rarely conclusive, and progress isn’t made by cowtowing to the researcher with the biggest mouth.
ricksunny 2y ago
I liked the piece. Interesting OP refers as 'Science 2' what many call 'the scientific enterprise': https://osf.io/preprints/metaarxiv/pkahc_v1 https://osf.io/preprints/metaarxiv/pkahc_v1 ^ This is a review I co-authored of a meta-science book that focuses on the scientific enterprise. (The book is oriented more to life sciences than compsci). Although my personal preference of a term is 'praxis of science', I go with one of the more frequently used terms. But I do appreciate the enumeration approach that inherently juxtaposes Science 1 vs. Science 2.
Warwolt 2y ago
Something about the website is melting my phone while trying to read the article. I think it's maybe those animations? Couldn't finish reading because the tab froze
godelski 2y ago
This is a terrible take. Grad students try the hardest to change things because they are the most affected. The problem with "tabling the issue for later" argument is that you just keep doing this and we end up with exactly the system we have. Maybe it isn't a PhD to do, but there's always something. Professors are still overworked. There was a good post on BlueSky recently[0] that quoted from the instructions for reviewing PNAS The purpose of peer review is not to demonstrate proficiency in identifying flaws I think this is an issue many have when doing any form of quality control. Every single work has flaws and every single work needs more. The problem, especially in machine learning, that I see is that we are not focusing on what matters: validating hypotheses. This requires far more than looking at plots and tables. It really requires you to think about the paper you read. But I think there's a fundamental alignment problem. An irony in ML, since surely this is far easier than the AI alignment problem. But the purpose of publishing is to communicate. Are we actually doing that? Is our review process improving communication? Or is it actually just gatekeeping or blocking out voices? It is one thing to reject works because they communicate poorly, don't evidence their hypothesis, or are outright fraud, but why are we blocking anything else? This stupid notion of prestige? That's never going to end well. Not to mention all the wasted time and money... [0] https://bsky.app/profile/docbecca.bsky.social/post/3lkbec2hib22h https://bsky.app/profile/docbecca.bsky.social/post/3lkbec2hi...
- timkam 2y ago
  Why is it a terrible take? You first have to understand a community in order to reform it. You can't just say "I actually have no idea how to do research in this field, have not contributed anything substantial, have no money or soft power, but let me tell y'all how to do better science according to my subjective and very limited understanding."
  - godelski 2y ago
    > You can't just say "I actually have no idea how to do research in this field I am a researcher in the field, I have SOTA models in ML, and I have a good number of citations for the experience level. Sure, I'm no rockstar, but neither am I below average. I'm not sure why you jumped to the conclusion that I'm not part of this field.
hoseja 2y ago
Has "science 2" actually acomplished anything worthwhile.
cherryteastain 2y ago
Horrible article telling bright eyed prospective grad students to fellate the prestige obsessed academic politics machine rather than actually try to advance science. Remember that Einstein told journal editors to piss off [1] when they tried to get his papers peer reviewed. [1] https://theconversation.com/hate-the-peer-review-process-einstein-did-too-27405 https://theconversation.com/hate-the-peer-review-process-ein...
- blitzar 2y ago
  If you dont play their game you are not allowed to play at all. Catch 22
  - HPsquared 2y ago
    Stockholm syndrome. Only if the game is getting grant money out of that system.
    - maleldil 2y ago
      For PhDs, the game is "finishing the degree".
      - HPsquared 2y ago
        That has always been an arcane ritual. Of course there are silly rules.
- jpfr 2y ago
  I happen to be an (associate) editor of an academic journal. What the people who critique the publication process are missing: 90% of submissions are crap - unfit for publication. We need some process to gate-keep. a) The venue of publication is a good signal, whether the time to read a paper is well-spent. b) The PhD students learning the craft need objective feedback. The supervising professior/university often has the incentive to "just submit" -- even if they know that a publication does not meet the quality standard. Before peer-review, somebody also needed to make a decision on what to publish. This typically fell to a single individual. The editor or some well-known member of the community who could recommend a paper for publication. On old journal issues they even mention the "recommender". So the question is not whether peer-review is bad, the question is which alternative gate-keeping process would be better. Otherwise we will drown in crap publications (even more) and the PhD students don't get a honest feedback signal upon which they can improve their craft.
enum 2y ago
What a PhD student hopefully gets from an advisor is more targeted advice than the type of boilerplate generic advice in this article and others like it. An advisor who knows what the student wants to accomplish, and is capable of accomplishing should be able to determine when building a working system is more valuable than pushing yet another paper, when reforming science in a small way is likely succeed, and so on.
nsagent 2y ago
Well, I resigned my postdoc and decided to pursue "Science 1" as he terms it. Let's see how successful I am in a year or two...
- cosmic_quanta 2y ago
  Best of luck (not being sarcastic). Hacker News celebrates the risk-taking of startups for a reason -- great leaps often requires taking a risky jump. What you're doing is similar. Please post about your experience! I'l love to read what happened
  - nsagent 2y ago
    Thanks! It's hard to describe, but ultimately I realized that neither academia nor industry is willing to invest in moonshot type ideas. Moreover, academics have to worry about funding and tenure (in addition to teaching and service). Funding is easiest when you propose research topics that people believe stand a chance of success. For wacky ideas it's difficult to make that case. Anyway, I have enough money to fund myself for at least a couple years, so my goal is to make the best of that time and see if I can upend the dominant paradigm in NLP (humans don't need as much data to learn — I'm going to pursue ideas that allow computers to be as data efficient).
Spacecosmonaut 2y ago
We are on the cusp of being able to plow through vast quantities of literature and data in an instant using multimodal machine learning models. Journal articles are written for people. The landscape is changing. We are headed toward a future where scientist upload data and thoughts en masse into the cloud to be consumed by interpreter models that in turn feed back into the scientific machine. Data quality and attribution (scientist performance rating) is automatically allocated by models.
trueismywork 2y ago
Growing up in a country where every single adult I saw was essentially part of a corrupt regime with no change, it made sense to reform science right from the get go. Even if I did it somewhat stupidly. Had to recalibrate after coming to west where things are somewhat better.
Al-Khwarizmi 2y ago
Naively, you’d think such a revolutionary paper [BERT] would be met with open arms. But when it was given the best paper award (at NAACL 2019), the postdocs I talked to universally grumbled about it. Why? It wasn’t interesting, they bemoaned. “It just scaled some stuff up.” I'm from that community, was there when they presented it, have used and still use BERT a lot, and still if it were my decision I wouldn't have given it the best paper award, even in hindsight. BERT the model has been, of course, enormously influential. I still use it, even after the generative LLM revolution (which also stands on it shoulders). I greatly respect its authors and am truly grateful that they published and open-sourced it. BERT the paper? It's not really well-written (almost everyone who wants to understand Transformers or BERT turn to a blog, because the papers are so bad) and it's not stellar in terms of scientific insights, because indeed, it scales some stuff up and comes up with a lot of magic numbers that are there presumably because other alternatives were tried and those happened to work. Or maybe not even that, because some stuff included in BERT actually turned up being useless (see RoBERTa), so I guess they just winged much of it and it worked. From a scientific paper, I would expect much more explanation: why the architecture is designed like this, why this number of layers/dimensions and not that, etc. which that paper thoroughly lacks. No one will learn to do better science from that paper, it's not a paper that a PhD student would benefit much from reading except as a curiosity (they of course can benefit from downloading, using and getting to know the model. Just not from the paper). Maybe create a "best model award", "best software award" or whatever, but in my view as an academic a best paper award is just not for this.
- deleted 2y ago
  [deleted]
- ImprobableTruth 2y ago
  Do you take issue with the 'purely empirical' approach (just trying out variants and seeing which sticks) or only with its insufficient documentation? I don't know how you'd improve on the former. For a lot of it there simply isn't any sound theoretical foundation, so you just end up with flimsy post-hoc rationalizations. While I agree that it's unfortunate that people often just present magic numbers without explaining where they come from, in my experience providing documentation for how one arrives at these often enough gets punished because it draws more attention to them. That is, reviewers will e.g. complain about preliminary experiments, asking for theoretical analysis or question why only certain variants were tried, whereas magic numbers are just kind of accepted.
charlieyu1 2y ago
This is expected. People doing frontier research on humanity on shambolic wages while their finances are controlled by stakeholders who know fuck all about science.
dopu 2y ago
There is an underlying precarity in the academy that is so deep it almost feels like a natural part of science. This is part of what makes reform so difficult. Early on, rocking the boat feels like career suicide. Later on, if you are lucky enough to become established, you are much less likely to feel like deep reform is necessary: after all, the system benefited you. And even then, the precarity doesn't go away. You still compete for grants like everyone else, and your trainees are attempting to become established. Why should they be the ones to shoulder the risk of, e.g., ignoring the glam journals and exclusively putting out preprints?
- MichaelZuo 2y ago
  This seems like a tautology, by definition most people are not so unique that they can’t be replaced in a large enough organization. This was different in the past because back then academia was vastly smaller, so practically everyone from post—doc on up was a literal genius, or close to it. And thus much harder to replace.
exprofmaddy 2y ago
This article and most of the comments ignore the social power dynamics and status quo institutional structures of academic science (Science 2): university administrators, power-broker faculty researchers, funding agencies (til recently), publishing companies, higher-ed consultants, etc. There are thousands of potential reforms that would bring Science 2 closer to Science 1 and generally make science life better. Those reforms are articulated by competent scientists and higher ed journalists every day. If you want to know why science reform isn't happening, ask which powerful interests are benefiting from the existing structures. Power makes people stupid: powerful people can't imagine a world other than the one that brought them their power. They will say, "That's the way the world is." Let's encourage students to continue to imagine other possible worlds in order to challenge the status quo.
- SQueeeeeL 2y ago
  I've never really understood the sentiment that "articulation of a problem = solving that problem." Articulation seems to me to be Step 0 in solving a problem, there needs to be people on the ground advocating for why this new ideological framework is "better" than the status quo and actively convincing decision makers or acquiring decision making positions. Otherwise any amount of highly articulate complaints are just sophistry.
  - deleted 2y ago
    [deleted]
  - exprofmaddy 2y ago
    I think calling problem articulation "just sophistry" is overly reductionist. People who make the effort to articulate the problems (e.g., some Chronicle of Higher Ed writers) offer thoughtful readers other possibilities for consideration. Then, in the rare case that a powerful decision-maker perceives a tension in the status quo, there exist well articulated potential actions to resolve the tension. This is why think-tanks write white papers. The narrative that "people on the ground" is a necessary condition for reform dissuades thoughtful problem articulation. "People on the ground" is one way to influence decision-makers, but it is not necessary. Watch CSPAN when a septuagenarian Senator references his/her granddaughter's comment as influencing his/her vote.
jebarker 2y ago
I did my PhD 2004-2007. I'm not sure whether it was the subject (pure math), my particular advisor or just being two decades ago but I was never aware of any pressure to "publish or perish". I didn't end up continuing in academia post-PhD, but my advisor certainly wanted me to do so, so would have demanded more publication if that was necessary. I'm thankful that I was able to just focus on the quality of research (type I science) and publish when it made sense. Having said that, I now work in industry research and frequently find myself having to remember and remind others (particularly junior researchers) not to fight the system of industry research too hard if they want to actually get anyone to take notice of their research.
stared 2y ago
During my PhD (2011-2014), I had naive dreams of changing science (http://offtopicarium.wikidot.com/v1:open-science-2-0 http://offtopicarium.wikidot.com/v1:open-science-2-0). Instead, I understood that I cannot change the world, but I can change myself - and moved to data science (https://p.migdal.pl/blog/2015/12/sci-to-data-sci https://p.migdal.pl/blog/2015/12/sci-to-data-sci). > Don’t try to reform science. Not yet. Not in your PhD. So, I don't agree with that. While yes, you won't succeed - without this kind of idealism, academia is doomed.
- varelse 2y ago
  [dead]
curtisszmania 2y ago
[dead]
sevensor 2y ago
Article seems to call for passively accepting the status quo until one is an associate professor, at a minimum. That’s a long time to wait, a lot of dues to pay. You can’t point out the naked emperors unless you wear a crown yourself. If you can’t be a frustrated idealist in grad school, when can you be?
YossarianFrPrez 2y ago
Speaking as someone who left tech to get a Ph.D. in a non CS field... Broadly speaking, I agree with the author's point, that one needs to learn the rules of the game before trying to futz with them, which means one will ultimately be more effective learning the ropes in the first few years of a Ph.D. program. Then, after, one will be in a much better position to change things. One big issue I see is that the skills that academic training engenders is almost orthogonal to management. And, unlike most of human history, we are now in an Information Age, with both private and public knowledge production economies. The private knowledge economy (i.e., tech, broadly writ) utilizes many practices that are barely heard of in Academia. Minor case in point: project management software is not the norm, at least not in my field. For those who are interested in this topic, there's a very interesting set of proposals for how to bring "Science 2" much closer to "Science 1" in Michael Nielsen's and Kanjun Qiu's monograph / book "A Vision of Meta-Science." [1] Fair Warning: it is very, very long. But the first part is quite short and proposes a number of interesting Science 2 reforms that should interested HN readers. Tenure Insurance (proposed by none other than Patrick Collison), funding by grant-rating variance, etc. I'm still finishing the essay, but so far it's the best thing on the state of science I've read to date. [1] https://scienceplusplus.org/metascience/ https://scienceplusplus.org/metascience/
QuadmasterXLII 2y ago
The logic “Bert came out and didn’t follow norms. The establishment awarded it best paper because it was a revolutionary result. This shows how the establishment values following norms over revolutionary results” doesn’t scan to me.
ssivark 2y ago
With the author's emphasis on Science 2, I found quite provocative an earlier post of his: https://maxwellforbes.com/posts/your-paper-is-an-ad/ https://maxwellforbes.com/posts/your-paper-is-an-ad/ Submitted it for discussion at https://news.ycombinator.com/item?id=43403596 https://news.ycombinator.com/item?id=43403596
FloorEgg 2y ago
Startups, when done well, and with the right attitude, are science 1. Especially if you operate off of design thinking and jobs-to-be-done type principles. Startups work best when the product fits perfectly into the market need. This requires meticulous investigation into the scope and shape of the market need as well as what approaches (business model, product, onboarding, value prop, etc) best satisfy it. Observe -> theory -> hypotheses -> test -> repeat

148 points by jxmorris12 2y ago

[dead]

Maybe nit-picking but the article started out problematic for me with their placement of "maybe?" after the scientific method in their definitions. Without the scientific method as the basis, science is meaningless. Both the ways they define the "idealized concept" are 100% applicable to religions and other lines of thought. It is the methodology of science that differentiates science from all the other ways of doing those things.

accurrent 2y ago
100% I wish more people paid attention to the _methodology_. A lot of people define science as the search for "truth", where it really it is the methodology that makes it the key differentiator from religion. The process of hypothesizing then subjecting a theory to rigorous, rational, evidence-based scrutiny is THE key differentiator.
- deleted 2y ago
  [deleted]
- floweronthehill 2y ago
  Also the fact that a theory is refutable. That a theory is not _the_ truth, but the best explanation so far and if something proves it completely or partially wrong then we move on to the new, better theory.
skybrian 2y ago
Yes, methodology is important, but it’s unclear that the way scientists really work has much to do with the naive description of the scientific method that’s taught in school.
somenameforme 2y ago
It's not the methodology in and of itself. The key factor that makes science science and not just an arbitrary belief is falsifiability. No matter how you come up with an idea, if it's falsifiable then it is worth pondering because if it's wrong then it can be proven wrong. It's only when we enter into the domain of unfalsifiable things that we enter more into systems of belief than systems that can challenged or tested. So for instance most of social science is not scientific, because the concepts are generally entirely unfalsifiable. The "Journal of Personality and Social Psychology" is one of the leading journals in its domain - and also one of the most frequently cited by the media due to its oft catchy headlines. It's also one of the best examples of the replication crisis where only about 20% of papers published in that journal are able to be replicated. Does that mean that 80% of the papers in it are thus fake and false? Nope. Because the entire domain is just completely unfalsifiable, so the complete inability to replicate the overwhelming majority of what that journal claims has done little to change its premier place in social psychology. It's just entertainment with some standard deviations attached. So looping back to religion, the issue isn't the methodology. It's the lack of falsifiability. You simply cannot disprove the concept of e.g. a spirit because it's inherently immeasurable, untestable, and unknowable. Yet the lack of falsifiability does not mean false. For instance the exact same is true of consciousness. If I claim you're a philosophical zombie [1], you can't prove I'm wrong (or right), because the entire notion of consciousness is unfalsifiable. [1] - https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
- AStonesThrow 2y ago
  > the concept of e.g. a spirit because it's inherently immeasurable, untestable, and unknowable Do you mean by artifical means or sensing machines or something? Because spirits are a metaphysical but very human concept, and I answer that they can be perceived, discerned, described, and known [perhaps not fully or objectively]. Faith and reason are both in operation for science and religion. There is a complex history of falsified doctrines, miracles, apparitions, and communities for mainstream Christianity, as I am sure all others. You can't dissect a Eucharistic host or throw it into a Mass Spectrometer and find Jesus molecules, [unless you're Carlo Acutis] but if one billion people can tell the difference, who are we to judge? It's better known than Prozac, and possibly more effective, what do you think?

Legend2440 2y ago

> BERT was nothing short of a revolution in the field when it happened. You could cleanly draw a line pre-BERT and post-BERT. After it came out, something absurd like 95% of papers used it. It was so good, nobody could ignore it. >Naively, you’d think such a revolutionary paper would be met with open arms. But when it was given the best paper award (at NAACL 2019), the postdocs I talked to universally grumbled about it. Why? It wasn’t interesting, they bemoaned. “It just scaled some stuff up.” I still hear people making this complaint, despite the extraordinary success of scaling over the last few years. It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repulsive.

deleted 2y ago
[deleted]
kelipso 2y ago
Ha, I remember when the scaling stuff started taking over in computer vision and how it annoyed us grad students. Suddenly companies (obviously, gross places where research and intellectual curiosity goes to die) were able to produce results that were much better than universities and they did it in a way that was inaccessible to us. Of course, not to mention in such a boring way. We didn’t have nearly the compute resources or any way of getting them. Now it’s slightly better I think.
aleph_minus_one 2y ago
> It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repulsive. "Winning" in the sense of "perhaps suitable to build practical applications that make a lot of money" - perhaps (even though I'd claim that whether this is true is still an open question). On the other hand, "winning" in the sense of "getting a deeper understanding why the method works and having a model that can be analyzed deeply very well", then I would clearly say that scaling is not the winning method. Scientific research is about truth-seeking, so many researchers are in particular interested in the second interpretation.
- 2y ago

jal278 2y ago

The idealized (Science 1) / realpolitik (Science 2) dichotomy is both real and at first depressing. I also did a PhD in machine learning, and became quite disillusioned after seeing how the sausage was made, and how different the process is from how I had imagined it. At the same time -- engaging in 'game change' within Science 2 (perhaps not as a PhD, but after you have some security), is I think one of science's highest moral callings. The aim is not necessarily to inch Science 2 towards an impossible Science 1, but to help science to take itself more seriously (it really is a messy social process & there are ways that social process can work better or worse towards the public good -- itself a scientific question) -- and contribute towards science 2 becoming a better (and ideally better-at-self-improving) science 2.

accurrent 2y ago
I really think a lot of it falls on the funding agencies. While communication is key, there need to be some real questions answered. If a professor has 100 researchers under him and is churning out 2 papers per day how much is he really an expert? If his cousin is getting a PhD under him without doing the legwork shouldn't he be fired? Should we keep funding the same person excessively? It doesnt help that professors often rely on immigrant labor meaning that they can have a real choke hold over the lives of their staff. I really think there should be upper limits of what a professor can get away with, they need to be answerable to how tax payer money is being burned.
- nickff 2y ago
  I think you’re pointing out a very important point; it seems like everyone pretends that you can ‘scale up’ scientific research, when it’s more like a service (that doesn’t scale). One problem might be that the people allocating the funding also stand to gain from pretending that they can “supervise” infinite amounts of research with no diminution of quality.

roenxi 2y ago

It is worth reflecting on group dynamics - every group has a small core of members who set the standards for everyone else. They can reform the group. No-one else can. Sometimes an outsider can manage through sheer force of personality. Groups display some sad behaviours where most of the members know that what they are doing but the leadership is committed to something silly so they go along anyway. Sometimes it gets bad enough that they quit the group, but that is all they can do since most never have a chance at reforming it. This turns up formally in corporations (the executive team have the power to reform) but that is actually just a mirror of a natural social dynamic. Consider religion for the more natural form of this. It gives a good indication of how the ratio of insider to outsiders pans out in practice. Catholicism is a very well established one, and how many people have the moral and practical authority to reform their doctrine? Not many.

derektank 2y ago
Yes, I think the story of Catholicism actually points to the idea that ignoring or even defying the norms of your community as part of an effort to form an alternative community is often more effective than trying to stick it out long enough to earn the clout to reform it. Luther's 95 theses were distributed in 1517 but the Catholic Church did not seriously reform their use until the Council of Trent in 1563, when their sale was banned. By that point a fifth of Europe was Protestant.
- gostsamo 2y ago
  There were a few reform movements before that with a bit more mixed results. See the cathari and Jan Hus.
- ashoeafoot 2y ago
  You made a important point for academic reform their. Regularly allowing for fast, unbeaucrati domation of groups and auto dissolving already formed hierarchical structures would help. The reactor internals cant be allowed to stratify.
AStonesThrow 2y ago
Perhaps Science and Religion are undergoing some sort of schism. Looking back several hundred years, the universities that were forming priests were also educating the elite and turning out polymaths. Perhaps there is a blossoming of specialization in these latter days, where kids don't need church or a noble family to put them on track for some really heady science work. In fact university systems are practically religions unto themselves, with perceptible and distinctive campus cultures, Greek fraternity/sorority and what have you clubs to join, there is certainly a churchy blueprint being followed in education. It seems to me that religion and medicine and architecture and other various sciences were tightly intertwined. It's why the reactionary Christians are rejecting Science qua Science and Scientism, because if science is not subject to religious ethics or morals, then science has power that church doesn't. I don't know about Asia and the rest of the world, but seeing as the Catholic Church set up systems such as universities and hospitals, it's altogether unsurprising that religion is woven into the DNA of those disciplines even as they secularize and syncretize.

deleted 2y ago

[deleted]

kelseydh 2y ago

Something rubs me the wrong way about this article. I struggle to articulate it, but it would be how success in science now requires political ability. It's rare to find people who are both good scientists and good politicians.

kazinator 2y ago
Is that something wrong with the article, or is it something that the article is right about, which is wrong in academia?
- kelseydh 2y ago
  Likely the latter. But the article expresses no interest or desire in changing it.
  - kazinator 2y ago
    Basically it covers only the topic of what you should not do if you're a new researcher, without getting into what you should do and when. So it skews towards cynicism.
BlackFly 2y ago
It is the stated assumption that competition for limited resources will lead to the most qualified person getting the position. This is true only in the most tautoligical sense. The biggest sin of science 2 is that all the survivors believe that they were the most qualified to practice science 1 as well.
rosstex 2y ago
As someone who left their PhD, it rubs me with how accurate it is lol
stereolambda 2y ago
True, but I also see the wisdom. As a PhD student, chances are you are overworked, in the clutches of the institution, dependent on the higher ups, and being mentally crushed by it all. (Source: know, have known people in such programs.) Trying to reform as well, while being a nobody, has lots of costs and not much expected value. I would focus on staying sane psychologically and socially, and not feeling I'm wasting my years as much as possible. Though if you can participate in some movement of your colleagues with potential for a positive change, do that. Just don't overburden yourself alone. I kind of think academia needs to be reformed and unwound from the outside. But I am more and more convinced by the news that the 'outside' might be unable to do it in ways that aren't appalingly dumb. So my faint hope would be that the much maligned "gen Z work ethic" will eventually force some change, and the old academia barons just die out.

kazinator 2y ago

> While lone wolves can go off and do Science 1 on their own, if you’re reading this, that’s probably not you. At the moment, yes it is; but when HackerNews buggers off, things will be back to normal.

bruce511 2y ago
In my day to day I write code libraries. Some of that code is boring, but some is making something very simple out of something very complicated. Since I do this commercially, not for science, and since I don't agree with software patents (on principle), I don't "describe" the method anywhere. Now it's possible that in that code is some novel thing. Perhaps am insight, perhaps an algorithm or technique. (I don't really think there is, such is my modesty, but its possible.) Obviously from a "science" point of view, my work is meaningless. And perhaps that speaks to the point of academic science versus the "real world". Their goal is to "write stuff down" - not "do stuff" as much as "record stuff". In the context of the above quote, I believe we might be doing scientific things (even novel things) but if we're not actively sharing that knowledge it's not "science". Now of course lots of us -do- document things in blogs etc. But this informal writing on the internet is adding to a haystack, in which there might be gems, but how can they ever be found?
- AStonesThrow 2y ago
  > Now of course lots of us -do- document things in blogs etc. But this informal writing on the internet is adding to a haystack, in which there might be gems, but how can they ever be found? Give Copilot and the others some time to chew through GitHub, I guess. Perhaps one day, an LLM will write the successor to Chrome.
  - kazinator 2y ago
    I suspect we may have hit peak LLM. Or if not yet, then just about. The AI that is capable of writing Chrome will be something other than an LLM.
    - 2y ago

foofoofoobar123 2y ago

[dead]

kelseydh 2y ago

Max Planck argued that change takes time because good ideas need enough staying power to outlive their detractors: > A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it … An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth. https://www.benwhite.com/misc/good-ideas-need-to-outlive-the-old-guard/ https://www.benwhite.com/misc/good-ideas-need-to-outlive-the...

godelski 2y ago
Why wait when we could just not be dicks?
- blitzar 2y ago
  Why risk it when you can be a dick and assure your success?
  - godelski 2y ago
    If you are using a first order approximation, then yes, this is the correct strategy. But when you consider the elements of time or consider that there are other players in the field (incorporating either will do, both is better), then this strategy becomes far from optimal. In fact, it will lead you down the wrong direction, making your own life harder. The thing is, bullshit compounds. Builds up slowly, but we all know that's how you boil a frog. Humans have always advanced through the formation of coalitions. To optimize your own success you have to simultaneously optimize the success of others.
- tgv 2y ago
  Because you don’t know which side is the “dicks”. Evidence is rarely conclusive, and progress isn’t made by cowtowing to the researcher with the biggest mouth.

ricksunny 2y ago

I liked the piece. Interesting OP refers as 'Science 2' what many call 'the scientific enterprise': https://osf.io/preprints/metaarxiv/pkahc_v1 https://osf.io/preprints/metaarxiv/pkahc_v1 ^ This is a review I co-authored of a meta-science book that focuses on the scientific enterprise. (The book is oriented more to life sciences than compsci). Although my personal preference of a term is 'praxis of science', I go with one of the more frequently used terms. But I do appreciate the enumeration approach that inherently juxtaposes Science 1 vs. Science 2.

Warwolt 2y ago

Something about the website is melting my phone while trying to read the article. I think it's maybe those animations? Couldn't finish reading because the tab froze

godelski 2y ago

This is a terrible take. Grad students try the hardest to change things because they are the most affected. The problem with "tabling the issue for later" argument is that you just keep doing this and we end up with exactly the system we have. Maybe it isn't a PhD to do, but there's always something. Professors are still overworked. There was a good post on BlueSky recently[0] that quoted from the instructions for reviewing PNAS The purpose of peer review is not to demonstrate proficiency in identifying flaws I think this is an issue many have when doing any form of quality control. Every single work has flaws and every single work needs more. The problem, especially in machine learning, that I see is that we are not focusing on what matters: validating hypotheses. This requires far more than looking at plots and tables. It really requires you to think about the paper you read. But I think there's a fundamental alignment problem. An irony in ML, since surely this is far easier than the AI alignment problem. But the purpose of publishing is to communicate. Are we actually doing that? Is our review process improving communication? Or is it actually just gatekeeping or blocking out voices? It is one thing to reject works because they communicate poorly, don't evidence their hypothesis, or are outright fraud, but why are we blocking anything else? This stupid notion of prestige? That's never going to end well. Not to mention all the wasted time and money... [0] https://bsky.app/profile/docbecca.bsky.social/post/3lkbec2hib22h https://bsky.app/profile/docbecca.bsky.social/post/3lkbec2hi...

timkam 2y ago
Why is it a terrible take? You first have to understand a community in order to reform it. You can't just say "I actually have no idea how to do research in this field, have not contributed anything substantial, have no money or soft power, but let me tell y'all how to do better science according to my subjective and very limited understanding."
- godelski 2y ago
  > You can't just say "I actually have no idea how to do research in this field I am a researcher in the field, I have SOTA models in ML, and I have a good number of citations for the experience level. Sure, I'm no rockstar, but neither am I below average. I'm not sure why you jumped to the conclusion that I'm not part of this field.

hoseja 2y ago

Has "science 2" actually acomplished anything worthwhile.

cherryteastain 2y ago

Horrible article telling bright eyed prospective grad students to fellate the prestige obsessed academic politics machine rather than actually try to advance science. Remember that Einstein told journal editors to piss off [1] when they tried to get his papers peer reviewed. [1] https://theconversation.com/hate-the-peer-review-process-einstein-did-too-27405 https://theconversation.com/hate-the-peer-review-process-ein...

blitzar 2y ago
If you dont play their game you are not allowed to play at all. Catch 22
- HPsquared 2y ago
  Stockholm syndrome. Only if the game is getting grant money out of that system.
  - maleldil 2y ago
    For PhDs, the game is "finishing the degree".
    - HPsquared 2y ago
      That has always been an arcane ritual. Of course there are silly rules.
jpfr 2y ago
I happen to be an (associate) editor of an academic journal. What the people who critique the publication process are missing: 90% of submissions are crap - unfit for publication. We need some process to gate-keep. a) The venue of publication is a good signal, whether the time to read a paper is well-spent. b) The PhD students learning the craft need objective feedback. The supervising professior/university often has the incentive to "just submit" -- even if they know that a publication does not meet the quality standard. Before peer-review, somebody also needed to make a decision on what to publish. This typically fell to a single individual. The editor or some well-known member of the community who could recommend a paper for publication. On old journal issues they even mention the "recommender". So the question is not whether peer-review is bad, the question is which alternative gate-keeping process would be better. Otherwise we will drown in crap publications (even more) and the PhD students don't get a honest feedback signal upon which they can improve their craft.

enum 2y ago

What a PhD student hopefully gets from an advisor is more targeted advice than the type of boilerplate generic advice in this article and others like it. An advisor who knows what the student wants to accomplish, and is capable of accomplishing should be able to determine when building a working system is more valuable than pushing yet another paper, when reforming science in a small way is likely succeed, and so on.

nsagent 2y ago

Well, I resigned my postdoc and decided to pursue "Science 1" as he terms it. Let's see how successful I am in a year or two...

cosmic_quanta 2y ago
Best of luck (not being sarcastic). Hacker News celebrates the risk-taking of startups for a reason -- great leaps often requires taking a risky jump. What you're doing is similar. Please post about your experience! I'l love to read what happened
- nsagent 2y ago
  Thanks! It's hard to describe, but ultimately I realized that neither academia nor industry is willing to invest in moonshot type ideas. Moreover, academics have to worry about funding and tenure (in addition to teaching and service). Funding is easiest when you propose research topics that people believe stand a chance of success. For wacky ideas it's difficult to make that case. Anyway, I have enough money to fund myself for at least a couple years, so my goal is to make the best of that time and see if I can upend the dominant paradigm in NLP (humans don't need as much data to learn — I'm going to pursue ideas that allow computers to be as data efficient).

Spacecosmonaut 2y ago

We are on the cusp of being able to plow through vast quantities of literature and data in an instant using multimodal machine learning models. Journal articles are written for people. The landscape is changing. We are headed toward a future where scientist upload data and thoughts en masse into the cloud to be consumed by interpreter models that in turn feed back into the scientific machine. Data quality and attribution (scientist performance rating) is automatically allocated by models.

trueismywork 2y ago

Growing up in a country where every single adult I saw was essentially part of a corrupt regime with no change, it made sense to reform science right from the get go. Even if I did it somewhat stupidly. Had to recalibrate after coming to west where things are somewhat better.

Al-Khwarizmi 2y ago

Naively, you’d think such a revolutionary paper [BERT] would be met with open arms. But when it was given the best paper award (at NAACL 2019), the postdocs I talked to universally grumbled about it. Why? It wasn’t interesting, they bemoaned. “It just scaled some stuff up.” I'm from that community, was there when they presented it, have used and still use BERT a lot, and still if it were my decision I wouldn't have given it the best paper award, even in hindsight. BERT the model has been, of course, enormously influential. I still use it, even after the generative LLM revolution (which also stands on it shoulders). I greatly respect its authors and am truly grateful that they published and open-sourced it. BERT the paper? It's not really well-written (almost everyone who wants to understand Transformers or BERT turn to a blog, because the papers are so bad) and it's not stellar in terms of scientific insights, because indeed, it scales some stuff up and comes up with a lot of magic numbers that are there presumably because other alternatives were tried and those happened to work. Or maybe not even that, because some stuff included in BERT actually turned up being useless (see RoBERTa), so I guess they just winged much of it and it worked. From a scientific paper, I would expect much more explanation: why the architecture is designed like this, why this number of layers/dimensions and not that, etc. which that paper thoroughly lacks. No one will learn to do better science from that paper, it's not a paper that a PhD student would benefit much from reading except as a curiosity (they of course can benefit from downloading, using and getting to know the model. Just not from the paper). Maybe create a "best model award", "best software award" or whatever, but in my view as an academic a best paper award is just not for this.

deleted 2y ago
[deleted]
ImprobableTruth 2y ago
Do you take issue with the 'purely empirical' approach (just trying out variants and seeing which sticks) or only with its insufficient documentation? I don't know how you'd improve on the former. For a lot of it there simply isn't any sound theoretical foundation, so you just end up with flimsy post-hoc rationalizations. While I agree that it's unfortunate that people often just present magic numbers without explaining where they come from, in my experience providing documentation for how one arrives at these often enough gets punished because it draws more attention to them. That is, reviewers will e.g. complain about preliminary experiments, asking for theoretical analysis or question why only certain variants were tried, whereas magic numbers are just kind of accepted.

charlieyu1 2y ago

This is expected. People doing frontier research on humanity on shambolic wages while their finances are controlled by stakeholders who know fuck all about science.

dopu 2y ago

There is an underlying precarity in the academy that is so deep it almost feels like a natural part of science. This is part of what makes reform so difficult. Early on, rocking the boat feels like career suicide. Later on, if you are lucky enough to become established, you are much less likely to feel like deep reform is necessary: after all, the system benefited you. And even then, the precarity doesn't go away. You still compete for grants like everyone else, and your trainees are attempting to become established. Why should they be the ones to shoulder the risk of, e.g., ignoring the glam journals and exclusively putting out preprints?

MichaelZuo 2y ago
This seems like a tautology, by definition most people are not so unique that they can’t be replaced in a large enough organization. This was different in the past because back then academia was vastly smaller, so practically everyone from post—doc on up was a literal genius, or close to it. And thus much harder to replace.

exprofmaddy 2y ago

This article and most of the comments ignore the social power dynamics and status quo institutional structures of academic science (Science 2): university administrators, power-broker faculty researchers, funding agencies (til recently), publishing companies, higher-ed consultants, etc. There are thousands of potential reforms that would bring Science 2 closer to Science 1 and generally make science life better. Those reforms are articulated by competent scientists and higher ed journalists every day. If you want to know why science reform isn't happening, ask which powerful interests are benefiting from the existing structures. Power makes people stupid: powerful people can't imagine a world other than the one that brought them their power. They will say, "That's the way the world is." Let's encourage students to continue to imagine other possible worlds in order to challenge the status quo.

SQueeeeeL 2y ago
I've never really understood the sentiment that "articulation of a problem = solving that problem." Articulation seems to me to be Step 0 in solving a problem, there needs to be people on the ground advocating for why this new ideological framework is "better" than the status quo and actively convincing decision makers or acquiring decision making positions. Otherwise any amount of highly articulate complaints are just sophistry.
- deleted 2y ago
  [deleted]
- exprofmaddy 2y ago
  I think calling problem articulation "just sophistry" is overly reductionist. People who make the effort to articulate the problems (e.g., some Chronicle of Higher Ed writers) offer thoughtful readers other possibilities for consideration. Then, in the rare case that a powerful decision-maker perceives a tension in the status quo, there exist well articulated potential actions to resolve the tension. This is why think-tanks write white papers. The narrative that "people on the ground" is a necessary condition for reform dissuades thoughtful problem articulation. "People on the ground" is one way to influence decision-makers, but it is not necessary. Watch CSPAN when a septuagenarian Senator references his/her granddaughter's comment as influencing his/her vote.

jebarker 2y ago

I did my PhD 2004-2007. I'm not sure whether it was the subject (pure math), my particular advisor or just being two decades ago but I was never aware of any pressure to "publish or perish". I didn't end up continuing in academia post-PhD, but my advisor certainly wanted me to do so, so would have demanded more publication if that was necessary. I'm thankful that I was able to just focus on the quality of research (type I science) and publish when it made sense. Having said that, I now work in industry research and frequently find myself having to remember and remind others (particularly junior researchers) not to fight the system of industry research too hard if they want to actually get anyone to take notice of their research.

stared 2y ago

During my PhD (2011-2014), I had naive dreams of changing science (http://offtopicarium.wikidot.com/v1:open-science-2-0 http://offtopicarium.wikidot.com/v1:open-science-2-0). Instead, I understood that I cannot change the world, but I can change myself - and moved to data science (https://p.migdal.pl/blog/2015/12/sci-to-data-sci https://p.migdal.pl/blog/2015/12/sci-to-data-sci). > Don’t try to reform science. Not yet. Not in your PhD. So, I don't agree with that. While yes, you won't succeed - without this kind of idealism, academia is doomed.

varelse 2y ago
[dead]

curtisszmania 2y ago

[dead]

sevensor 2y ago

Article seems to call for passively accepting the status quo until one is an associate professor, at a minimum. That’s a long time to wait, a lot of dues to pay. You can’t point out the naked emperors unless you wear a crown yourself. If you can’t be a frustrated idealist in grad school, when can you be?

YossarianFrPrez 2y ago

Speaking as someone who left tech to get a Ph.D. in a non CS field... Broadly speaking, I agree with the author's point, that one needs to learn the rules of the game before trying to futz with them, which means one will ultimately be more effective learning the ropes in the first few years of a Ph.D. program. Then, after, one will be in a much better position to change things. One big issue I see is that the skills that academic training engenders is almost orthogonal to management. And, unlike most of human history, we are now in an Information Age, with both private and public knowledge production economies. The private knowledge economy (i.e., tech, broadly writ) utilizes many practices that are barely heard of in Academia. Minor case in point: project management software is not the norm, at least not in my field. For those who are interested in this topic, there's a very interesting set of proposals for how to bring "Science 2" much closer to "Science 1" in Michael Nielsen's and Kanjun Qiu's monograph / book "A Vision of Meta-Science." [1] Fair Warning: it is very, very long. But the first part is quite short and proposes a number of interesting Science 2 reforms that should interested HN readers. Tenure Insurance (proposed by none other than Patrick Collison), funding by grant-rating variance, etc. I'm still finishing the essay, but so far it's the best thing on the state of science I've read to date. [1] https://scienceplusplus.org/metascience/ https://scienceplusplus.org/metascience/

QuadmasterXLII 2y ago

The logic “Bert came out and didn’t follow norms. The establishment awarded it best paper because it was a revolutionary result. This shows how the establishment values following norms over revolutionary results” doesn’t scan to me.

ssivark 2y ago

With the author's emphasis on Science 2, I found quite provocative an earlier post of his: https://maxwellforbes.com/posts/your-paper-is-an-ad/ https://maxwellforbes.com/posts/your-paper-is-an-ad/ Submitted it for discussion at https://news.ycombinator.com/item?id=43403596 https://news.ycombinator.com/item?id=43403596

FloorEgg 2y ago

Startups, when done well, and with the right attitude, are science 1. Especially if you operate off of design thinking and jobs-to-be-done type principles. Startups work best when the product fits perfectly into the market need. This requires meticulous investigation into the scope and shape of the market need as well as what approaches (business model, product, onboarding, value prop, etc) best satisfy it. Observe -> theory -> hypotheses -> test -> repeat