8 ms·

I've been on the other end of such reviewers. Building community and engaging in it takes a whole ton of seemingly non-core work, but community-of-practice res

I've been on the other end of such reviewers. Building community and engaging in it takes a whole ton of seemingly non-core work, but community-of-practice research is totally a thing, and for folks who empirically study innovation, one of the best approaches in practice. Likewise, it's common wisdom by systems researchers to invest in platform early to get compounding results later. Two of my best projects, include two awarded top of the year / decade in my field, came from the former, and colleagues made a few $B companies from the latter. And yet, reviewers assume they know best. Just because it's not your approach... (And yes, I spent time on logos, websites, and whatnot that never make it into papers: it all adds up and you never really know what is ok to drop until after the fact. More so, the paper is the worst form of result and better as just a stepping stone, IMO: a paper is the failure to do anything more meaningful & expansive, like a community of practice or company.)

Topolomancer 5y ago
Author here---I love the term 'non-core work.' It encompasses a lot of the things that irk me in academia (and it's a little bit hypocritical because in the end, academic projects also benefit quite a lot from this non-core work...at least, if your goal is to build something that people actually want to use, as opposed to just pushing out papers.) PS: And upon writing this comment, I had the epiphany that 'pushing out papers' might indeed be the primary objective for some. :-|
- throwawaygh 5y ago
  > academic projects also benefit quite a lot from this non-core work Yes, of course they do. The whole point is that this type of labor is valuable. > if your goal is to build something that people actually want to use, as opposed to just pushing out papers. Neither of those things is the purpose of a phd... If a student's goal is to build something that people actually want to use, they are much better served by taking an industry position in a product group. > pushing out papers From my perspective, the obsession with publishing is just another version of the same problem.
  - Topolomancer 5y ago
    > If a student's goal is to build something that people actually want to use, they are much better served by taking an industry position in a product group. Agreed, I should have phrased that better. Fundamentally, the focus should be on 'generating new knowledge,' right? > From my perspective, the obsession with publishing is just another version of the same problem. Yes. :(
  - lmeyerov 5y ago
    A lot of systems research is socially guided where community-of-practice approaches help. If you haven't, I recommend reading something like Rogers' surveys of papers in 'Diffusion of Innovation' for why, especially before writing publicly that students following such methods would be better off abandoning them for industry jobs. Industry jobs, including R&D, are generally not like the old days of Bell Labs. Startups are kind of like that, but still quite different because VC/P&L/growth/sales still warp things. (And PhD programs are great places to experiment before that jump, as the learning & investigation can take an unknown amount of time.)
- saltcured 5y ago
  I am bemused that the vagueness of "non-core work" means everyone can consider this topic and imagine entirely different stories. As a computer science type peering into biology research with respect to bioinformatics, I have found it distressing how much biology researchers can treat their own experimental data and metadata management as "non-core work" to be avoided. Some biology students and postdocs have described to me an antagonistic relationship to the funder or PI goals where they dismiss their responsibilities for data quality and reuse. For them, it isn't on the critical path to their paper and their career. Or, having data in disarray is, in their minds, giving them an edge to not be "scooped" in the lab. These cynical attitudes are almost subversive, in how the junior worker tries to evade or sabotage aspects of their job responsibility which do not serve them personally. I also see some who seem to engage in magical thinking, as if some other staff or informatician should come by later and clean up their mess of data, without the researcher putting in the effort. I often wonder whether they really misunderstand the problem so severely, or merely use this fig leaf to avoid feelings of shame. I know that lines should be drawn between different roles, but there are inappropriate expectations on both ends of a spectrum. PIs should not treat junior researchers like assistants to get coffee, run personal errands, etc. But neither should junior researchers believe that the day to day grind of the scientific process is beneath them. At the scales that most labs work, there are many essential yet mundane tasks that need to be shouldered by "core" participants. The overhead of trying to delegate these tasks to a different person would either require more time than doing the task oneself, or result in abysmal quality when the hand-off is made with insufficient supporting effort. I think that people exposed to the pop culture of startups and large corporations can absorb a mistaken idea of how easily human organizations scale. With vast numbers of skilled staff, you can create processes which scale beyond the capabilities of a typical small lab or group. These tend to develop lots of specialized roles to support that process. But, I think there is a huge inflection point between the efficient small group and the scalable large organization, where the marginal value of adding another staff member can be very small, or even negative. Many (most?) grant-funded research groups exist only because they can function in that small team mode, with relatively thin budgets and the need to wear many hats. The only safe way to scale work under these grants is what we see already: schools expand to have additional PIs running their own small, independent labs. These eventually compete with each other almost as much as they compete with outside labs. From the host school, they get actual "non-core" services like building facilities, human resources, payroll, contracts and legal support, etc. But, no one group can really scale to have more staff and specialization focused on a single research effort. There is a reason that "big science" projects are few and far between and are mired in much more politics---the granting agencies have to be motivated to write different kinds of funding opportunity to bootstrap such projects and to continue their operation. The budgets and timelines are vastly different.
rscho 5y ago
Good for you if your products do well. But making logos and such is your job as PI if you want it to happen. It's certainly not your students' job. They're here to learn academic skills and advance the field, not to make products that do well. Any material advantage they get from that should be incidental, and not a goal in itself.
- lmeyerov 5y ago
  The student picks the PI, and as part of that, the field of methods they want to learn, which may include scrappy prototyping. If they don't think that kind of R&D is their thing, they should pick another PI and, potentially, field. Ex: My wife's lab had paid staff, but she still had to feed cell cultures at 1am for the research she wanted to do, and later decided those methods were not for her and no longer does wetlab projects. That doesn't invalidate people still doing cell cultures as part of their methods, and I'd still love to see more robots & no-code & budget there to make it easier. Edit: RE:products, Whoosh. Academically derived products at that scale imply a community of practice and development: societal impact and dedicated makers growing it, including more knowledge. That should have the level of respect that a paper that opens up a small subfield gets. From a norms perspective, it's disappointing to see what sounds like a practicing academic to not value that. Separately, again, community of practice work values being embedded, so a student not enjoying it is their personal methods choice, not rules for others.
  - rscho 5y ago
    Well, I guess we have different opinions about what academia should be, then. IMO, material incentives outside core academic activities are cancer and I think university spin-offs are heresy. If even tenured staff now attempts to "develop products" and thinks academia and industry R&D are similar, I think western science is finished.
    - lmeyerov 5y ago
      I don't think academia should be one thing, that's kind of the point of tenure You're promoting both a super traditional view yet also one that seems divorced from the last ~100 years of sociology studies wrt how society progresses wrt inventions and how people fit into that. So, it's great you follow old beliefs handed down by generations of giants from much earlier ages -- there's clearly precedent -- and I encourage you to learn about new ones from the last few generation of scientists who took a more scientific method to learning how recent progress often works. These methods are not for everyone, but it's useful to at least know about them before calling the techniques of your peers 'cancer'. Personally, as technology keeps speeding up, I expect this to become even more important, but at the same time, do value head-in-the-sand research too ;-) Also, to be clear: I am not rejecting the scientific method here. Products, and more importantly, the communities of practice around them, have a lot to do with hypothesis generation, methods, and evaluation. Most industrial labs, and ironically, especially ones that force researchers to embed into existing internal product teams, put a lot of hand cuffs on this process so academic env helps. It does take humility and empathy and time to incorporate them, so not so easy (kind of like startup-lite), but it is a family of tools and one aimed at more intentional progress. And no, 'products' is not synonymous with financial motivation: most OSS projects have no paid piece, for example.
throwawaygh 5y ago
I don't dispute any of this post. Actually, to be blunt, the fact that you think any of this is a retort to "pay people for the type of work you expect them to do" is deeply concerning. > reviewers assume they know best. You are probably more adept at running a grant-funded research group than me. I have no doubt. My concern here is ethical. I do think the average PI is in waaaay too deep to admit how horrible even the "good" PIs end up treating their phd students. So, in that sense, I do generally assume I know best. Not about how to do successful research. But about the true reality of how the people in those salt mines are being treated. On that note: of course all that non-core work is valuable! Incredibly valuable! And PIs should probably hire someone to do that work! At fair market wages! With accurate job titles and job descriptions! I agree with everything you're saying... except about who should do the labor and what they should be paid for that labor. I'm going to reiterate two sentences from my original post. >> Want a devops engineer? Hire a devops engineer ("systems scientist" or so on). Want a graphic designer? Hire a graphic designer. >>> ‘I’m a researcher, not a(n) X,’ with X∈ {software developer, designer, marketing person, artist, …} >> Sure, but here's the rub. What does everything in that set have in common? They make at least 2x+ the average PhD student and have much better benefits. >> If you want a "force multiplier" person in your lab, do your job: get a grant and pay a fair price for that sort of labor. PhD students are NOT ∈ {software developer, designer, marketing person, artist, …}. If they were, you'd be paying quite a bit more and/or getting far less exceptional candidates. Labs that have successful spin-offs are usually the worst about this sort of thing. They're running their research lab like a seedling incubator, but paying each of the one man shops printing their lottery tickets low five figures without retirement benefits.
- lmeyerov 5y ago
  My response is to the Reviewer #2 aspect of, instead of say championing a proposal on merit and, for their allocation issue, advocating for say increased budget for extra staff... rejecting it because of belief in Not My Job for something a good team would often either do scrappily or consider to be part of their job. One of the main costs and bottlenecks in modern academic R&D is the crazy low grant acceptance rates, forcing leaders to 5X grant writing time vs doing time, which makes this reviewer response 5X more nasty than you might otherwise think. If you want to go ethical - * PhD students, most of all, are learning how to do R&D: picking valuable problems, picking how to solve them, and most efficiently in time/money/etc solving then. Doing scrappy Day 1 work is very much a part of that skillset, surprisingly similar to a founding MVP team: Not My Job is teaching the wrong mindset and divorces you from the community + work, which may be what you are trying to learn. * ... However, Learning to Outsource/delegate is indeed part of efficiency training, so let's dig in. The default answer in most prototyping is do a crappy cheapo version, replace with a more costly version when it becomes the need, and reserve your shoestring budget for a few really important things. So likely no to a commercial grade design firm on rerainer, yes to either a $50 logo on Fiverr or more likely, something funny you can make via a free tool, done. Later, when there is more need and budget, say Year 2, easy to revisit. * Some things do merit staff. Should a bio lab partner with a CS team who wants to learn their needs as part of making new CS methods? That's T-shaped collaboration and good, and part of why academic teams can be so scrappy: there's ~always someone somewhere where your non-core work is their baby. Or, maybe they want to inhouse to get more speed/reliability/control. If that person will be like the CS team, ultimately making new CS methods, or a more computationally minded bio researcher helping with some of their own coding for faster EDA, great. But if they will always be Author #2 and has to code on projects they are not doing the science on, that's not a postdoc, that's a staff programmer. We should have more of those.

Topolomancer 5y ago

Author here---I love the term 'non-core work.' It encompasses a lot of the things that irk me in academia (and it's a little bit hypocritical because in the end, academic projects also benefit quite a lot from this non-core work...at least, if your goal is to build something that people actually want to use, as opposed to just pushing out papers.) PS: And upon writing this comment, I had the epiphany that 'pushing out papers' might indeed be the primary objective for some. :-|

throwawaygh 5y ago
> academic projects also benefit quite a lot from this non-core work Yes, of course they do. The whole point is that this type of labor is valuable. > if your goal is to build something that people actually want to use, as opposed to just pushing out papers. Neither of those things is the purpose of a phd... If a student's goal is to build something that people actually want to use, they are much better served by taking an industry position in a product group. > pushing out papers From my perspective, the obsession with publishing is just another version of the same problem.
- Topolomancer 5y ago
  > If a student's goal is to build something that people actually want to use, they are much better served by taking an industry position in a product group. Agreed, I should have phrased that better. Fundamentally, the focus should be on 'generating new knowledge,' right? > From my perspective, the obsession with publishing is just another version of the same problem. Yes. :(
- lmeyerov 5y ago
  A lot of systems research is socially guided where community-of-practice approaches help. If you haven't, I recommend reading something like Rogers' surveys of papers in 'Diffusion of Innovation' for why, especially before writing publicly that students following such methods would be better off abandoning them for industry jobs. Industry jobs, including R&D, are generally not like the old days of Bell Labs. Startups are kind of like that, but still quite different because VC/P&L/growth/sales still warp things. (And PhD programs are great places to experiment before that jump, as the learning & investigation can take an unknown amount of time.)
saltcured 5y ago
I am bemused that the vagueness of "non-core work" means everyone can consider this topic and imagine entirely different stories. As a computer science type peering into biology research with respect to bioinformatics, I have found it distressing how much biology researchers can treat their own experimental data and metadata management as "non-core work" to be avoided. Some biology students and postdocs have described to me an antagonistic relationship to the funder or PI goals where they dismiss their responsibilities for data quality and reuse. For them, it isn't on the critical path to their paper and their career. Or, having data in disarray is, in their minds, giving them an edge to not be "scooped" in the lab. These cynical attitudes are almost subversive, in how the junior worker tries to evade or sabotage aspects of their job responsibility which do not serve them personally. I also see some who seem to engage in magical thinking, as if some other staff or informatician should come by later and clean up their mess of data, without the researcher putting in the effort. I often wonder whether they really misunderstand the problem so severely, or merely use this fig leaf to avoid feelings of shame. I know that lines should be drawn between different roles, but there are inappropriate expectations on both ends of a spectrum. PIs should not treat junior researchers like assistants to get coffee, run personal errands, etc. But neither should junior researchers believe that the day to day grind of the scientific process is beneath them. At the scales that most labs work, there are many essential yet mundane tasks that need to be shouldered by "core" participants. The overhead of trying to delegate these tasks to a different person would either require more time than doing the task oneself, or result in abysmal quality when the hand-off is made with insufficient supporting effort. I think that people exposed to the pop culture of startups and large corporations can absorb a mistaken idea of how easily human organizations scale. With vast numbers of skilled staff, you can create processes which scale beyond the capabilities of a typical small lab or group. These tend to develop lots of specialized roles to support that process. But, I think there is a huge inflection point between the efficient small group and the scalable large organization, where the marginal value of adding another staff member can be very small, or even negative. Many (most?) grant-funded research groups exist only because they can function in that small team mode, with relatively thin budgets and the need to wear many hats. The only safe way to scale work under these grants is what we see already: schools expand to have additional PIs running their own small, independent labs. These eventually compete with each other almost as much as they compete with outside labs. From the host school, they get actual "non-core" services like building facilities, human resources, payroll, contracts and legal support, etc. But, no one group can really scale to have more staff and specialization focused on a single research effort. There is a reason that "big science" projects are few and far between and are mired in much more politics---the granting agencies have to be motivated to write different kinds of funding opportunity to bootstrap such projects and to continue their operation. The budgets and timelines are vastly different.

rscho 5y ago

Good for you if your products do well. But making logos and such is your job as PI if you want it to happen. It's certainly not your students' job. They're here to learn academic skills and advance the field, not to make products that do well. Any material advantage they get from that should be incidental, and not a goal in itself.

lmeyerov 5y ago
The student picks the PI, and as part of that, the field of methods they want to learn, which may include scrappy prototyping. If they don't think that kind of R&D is their thing, they should pick another PI and, potentially, field. Ex: My wife's lab had paid staff, but she still had to feed cell cultures at 1am for the research she wanted to do, and later decided those methods were not for her and no longer does wetlab projects. That doesn't invalidate people still doing cell cultures as part of their methods, and I'd still love to see more robots & no-code & budget there to make it easier. Edit: RE:products, Whoosh. Academically derived products at that scale imply a community of practice and development: societal impact and dedicated makers growing it, including more knowledge. That should have the level of respect that a paper that opens up a small subfield gets. From a norms perspective, it's disappointing to see what sounds like a practicing academic to not value that. Separately, again, community of practice work values being embedded, so a student not enjoying it is their personal methods choice, not rules for others.
- rscho 5y ago
  Well, I guess we have different opinions about what academia should be, then. IMO, material incentives outside core academic activities are cancer and I think university spin-offs are heresy. If even tenured staff now attempts to "develop products" and thinks academia and industry R&D are similar, I think western science is finished.
  - lmeyerov 5y ago
    I don't think academia should be one thing, that's kind of the point of tenure You're promoting both a super traditional view yet also one that seems divorced from the last ~100 years of sociology studies wrt how society progresses wrt inventions and how people fit into that. So, it's great you follow old beliefs handed down by generations of giants from much earlier ages -- there's clearly precedent -- and I encourage you to learn about new ones from the last few generation of scientists who took a more scientific method to learning how recent progress often works. These methods are not for everyone, but it's useful to at least know about them before calling the techniques of your peers 'cancer'. Personally, as technology keeps speeding up, I expect this to become even more important, but at the same time, do value head-in-the-sand research too ;-) Also, to be clear: I am not rejecting the scientific method here. Products, and more importantly, the communities of practice around them, have a lot to do with hypothesis generation, methods, and evaluation. Most industrial labs, and ironically, especially ones that force researchers to embed into existing internal product teams, put a lot of hand cuffs on this process so academic env helps. It does take humility and empathy and time to incorporate them, so not so easy (kind of like startup-lite), but it is a family of tools and one aimed at more intentional progress. And no, 'products' is not synonymous with financial motivation: most OSS projects have no paid piece, for example.

throwawaygh 5y ago

I don't dispute any of this post. Actually, to be blunt, the fact that you think any of this is a retort to "pay people for the type of work you expect them to do" is deeply concerning. > reviewers assume they know best. You are probably more adept at running a grant-funded research group than me. I have no doubt. My concern here is ethical. I do think the average PI is in waaaay too deep to admit how horrible even the "good" PIs end up treating their phd students. So, in that sense, I do generally assume I know best. Not about how to do successful research. But about the true reality of how the people in those salt mines are being treated. On that note: of course all that non-core work is valuable! Incredibly valuable! And PIs should probably hire someone to do that work! At fair market wages! With accurate job titles and job descriptions! I agree with everything you're saying... except about who should do the labor and what they should be paid for that labor. I'm going to reiterate two sentences from my original post. >> Want a devops engineer? Hire a devops engineer ("systems scientist" or so on). Want a graphic designer? Hire a graphic designer. >>> ‘I’m a researcher, not a(n) X,’ with X∈ {software developer, designer, marketing person, artist, …} >> Sure, but here's the rub. What does everything in that set have in common? They make at least 2x+ the average PhD student and have much better benefits. >> If you want a "force multiplier" person in your lab, do your job: get a grant and pay a fair price for that sort of labor. PhD students are NOT ∈ {software developer, designer, marketing person, artist, …}. If they were, you'd be paying quite a bit more and/or getting far less exceptional candidates. Labs that have successful spin-offs are usually the worst about this sort of thing. They're running their research lab like a seedling incubator, but paying each of the one man shops printing their lottery tickets low five figures without retirement benefits.

lmeyerov 5y ago
My response is to the Reviewer #2 aspect of, instead of say championing a proposal on merit and, for their allocation issue, advocating for say increased budget for extra staff... rejecting it because of belief in Not My Job for something a good team would often either do scrappily or consider to be part of their job. One of the main costs and bottlenecks in modern academic R&D is the crazy low grant acceptance rates, forcing leaders to 5X grant writing time vs doing time, which makes this reviewer response 5X more nasty than you might otherwise think. If you want to go ethical - * PhD students, most of all, are learning how to do R&D: picking valuable problems, picking how to solve them, and most efficiently in time/money/etc solving then. Doing scrappy Day 1 work is very much a part of that skillset, surprisingly similar to a founding MVP team: Not My Job is teaching the wrong mindset and divorces you from the community + work, which may be what you are trying to learn. * ... However, Learning to Outsource/delegate is indeed part of efficiency training, so let's dig in. The default answer in most prototyping is do a crappy cheapo version, replace with a more costly version when it becomes the need, and reserve your shoestring budget for a few really important things. So likely no to a commercial grade design firm on rerainer, yes to either a $50 logo on Fiverr or more likely, something funny you can make via a free tool, done. Later, when there is more need and budget, say Year 2, easy to revisit. * Some things do merit staff. Should a bio lab partner with a CS team who wants to learn their needs as part of making new CS methods? That's T-shaped collaboration and good, and part of why academic teams can be so scrappy: there's ~always someone somewhere where your non-core work is their baby. Or, maybe they want to inhouse to get more speed/reliability/control. If that person will be like the CS team, ultimately making new CS methods, or a more computationally minded bio researcher helping with some of their own coding for faster EDA, great. But if they will always be Author #2 and has to code on projects they are not doing the science on, that's not a postdoc, that's a staff programmer. We should have more of those.