3 ms·

I am helping lead the first research group at a recently-founded institute in Germany. What is the mission of your institute? Is this a collection of individua

I am helping lead the first research group at a recently-founded institute in Germany. What is the mission of your institute? Is this a collection of individuals doing their own research, or a group of collaborators working towards a shared goal? How do your ideal members measure their own performance? Is it through papers, blog posts, tech demos, products, or something else? With academics you need to be especially vigilant about the difference between vanity metrics and real output. For funding, are you allocating cash on an annual basis, up-front, on-demand, or through some other scheme? What sort of timeline do you want for people to stay at the institute? And do you intend the budgets to be competitive, either locally or internationally? What lab infrastructure will you build and maintain, versus having the researchers handle? As a lab-based researcher, setting things up can easily consume the first 6-12 months (plumbing, electrical, HVAC, optics, wetlab equipment, regulations...). Once someone goes through the effort to build this they can easily become quite protective, potentially locking out newcomers or creating an internal system of patronage. Having answers for these questions and others will be essential for recruitment. Your first few hires will dictate the direction of the institute, since further recruitment will be driven largely by their networks. Feel free to email me (username at gmail) if you would like to discuss further.

_Nat_ 5y ago
> With academics you need to be especially vigilant about the difference between vanity metrics and real output. What would you consider "real output", especially in contrast to "vanity metrics"? Asking mostly because I'm curious how someone in your position would see it. For a lot of folks, academia's a fun ecosystem to participate in: they enjoy it and it pays the bills. To participate, they need to maintain popularity, much as a living organism needs to maintain energy. So their focus is on optimizing their popularity. They'd be primarily interested in metrics that represent their popularity, especially if those metrics help them plan how to gain more of it to further advance their career. But other folks participate in academia to do research for other reasons, e.g. advancing technology. They're more likely to focus on maintaining sustenance-level popularity rather than maximizing popularity. They'd tend to dismiss many popularity-centric metrics as frivolous. So if you're helping lead a research-institution, then are you primarily concerned with "real output" in the sense of "stuff that makes us popular enough to garner continued support"? Or are you more interested in another goal, like advancing science/technology? And how do you assess the value of a researcher's work given these goals?
- tsbischof 5y ago
  This is an important question, and something we have considered when building our group. My background is largely in experimental physical sciences, and one of the vanity metrics in my fields is publication in a Nature- or Science-family journal, or equivalent high-impact journal. In some cases publishing there has led to high-quality peer review and publication, but the pursuit of these papers drives many to over-sell or over-complicate the work. I tend to focus more on smaller studies which can be reasonably completely described, versus anthology publication. But more directly, we measure output in three categories, in order of importance: people, validated concepts, and publication/dissemination. For people, we want to develop a sense of integrity and curiosity, which means some mixture of freedom to roam and an ability to focus on defining and answering a concrete question. We want people to leave our group as good citizens and scientists. Validated concepts take several forms. In the physical sciences it is often expensive to recreate an experiment, so performing and reporting the correct set of controls and validations is critical. A successful experiment is one which does not need to be repeated, because there is sufficient clarity and design to assure the next person that what was found is correct. Often this means obtaining a proper negative result which shuts down entire lines of inquiry. Especially since many projects require months or years of build-up, we must ensure that the foundation is solid. As for publication and dissemination, our goal is that someone else would look at the work and consider building on top of it. In order for someone else to accept the work there needs to be a real trust or means for validation. We are open with our data, code, and methods, but much of the challenge lies in ensuring the dataset is valid and complete in the first place. You are right to note that popularity drives academia, in the same way that charisma and a good pitch can get a startup off the ground. High citation counts, big-name grants, fancy titles, and time spent at big-name institutions are the standard metrics for success. I come from a somewhat outside perspective since I spent some time in start-up prior to this position, where our focus was more on actionable outcomes than positive results. But for the students and postdocs, these pressures for advancement are real and not easy to reckon with. I have no good answer, apart from attempting to impart personal ethics and aligning interests where possible. The longevity and source of funding is a key factor in fighting the popularity contest. For our institute this is largely solved by solid political capital, which gives us some cover to pursue things without significant concern about short-term popularity. Long-term we want to be known for producing useful research, which others incorporate into their own work and become more productive. In short, a given researcher is successful if they have identified a self-contained question and answered it sufficiently to be a building block for further work, either as a positive or negative result.

_Nat_ 5y ago

> With academics you need to be especially vigilant about the difference between vanity metrics and real output. What would you consider "real output", especially in contrast to "vanity metrics"? Asking mostly because I'm curious how someone in your position would see it. For a lot of folks, academia's a fun ecosystem to participate in: they enjoy it and it pays the bills. To participate, they need to maintain popularity, much as a living organism needs to maintain energy. So their focus is on optimizing their popularity. They'd be primarily interested in metrics that represent their popularity, especially if those metrics help them plan how to gain more of it to further advance their career. But other folks participate in academia to do research for other reasons, e.g. advancing technology. They're more likely to focus on maintaining sustenance-level popularity rather than maximizing popularity. They'd tend to dismiss many popularity-centric metrics as frivolous. So if you're helping lead a research-institution, then are you primarily concerned with "real output" in the sense of "stuff that makes us popular enough to garner continued support"? Or are you more interested in another goal, like advancing science/technology? And how do you assess the value of a researcher's work given these goals?

tsbischof 5y ago
This is an important question, and something we have considered when building our group. My background is largely in experimental physical sciences, and one of the vanity metrics in my fields is publication in a Nature- or Science-family journal, or equivalent high-impact journal. In some cases publishing there has led to high-quality peer review and publication, but the pursuit of these papers drives many to over-sell or over-complicate the work. I tend to focus more on smaller studies which can be reasonably completely described, versus anthology publication. But more directly, we measure output in three categories, in order of importance: people, validated concepts, and publication/dissemination. For people, we want to develop a sense of integrity and curiosity, which means some mixture of freedom to roam and an ability to focus on defining and answering a concrete question. We want people to leave our group as good citizens and scientists. Validated concepts take several forms. In the physical sciences it is often expensive to recreate an experiment, so performing and reporting the correct set of controls and validations is critical. A successful experiment is one which does not need to be repeated, because there is sufficient clarity and design to assure the next person that what was found is correct. Often this means obtaining a proper negative result which shuts down entire lines of inquiry. Especially since many projects require months or years of build-up, we must ensure that the foundation is solid. As for publication and dissemination, our goal is that someone else would look at the work and consider building on top of it. In order for someone else to accept the work there needs to be a real trust or means for validation. We are open with our data, code, and methods, but much of the challenge lies in ensuring the dataset is valid and complete in the first place. You are right to note that popularity drives academia, in the same way that charisma and a good pitch can get a startup off the ground. High citation counts, big-name grants, fancy titles, and time spent at big-name institutions are the standard metrics for success. I come from a somewhat outside perspective since I spent some time in start-up prior to this position, where our focus was more on actionable outcomes than positive results. But for the students and postdocs, these pressures for advancement are real and not easy to reckon with. I have no good answer, apart from attempting to impart personal ethics and aligning interests where possible. The longevity and source of funding is a key factor in fighting the popularity contest. For our institute this is largely solved by solid political capital, which gives us some cover to pursue things without significant concern about short-term popularity. Long-term we want to be known for producing useful research, which others incorporate into their own work and become more productive. In short, a given researcher is successful if they have identified a self-contained question and answered it sufficiently to be a building block for further work, either as a positive or negative result.