23 ms·

Every attempt to manage academia makes it worse

549 points by balsam 10y ago

balsam 10y ago
Please look at the comments section of that post too.
najati83 10y ago
>as soon as you try to measure how well people are doing, they will switch to optimising for whatever you’re measuring, rather than putting their best efforts into actually doing good work Similar to https://en.wikipedia.org/wiki/Hawthorne_effect https://en.wikipedia.org/wiki/Hawthorne_effect ?
- stdbrouw 10y ago
  The Hawthorne effect describes how any change in environment can temporarily lead to positive effects. The sentence you quote is more akin to Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law).
tempodox 10y ago
> as soon as you try to measure how well people are doing, they will switch to optimising for whatever you’re measuring, rather than putting their best efforts into actually doing good work. The Heisenberg Uncertainty Principle has found a new refuge. While atoms are evading it ([1]), people implement it. [1] https://www.scientificamerican.com/article/atomic-spins-evade-heisenberg-uncertainty-principle/ https://www.scientificamerican.com/article/atomic-spins-evad...
jrs95 10y ago
While I think there is a lot of truth to this, I don't see how it can effectively be applied to our existing educational institutions. They've sort of become a self-serving bureaucracy at this point, so even without misguided policy pushing them in the wrong direction, I don't think doing the best work they can is actually their primary focus.
- worldsayshi 10y ago
  Naive response: Throw out the bureaucrats. Leave those with subject knowledge. Tell them to do their work to the best of their ability? Stop measuring things. Talk to them once in a while.
  - BoorishBears 10y ago
    What about the bureaucrats with tenure?
    - pdimitar 10y ago
      Your question contains a good chunk of our entire world's economic and bureaucratic problems -- namely people in power with the emotional intelligence and experience of a chihuahua.
  - dagw 10y ago
    Throw out the bureaucrats. Leave those with subject knowledge. My wife works in academia and all the 'managers' in her research department are experienced researchers that often split their time between management and research. And it just does not work according to her. They don't understand the job of a manager, deem it less important than their research duties and basically see dealing with 'minor' personnel issues as a waste of their time. To run any organization of a non-trivial size requires some structure and bureaucracy. And you need people to understand, set up and own those structures.
    - madez 10y ago
      Maybe the personnel issues are sometimes indeed minor and a waste of time. What are the symptoms your wife is observing?
      - dagw 10y ago
        Minor stuff like spending 3 weeks "unemployed" and without a salary because her boss couldn't be arsed/was too busy to sort out the paperwork, grant money almost getting lost in the shuffle, stuff like that. And even if you perceive a personnel issue as genuinely minor and you genuinely do consider it a waste of time, telling someone that problems they perceive as genuine are so insignificant to you that you don't even want to consider them, despite that being part of your job, is a great way to build resentment and destroy teams. A good manager should be able to quickly and efficiently handle minor issues without making people feel ignored and insignificant.
bcrack 10y ago
>"could it possibly be … that the best way to get good research and publications out of scholars is to hire good people, pay them the going rate and tell them to do the job to the best of their ability?" Sounds very reasonable. Does anybody have good examples of this approach? I have the impression that this is similar to the environment of Bell Labs during the "golden years".
- johan_larson 10y ago
  You're going to need some sort of controls, if only to kick out the folks who retire in place.
  - dasmoth 10y ago
    If one were being truly radical about this, a possible solution is to replace researchers' salaries with pensions -- remove the incentive to retire in place by allowing you to retire at home if that's what you want.
    - _jal 10y ago
      And then you have a pool of professors made up of people who can afford to teach for free, or perhaps create a system where some finance leech buys your future pension for NPV payouts now.
      - dasmoth 10y ago
        a system where some finance leech buys your future pension for NPV payouts now. Isn't that called a loan? I imagine that smart finance-leeches will tend to avoid loaning against 100% of the value of the pension, for much the same reason it's hard to get a mortgage that takes 100% of your income to repay. Beyond that -- where is the problem?
        _jal 10y ago
        I said NPV. Beyond that, the problem would be getting the same results as paying someone directly, while reducing efficiency by inserting a middleman to take a piece for... why, exactly?
n4r9 10y ago
I'm sure this resonates with university lecturers all over the Western world. The late writer and university academic Mark Fisher has some interesting points about this effect in his 2009 book "Capitalist Realism: Is there no alternative?"[0]. He saw it as an inevitable byproduct of the infiltration of neo-liberal ideology into the academic sphere. Setting targets and measuring performance indicators are justified with efficiency arguments, but ultimately cause an increase in bureaucracy and a decline in mental health. [0] https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_no_alternative%3F https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_n...
- erelde 10y ago
  I have often argued that "being realist" is a failure in and of itself. If people said "well, there's a king and that's what it is", we'd never have got to where we are now. Ideologies (and ideas) are, in my view, stronger than facts, which often only represent the past, while ideas shape the future. (these days the word "fact" is a bit of a trigger, but I thought about that well before all that jazz, and the current phenomenom seemed to prove my point to many of my friends)
  - deleted 10y ago
    [deleted]
  - aaron-lebo 10y ago
    Can't you be a realist while still being idealistic? Isn't that a bit of an advantage, too - seeing the world for what it actually is? Searching for historical examples, JFK was both?
    - icebraining 10y ago
      Fun fact: that's the original definition of the term Realpolitik, as coined by Ludwig von Rochau, a German writer and politician in the 19th century. He said that the great achievement of the Enlightenment had been to show that might is not necessarily right. The mistake liberals made was to assume that the law of the strong had suddenly evaporated simply because it had been shown to be unjust. Rochau wrote that "to bring down the walls of Jericho, the Realpolitiker knows the simple pickaxe is more useful than the mightiest trumpet." I've been reading "Realpolitik: A History" and I think there are some useful lessons for today.
sosodaft 10y ago
Reminds me of James C. Scott's "Seeing like a State", which chronicles how various attempts to make complex, organically developed institutions organized and intelligible has historically made things much, much worse, from planned cities to collective farms. If it ain't broke...
Y_Y 10y ago
So many early-career researchers (myself included) bemoan this system and it pushes many of the away from academia and towards fields where you aren't constantly playing tricks (fake co-authorship, salami publishing, over-hyping conclusions) to get ahead in a game where everyone's striving for a tiny number of badly paid jobs.
- deleted 10y ago
  [deleted]
yummyfajitas 10y ago
This is a bunch of cool sounding scientific nihilism, but it's false. Nclb was a great attempt to manage academia and it worked. http://www.nber.org/papers/w15531 http://www.nber.org/papers/w15531 http://www.nber.org/papers/w20511 http://www.nber.org/papers/w20511 http://www.nber.org/papers/w16745 http://www.nber.org/papers/w16745
- HarryHirsch 10y ago
  The authors themselves admit that NCLB was a mixed bag: "Specifically, we find that NCLB generated large and broad gains in the math achievement of 4th and (to a somewhat lesser extent) 8th graders. However, our results suggest that NCLB had no impact on reading achievement for 4th or 8th graders." You'd think that you can employ drill practice to improve results in mathematics, but that approach won't work in English. American math instruction feels quite alien for foreigners, the emphasis is on the application of algorithms, not on understanding why and how they work. Equipped with plenty drill practice, an Amarican child can fill in the correct bubble on the test, no matter if or not it understands mathematics. Any understanding of mathematics gained in that system is purely incidental. That's all a hypothesis and would require further testing.
  - maxerickson 10y ago
    Drill practice is easy for English. Read. The hard part is motivating people to actually do it.
  - yummyfajitas 10y ago
    That's net positive - an increase in some subjects and neutral in others.
csneeky 10y ago
All employers want their employees to "do the job to the best of their ability". But the freedom given to you to decide how your success is measured is proportional to your experience and past performance. There are those in both academia and private industry that get a lot of freedom to do their work/research. But there is a hierarchy to this just like everything in nature. The exceptionally strong and bright animals have more freedom in why they choose to do and how they do it. And that system has worked to get us all this far. The system this article is pushing against isn't fun for many but inevitable. The further you are from the top the more subject your performance is to poor evaluation schemes conceived by those far from the top too.
- pdimitar 10y ago
  Scientists aren't employees. They are given the tools, real estate and funding to do their thing, usually through research grants or private funding. The fact that nowadays they are treated as employees is exactly the problem. ...Oh, and by the way, nature has no hierarchy.
  - csneeky 10y ago
    Scientists are employees in many contexts (almost all contexts). We may just have to agree to disagree on that one. Be they in service for an NGO, school, or oligarch. But to say nature has no hierarchy? I suppose "hierarchy" can mean many things. But in my case I mean fitness and selection for it. The most fit are at the top. Nature tends to select for those better suited to their environment. And those best suited thrive and push the success of us all forward. It is how you got here and have what you have. Intellectual, emotional, and physical traits are all subject to this selection. The thriving academics with the most freedom are closer to the top. The best employees (however you want to define employee) tend to be given the most freedom and trust.
    - pdimitar 10y ago
      If you look at any nature's habitat of N kinds of animals, even there what you see can't be called a hierarchy. It's a semi-stable state on the verge of chaos which can be very easily broken by introducing a species of animals new to the area. So no, it's not hierarchy. Lions can kill almost anything if they feel like it, but hyenas kill lions on a regular basis. Does that make them the top of the food chain in the savannah? Definitely not. It only makes a pack of hyenas stronger than a single lion or lioness. Rock, paper, scissors. No hierarchy. It's the same in the human systems. Hierarchy is an artificial system enforced by humans and has almost zero relation to anything natural.
baldfat 10y ago
I have a different take its the lack of pay for professors and the lack of respect for public education (On average a college professor makes less then a public school teacher, in my discipline I would have averaged about $20k less IF I got tenure) As a former academic professional (Librarian at a college) The high cost of college is based on the demands of students. 30 years ago there was no internet, no smoothy bar equipped fitness rooms, no pretty dorms and everything upscale. No pay increase for staff and I actually went through 3 years with no cost of living increases while the cost for college went up 15%. Then the idea that Public Education in the US is a failure (Totally false narrative except in the cities and other low income communities). So people believe that private investment and legislation will fix things. This just has people coming in making and taking the money from teachers and the community and taking them else where.
- marcelluspye 10y ago
  >30 years ago there was no internet, no smoothy bar equipped fitness rooms, no pretty dorms and everything upscale. How certain are you that these are the reasons school has gotten more expensive? I'm a student at a school with a decent amount of cushy stuff and a high tuition, and I can't say it looks like that's where the money's going (or even being sunk). Housing got cushy once university policy stopped forcing us to live on campus, at which point it had to compete with surrounding options (before that we were charged through the nose for pretty tight quarters). The school-provided food options are, ... well, the university's definitely making money on those, and if they're not the company they contract out to is. I gotta say, I don't think paying for gym equipment has significantly held up salary increases on the academic side of things. You might know better than I (and we're probably in different places), but looking around campus the things that seem to be sucking money away from paying the professors more is a) the ever-expanding number of administrative offices and b) the constant renovation/replacement/addition of buildings. I'll be the first to tell you my campus is an eyesore, but it feels more and more every year like they did that on purpose in the 70s so they wouldn't face much resistance in sinking a ton of money to replace it with a shinier one now. I'm also not totally unwilling to believe the standard economic argument that gets thrown out all the time: with so many more people leaving school with advanced degrees, finding people to fill teaching positions is a lot easier.
dasmoth 10y ago
“If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it. -- Attributed to J.J. Thomson (although I've not been able to turn up a definitive citation -- anyone know where it comes from?) Does make me think some people have been pondering this for a long while though.
- pdimitar 10y ago
  I don't know where the quote is from but it represents my only hope for the future of research. Of course, the hardcore bean-counters would not let that middle ground happen either; they are people in power who absolutely positively won't give you almost any leisure. It's still a good idea however. And some managers would embrace it since the notion of "paying somebody for possible benefits 5-10 years in the future" is something which our current form of capitalism hates. But some of them are reasonable people and would accept a middle ground. Another thing I was thinking about in the past is rotational work: given a team of 6 people, have 1 person always working on a research for at least a month with zero responsibility about the money-making activities. After that month -- or two, or six -- expire, return the person back to the capitalistic loop and put somebody else on the research duty. This however is rather clunky because people will have to be constantly caught up with where are the things currently in the money-making job and the research job. Perhaps there are other ideas as well, but I am not aware of them.
  - quantum_state 10y ago
    In a university setting, it seems to me that teaching in combination with support from endowment (not research grants) can be one of the scenario for this ... key is still to have fundamentally good people with passion and joy for the endeavor ..
ZeroGravitas 10y ago
This isn't an academic issue, the same is true of almost every white collar profession. To apply standard numerical management techniques you either need to know exactly what a person needs to do as part of the process, or be able to evaluate the output, ideally both, and generally neither are true.
pmilot 10y ago
My wife works as a teacher at one of the largest schoolboards in Québec, and she can tell you that the solution "hire good people, pay them the going rate and tell them to do the job to the best of their ability" doesn't work in a unionized system. Complacency a systemic problem. When your union protects your job, you don't need to actually put effort into doing the job. Of course, that doesn't mean ALL teachers are lazy; but it does mean that a teacher who works hard and gets better results than average is not rewarded for it. It also means that bad teachers cannot be replaced by good ones. Of course, preventing public school teachers from unionizing will simply allow the government to exploit them. Perhaps the solution would be to replace unions with a professional order, similar to how engineers and lawyers are regulated.
- wycx 10y ago
  An infernal situation. There is no solidarity in a meritocracy, and no reward for merit with solidarity.
  - trentnix 10y ago
    On the contrary, consider team sports as a perfect example of a meritocracy which often also promotes solidarity. A good team, led by a good coach, recognizes merit is earned by the performance of the whole. The whole wins and the whole loses together. The best of sport very clearly defines success and failure. It uses competition to spur innovation and the pursuit of excellence. Solidarity is often the consequence of that pursuit. Very little about education, from how teachers are managed to how schools are run, does the same.
    - geodel 10y ago
      For sports one very important difference is that real time performance is monitored by a large number of people. People who do not pay salaries of sports person and do not meddle with working of sports team directly. For any other kind of job performance is measured by insiders. Outsiders may gauge performance on quarterly / annual basis.
golemotron 10y ago
It would be great to see Nassim Nicholas Taleb's response to this. He's been working on a draft called Skin In The Game. Overall, I think that the GP is right but seeing that requires us to consider that nebulous ideas like character and cohesive culture need to be at the forefront more than market reasoning.
kev009 10y ago
I think this is somewhat well understood in things like Sports Psychology. You don't tell a receiver to catch X number of throws per game, you just encourage them perform. Likewise, training that more accurately simulates performance is more effective than working toward some metric. I learned it from Pat McNamara who has a special way with words and delivery: "When we are consumed by the outcome it will sabotage our ability to perform." - http://soldiersystems.net/2012/09/15/gunfighter-moment-pat-mcnamara/ http://soldiersystems.net/2012/09/15/gunfighter-moment-pat-m...
- mseebach 10y ago
  This works because both sides have a well-understood model of what 'performing' looks like. And if you don't, you'll be shown the door very quickly.
  - hn_throwaway_99 10y ago
    I think this is key. In the sports arena, the metrics are VERY clear: if you don't win games, and your stats are poor, you're fired. I understand where academics are coming from, but at the same time it seems like many people want to be isolated from any objective assessment of their performance at all. FWIW, I see this a LOT among software developers as well.
- CuriouslyC 10y ago
  Current research indicates that the best way to promote performance is not to encourage someone's current performance, but instead encourage their development of skill mastery. In the short term, rewards (like money) produce the best results of all, but they also seem to impair learning.
- mason240 10y ago
  >You don't tell a receiver to catch X number of throws per game They kind of do though. In the NFL, many contracts are incentive laden with bonuses for (in the case of a wide receiver) X number of catches or Y number of yards.
pjmorris 10y ago
If the blog post seems anecdotal, there's a great, underused book, 'Measuring and Managing Performance in Organizations' by Robert Austin that covers his CMU PhD work, which came to the same conclusion as the quote attributed to Harford: "If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability." It doesn't hurt to make sure they don't have to worry about where to get, e.g., paper clips, lunch, or health care, so some level of administration is helpful.
kesor 10y ago
Back in 2010 there was an attempt to put Deliverology in place at California State University. It basically means that academia will be measured by targets. A horrible idea that made the California Faculty Association meet and decide on almost mutiny. They even invited John Seddon who wrote several books on the subject of how Deliverylogy twisted and perverted systems in the UK. Fortunately for us, the video of his presentation is available on YouTube at https://youtu.be/8EGMlCau6iU https://youtu.be/8EGMlCau6iU for part 1, and has about 5-6 parts. If you liked that, then I would highly recommend watching John Seddon's presentation about how settings targets and implementing broken IT systems to enforce those targets are a horrible idea. https://youtu.be/hbNsQFd8DQw https://youtu.be/hbNsQFd8DQw
- kesor 10y ago
  The full explanation of CFA Deliverology problem in 2010 is located on their page on the subject: http://www.calfac.org/resource/deliverology-etc-resources http://www.calfac.org/resource/deliverology-etc-resources
__roland__ 10y ago
I think one way to 'treat' this would be to acknowledge that certain things just cannot be optimized easily with outside incentives, and to simply accept that there will be some inevitable waste when you let 'good people do their job as best as they can'. It's ironic that we fret so much about professors that 'retire on the job', but at the same time think it is perfectly fine to steal time from the motivated researchers, by making them playing these silly games too. Who really thinks the next breakthrough will come from a person that was forced into writing another paper, just to pass some arbitrary performance threshold? If you consider research as a 'stochastic race', i.e. many people working on a problem with their own intuitions and approaches, it makes no sense to slow down those that are most likely to come up with a good solution (those that are self-motivated). IMHO, it does not really matter if those that are unlikely to contribute much (retired on the job) get a free pass. After all, even to receive tenure takes a lot of effort (and thus, ambition) nowadays, so at least in fields with a competitive job market (i.e. good salaries outside academia) it makes no sense to strive for a tenured position and then not do what you love (i.e., research). BTW Daniel Lemire's blog has many interesting posts on the more absurd aspects of academia, including research grants (e.g. http://lemire.me/blog/2009/09/15/the-hard-truth-about-research-grants/ http://lemire.me/blog/2009/09/15/the-hard-truth-about-resear...). IIRC one of his suggestions was so fund people (for longer terms), not 'projects'. This may not always be possible (e.g., if you need expensive equipment), but in many fields it would make much more sense. EDIT: language
Lxr 10y ago
If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability. Absolutely, could not agree more. This is the only way to beat gamification of metrics in my opinion. There is a worrying trend of treating staff as equally capable but untrusted cogs in a wheel, and metricising performance. I fundamentally believe that no amount of rules or metrics can turn a bad academic into a good one, an unsafe doctor into a safe one or a crooked politician into an honest public servant. We have to calm down with the metrics, accreditations and regulations, and accept that some people are better than others at certain things, hire the right people and trust them to do a good job.
- pg_is_a_butt 10y ago
  yeah, white men. you're all idiots.
- merpnderp 10y ago
  Seems like in IT the spread of different process systems is exactly this. To reduce each job to a set of functions that people with similar levels of training can do the job at the same level of output, interchangeably. So we're left with jobs that are 90% following inane process and 10% producing something useful. All to make sure if we quit, we can be replaced in a few days.
  - LanceH 10y ago
    Where do you sit? Let's go to lunch today.
  - colechristensen 10y ago
    Seems like if the tedious portion of the job reaches such a high proportion it's significant lack of automation.
  - wpietri 10y ago
    The folks at Toyota talk about two different kinds of process: controlling and supportive. Controlling is that sort of BS. And I had seen it so much early in my career that I thought that's all there was. Supportive process is bottom up, done to improve consistency and reliability. E.g., by my door I have checklists for going running and swimming, because I was tired of going out to exercise and realizing I had forgotten something. I say this because I think process can be amazingly helpful if it's done bottom up, if the workers are the ones who design and improve it. But that only happens if managers have another thing that Toyota talks a lot about: "respect for people" [1]. That's what I think is really missing in these metrics-driven, people-are-cogs schemes. [1] https://www.lean.org/womack/DisplayObject.cfm?o=755 https://www.lean.org/womack/DisplayObject.cfm?o=755
kesor 10y ago
“Tell me how you measure me and I will tell you how I will behave. If you measure me in an illogical way… do not complain about illogical behavior…” - Dr. Eliyahu M. Goldratt
Fede_V 10y ago
That list is correct, but it's only half of the problem. The other other half of the problem is supply. In fields like mathematics, physics, computer science, grad students have excellent non-academic options. There is an oversupply of PhD students with respect to the number of available positions, but the excess PhD students have fantastic career avenues. In fields like the humanities, and, to a lesser extent, the molecular biology/ecology, there is a huge excess of PhD students, and there is a very limited number of career options outside of research (for bio, there is pharma, for everything else, eh). This is compounded by the fact that for professors, postdocs are an absolute steal (http://www.sciencemag.org/careers/2017/01/price-doing-postdoc http://www.sciencemag.org/careers/2017/01/price-doing-postdo...). Professors get to pay 50-55k (and that's on the high very high end) a year for a PhD trained researcher who will often work themselves to the bone in the hope of getting the publications that will allow them to become a professor. Thus - the very people who have the power to change the system instead greatly benefit from the status quo. The real reform that has to take place is that PhD programs need to become way more selective. Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis.
- jessriedel 10y ago
  > Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis. There are lots of industries like this, e.g., corporate law, sports, acting, music. Are they all "in crisis"? http://slatestarcodex.com/2015/07/28/non-dual-awareness/ http://slatestarcodex.com/2015/07/28/non-dual-awareness/ [EDIT] Others: military officers, investment banking, organized crime, tech start-up, trade guilds, etc.
  - zepto 10y ago
    Yes
  - anoonmoose 10y ago
    Law is in crisis, the other three are all so (relatively speaking) small, unimportant, and gated by the genetic lottery that they're not worth discussing.
maxforce 10y ago
The system is managed perfectly to minimise agendas being undermined. Those in power have worked their entire lives to gain it and will have no issue papering over any accusation that veracity is not their sole priority, let alone a complete farce. "In many scientific fields, results are often difficult to reproduce accurately, being obscured by noise, artefacts, and other extraneous data. That means that even if a scientist does falsify data, they can expect to get away with it – or at least claim innocence if their results conflict with others in the same field. There are no "scientific police" who are trained to fight scientific crimes; all investigations are made by experts in science but amateurs in dealing with criminals. It is relatively easy to cheat although difficult to know exactly how many scientists fabricate data" https://en.wikipedia.org/wiki/Scientific_misconduct https://en.wikipedia.org/wiki/Scientific_misconduct
dkural 10y ago
I had a similar problem with hiring product designers. How do you get & measure great design? What I ultimately decided was, the only way to get great design, is by hiring great designers, and only other great designers recognize who's very promising vs. who's not, with some degree of success. Research is the same. However, due to the inherent entanglement of historical biases with merit, it became very fraught in the U.S. to rely on expert judgment to pick new professors, every school clamoring to find some objective metrics to avoid charges of discrimination. This is coupled with the fact that most departments are filled with mediocre people who are not particularly qualified to pick promising young scientists to begin with. So ironically, discrimination is still there & at times worse, but now there is metricized cover for this bias. "We found that tall white dude X has more publications than Y". Meanwhile, academia right now is almost irrevocably filled with pedantic scientists, drip-feeding papers with maximal verbiage at a steady pace.
gustosoo 10y ago
I would suggest to teach and emphasize the role of science and technology for our future. Social esteem for the role of the scientist and technologist is important to nurture scientific talent. Reading Paul Drucker book about how America respect for the technological man and British respect for science man explains why America took the lead in technology. For young people motivation to solve some very important problem by means of science and technology and giving them freedom, good money and respect is the right combination to make the any field advance.
gustosoo 10y ago
Could we use deep learning to select who is going to do a good job in academia?
- dredmorbius 10y ago
  DL is a great way to reimplement extant bias whilst obscuring the reimplementation.
gustosoo 10y ago
I think than more than never working in a team is important. But a team must be prepared to accept A people what is not a small feat.
gustosoo 10y ago
There a a lot of comments here and the take-away for me is that managing academia is a very difficult task and nobody knows how to do it. Some people recognize that we could make more harm than good by adopting short-sight metrics, I consider fortunate that people recognize that sometimes we don't know all the answers and that the best we can do is to share information about this complex task.
gustosoo 10y ago
Since here many things are about start-ups, what about a start-up for developing a better academia, a way for great people to expand the frontiers of knowledge? Tim harford Ted talk about error and evolution is about how the only way to manage complex system is about trial and error, perhaps he would subscribe that trial and error is the only way to search for a better way of managing academia. http://www.ted.com/talks/tim_harford/transcript http://www.ted.com/talks/tim_harford/transcript
ThomPete 10y ago
I think we need to rethink what we consider academically important which should be taught at universities and then what should be taught as self studies. I can't help but thinking that one of the real problems is that a lot of academia isn't that valuable anymore as a field of study. Don't get me wrong I consider ex. philosophy as one of the most important fields but I also feel like it doesn't belong as an indiependent discipline anymore. Maybe I am wrong but after postmodernism I really don't see what new revelations are going to be of fundamental value to society. I see plenty of need for applying philosophy in various fields as interpretation of those fields such as neuroscience, quantum mechanics, technology etc. but as a stand alone field I think it gave us what we needed. And even if I am wrong you don't need an academic faculty to support it anymore. I feel the same with something like psychology which to me is even more problematic as it it's based on a premise that you can learn about the human mind by studying humans. Again it should be tied to something like Neuroscience instead. That way the fields are put to good use rather than being self servant. I am aware this might not be a popular opinion and it's not something that I am 100% certain of myself but it seems to be that a lot of academia could be learned without large institutions to support and and thus with the need to measure it the only way large institutions normally can. Thoughts?
- trishume 10y ago
  It's really easy to make punchy-sounding arguments about why a certain field isn't as important to teach, so we should be rather reluctant to go along with them. For example your argument about psychology could equally applied to almost any field and physics. "I feel the same way about (Computer Science|Biology|Chemistry) which to me is even more problematic as it is based on the premise that we can learn about (computers|life|chemicals) by studying (code|proteins|chemicals). Again it should be tied to something like quantum physics instead." In fact this argument makes no sense to me, why wouldn't studying human behaviour be the best way to learn about human behaviour? Same for the argument about philosophy, it's too easy to assume something valuable a field produces is too amazing to ever be topped. "After [the light bulb] I really don't see what new revelations [about electricity] are going to be of fundamental value to society."
  - 10y ago
jccalhoun 10y ago
I'm currently a lecturer at a small midwestern liberal arts university. I don't know what kind of job I will have next year because the Dean decided that there will no longer be one year full time contracts and they won't add any more three year contracts. Theoretically they want to have more tenure track job but in practice our department isn't getting any more. So at worst I will be part time next year which means I won't have insurance. Best case one of the job applications I have pans out and I get a tenure track job. The reason for the change is, of course, money. It is cheaper to hire 5 part time people to teach the load of 3 full time people because the part timers get less pay and no benefits. Of course finding 2 extra people who are qualified doesn't seem to be something they are worried about... Strangely, they haven't suggested switching to part-time administrators...
- flamedoge 10y ago
  of course that would be outrageous! we should be giving them millions in bonus!
uaaa 10y ago
Management and other myths 1. Complex systems tend to produce complex responses (not solutions) to problems. 2. Great advances are not produced by systems designed to produce great advances. https://en.wikipedia.org/wiki/Systemantics https://en.wikipedia.org/wiki/Systemantics
- dredmorbius 10y ago
  "The chief cause of problems is solutions." -- Sevaried's Law
- dredmorbius 10y ago
  Also, thank you so very much for Systemantics. I've run across it before, but very much needed to run across it here and now.
jknoepfler 10y ago
My take on the desire to regulate/manage is that some people would rather have an economy of perverse incentives than tolerate the inevitable abuses caused by lack of accountability. Management in the programming world feels the same way. At first, to many of us, this seems foolish and counter-productive, but then think: how does a university policy maker or a manager explain the rotten eggs (low output academics, unproductive developers) to their "boss," which might be the tax paying public or a budget strapped firm? How does a competitive university distinguish itself in a communicable/verifiable way? Could I be fired, or in the private sector sued for letting smart people fuck up unsupervised? It is difficult to trust others when the worst case outcome exposes us to catastrophic consequences. I personally agree with the author, but I think there's serious work that could be done to mitigate or remove the incentives to manage, and I don't think those are as thin as "tell people to stop trying to control what they can't"
cr0sh 10y ago
This article, and the follow-up on it (link is in the article) is about something very important, and should be more widely known about and discussed by anyone with an interest in science, research, and funding for it all. It points (yet again - as we've all seen the various p-hacking commentary) to the very real problems in scientific research that seemingly threaten to undermine it - especially from the public's point of view. It has been said that "religion poisons everything" - there is probably a corollary of "politics poisons everything" as well; both of these are really just social control and exhibition proxies. Ultimately, human nature rears its head, and the want for more money, power, and prestige (at all levels) leads to these results. ...and society becomes poorer for it. The questions and the proposed ideas aiming at solutions to the problems discussed sound plausible (or workable) on the surface, but I tend to wonder if any and all solutions will just be able to be gamed anyhow? At any rate, I'm glad that this was posted, though I despair at it leading to any solution, as it seems the problems lie within our psychology and society, and are thus nearly intractable.
jeremyt 10y ago
In the most non-trolling way that I can manage, why can this not be generalized to the economy? Why is academia a special case scenario where bureaucracy and regulations create unintended consequences and cause people to expend effort to get around them, but we can't say the same thing about the economy as a whole? Why do we believe that people can't even manage PhD's at a university, but we can take the same people, put them in government, and expect them to, for example, manage the monetary supply or see bubbles in advance and create proper policy to avoid them?
- MooMooMilkParty 10y ago
  I think one of the hard things about academia (by ways of "pure research"), is measuring impacts. In some positions there are metrics which are more representative of real progress than others. My belief is that measuring an academic's "impact" is a very fuzzy thing in the long term, especially when you bring in people who don't specialize in the same field. That's a reason this can't be generalized fully, though you can probably find corollaries. But, your last sentence to me, seems like a good one. Presumably, that would be an excellent topic to pursue research in, considering we clearly don't have a working theory on it (but that's not my specialty so who knows, maybe there's a bunch of smart people being ignored about it). Thus, it's hard to measure progress in, and we institute perverse incentives per the article.
otempomores 10y ago
You cant put a metric on the future.. In the literal sense..you cant measure whatyou yet dont know..or how valueable it will be for humanity.
bluefox 10y ago
I'll just note that this entry was posted 8 times by different accounts. Someone wants you to adopt an opinion.

549 points by balsam 10y ago

balsam 10y ago

Please look at the comments section of that post too.

najati83 10y ago

stdbrouw 10y ago
The Hawthorne effect describes how any change in environment can temporarily lead to positive effects. The sentence you quote is more akin to Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law).

tempodox 10y ago

> as soon as you try to measure how well people are doing, they will switch to optimising for whatever you’re measuring, rather than putting their best efforts into actually doing good work. The Heisenberg Uncertainty Principle has found a new refuge. While atoms are evading it ([1]), people implement it. [1] https://www.scientificamerican.com/article/atomic-spins-evade-heisenberg-uncertainty-principle/ https://www.scientificamerican.com/article/atomic-spins-evad...

jrs95 10y ago

While I think there is a lot of truth to this, I don't see how it can effectively be applied to our existing educational institutions. They've sort of become a self-serving bureaucracy at this point, so even without misguided policy pushing them in the wrong direction, I don't think doing the best work they can is actually their primary focus.

worldsayshi 10y ago
Naive response: Throw out the bureaucrats. Leave those with subject knowledge. Tell them to do their work to the best of their ability? Stop measuring things. Talk to them once in a while.
- BoorishBears 10y ago
  What about the bureaucrats with tenure?
  - pdimitar 10y ago
    Your question contains a good chunk of our entire world's economic and bureaucratic problems -- namely people in power with the emotional intelligence and experience of a chihuahua.
- dagw 10y ago
  Throw out the bureaucrats. Leave those with subject knowledge. My wife works in academia and all the 'managers' in her research department are experienced researchers that often split their time between management and research. And it just does not work according to her. They don't understand the job of a manager, deem it less important than their research duties and basically see dealing with 'minor' personnel issues as a waste of their time. To run any organization of a non-trivial size requires some structure and bureaucracy. And you need people to understand, set up and own those structures.
  - madez 10y ago
    Maybe the personnel issues are sometimes indeed minor and a waste of time. What are the symptoms your wife is observing?
    - dagw 10y ago
      Minor stuff like spending 3 weeks "unemployed" and without a salary because her boss couldn't be arsed/was too busy to sort out the paperwork, grant money almost getting lost in the shuffle, stuff like that. And even if you perceive a personnel issue as genuinely minor and you genuinely do consider it a waste of time, telling someone that problems they perceive as genuine are so insignificant to you that you don't even want to consider them, despite that being part of your job, is a great way to build resentment and destroy teams. A good manager should be able to quickly and efficiently handle minor issues without making people feel ignored and insignificant.

bcrack 10y ago

>"could it possibly be … that the best way to get good research and publications out of scholars is to hire good people, pay them the going rate and tell them to do the job to the best of their ability?" Sounds very reasonable. Does anybody have good examples of this approach? I have the impression that this is similar to the environment of Bell Labs during the "golden years".

johan_larson 10y ago
You're going to need some sort of controls, if only to kick out the folks who retire in place.
- dasmoth 10y ago
  If one were being truly radical about this, a possible solution is to replace researchers' salaries with pensions -- remove the incentive to retire in place by allowing you to retire at home if that's what you want.
  - _jal 10y ago
    And then you have a pool of professors made up of people who can afford to teach for free, or perhaps create a system where some finance leech buys your future pension for NPV payouts now.
    - dasmoth 10y ago
      a system where some finance leech buys your future pension for NPV payouts now. Isn't that called a loan? I imagine that smart finance-leeches will tend to avoid loaning against 100% of the value of the pension, for much the same reason it's hard to get a mortgage that takes 100% of your income to repay. Beyond that -- where is the problem?
      - _jal 10y ago
        I said NPV. Beyond that, the problem would be getting the same results as paying someone directly, while reducing efficiency by inserting a middleman to take a piece for... why, exactly?

n4r9 10y ago

I'm sure this resonates with university lecturers all over the Western world. The late writer and university academic Mark Fisher has some interesting points about this effect in his 2009 book "Capitalist Realism: Is there no alternative?"[0]. He saw it as an inevitable byproduct of the infiltration of neo-liberal ideology into the academic sphere. Setting targets and measuring performance indicators are justified with efficiency arguments, but ultimately cause an increase in bureaucracy and a decline in mental health. [0] https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_no_alternative%3F https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_n...

erelde 10y ago
I have often argued that "being realist" is a failure in and of itself. If people said "well, there's a king and that's what it is", we'd never have got to where we are now. Ideologies (and ideas) are, in my view, stronger than facts, which often only represent the past, while ideas shape the future. (these days the word "fact" is a bit of a trigger, but I thought about that well before all that jazz, and the current phenomenom seemed to prove my point to many of my friends)
- deleted 10y ago
  [deleted]
- aaron-lebo 10y ago
  Can't you be a realist while still being idealistic? Isn't that a bit of an advantage, too - seeing the world for what it actually is? Searching for historical examples, JFK was both?
  - icebraining 10y ago
    Fun fact: that's the original definition of the term Realpolitik, as coined by Ludwig von Rochau, a German writer and politician in the 19th century. He said that the great achievement of the Enlightenment had been to show that might is not necessarily right. The mistake liberals made was to assume that the law of the strong had suddenly evaporated simply because it had been shown to be unjust. Rochau wrote that "to bring down the walls of Jericho, the Realpolitiker knows the simple pickaxe is more useful than the mightiest trumpet." I've been reading "Realpolitik: A History" and I think there are some useful lessons for today.

sosodaft 10y ago

Reminds me of James C. Scott's "Seeing like a State", which chronicles how various attempts to make complex, organically developed institutions organized and intelligible has historically made things much, much worse, from planned cities to collective farms. If it ain't broke...

Y_Y 10y ago

So many early-career researchers (myself included) bemoan this system and it pushes many of the away from academia and towards fields where you aren't constantly playing tricks (fake co-authorship, salami publishing, over-hyping conclusions) to get ahead in a game where everyone's striving for a tiny number of badly paid jobs.

deleted 10y ago
[deleted]

yummyfajitas 10y ago

This is a bunch of cool sounding scientific nihilism, but it's false. Nclb was a great attempt to manage academia and it worked. http://www.nber.org/papers/w15531 http://www.nber.org/papers/w15531 http://www.nber.org/papers/w20511 http://www.nber.org/papers/w20511 http://www.nber.org/papers/w16745 http://www.nber.org/papers/w16745

HarryHirsch 10y ago
The authors themselves admit that NCLB was a mixed bag: "Specifically, we find that NCLB generated large and broad gains in the math achievement of 4th and (to a somewhat lesser extent) 8th graders. However, our results suggest that NCLB had no impact on reading achievement for 4th or 8th graders." You'd think that you can employ drill practice to improve results in mathematics, but that approach won't work in English. American math instruction feels quite alien for foreigners, the emphasis is on the application of algorithms, not on understanding why and how they work. Equipped with plenty drill practice, an Amarican child can fill in the correct bubble on the test, no matter if or not it understands mathematics. Any understanding of mathematics gained in that system is purely incidental. That's all a hypothesis and would require further testing.
- maxerickson 10y ago
  Drill practice is easy for English. Read. The hard part is motivating people to actually do it.
- yummyfajitas 10y ago
  That's net positive - an increase in some subjects and neutral in others.

csneeky 10y ago

All employers want their employees to "do the job to the best of their ability". But the freedom given to you to decide how your success is measured is proportional to your experience and past performance. There are those in both academia and private industry that get a lot of freedom to do their work/research. But there is a hierarchy to this just like everything in nature. The exceptionally strong and bright animals have more freedom in why they choose to do and how they do it. And that system has worked to get us all this far. The system this article is pushing against isn't fun for many but inevitable. The further you are from the top the more subject your performance is to poor evaluation schemes conceived by those far from the top too.

pdimitar 10y ago
Scientists aren't employees. They are given the tools, real estate and funding to do their thing, usually through research grants or private funding. The fact that nowadays they are treated as employees is exactly the problem. ...Oh, and by the way, nature has no hierarchy.
- csneeky 10y ago
  Scientists are employees in many contexts (almost all contexts). We may just have to agree to disagree on that one. Be they in service for an NGO, school, or oligarch. But to say nature has no hierarchy? I suppose "hierarchy" can mean many things. But in my case I mean fitness and selection for it. The most fit are at the top. Nature tends to select for those better suited to their environment. And those best suited thrive and push the success of us all forward. It is how you got here and have what you have. Intellectual, emotional, and physical traits are all subject to this selection. The thriving academics with the most freedom are closer to the top. The best employees (however you want to define employee) tend to be given the most freedom and trust.
  - pdimitar 10y ago
    If you look at any nature's habitat of N kinds of animals, even there what you see can't be called a hierarchy. It's a semi-stable state on the verge of chaos which can be very easily broken by introducing a species of animals new to the area. So no, it's not hierarchy. Lions can kill almost anything if they feel like it, but hyenas kill lions on a regular basis. Does that make them the top of the food chain in the savannah? Definitely not. It only makes a pack of hyenas stronger than a single lion or lioness. Rock, paper, scissors. No hierarchy. It's the same in the human systems. Hierarchy is an artificial system enforced by humans and has almost zero relation to anything natural.

baldfat 10y ago

I have a different take its the lack of pay for professors and the lack of respect for public education (On average a college professor makes less then a public school teacher, in my discipline I would have averaged about $20k less IF I got tenure) As a former academic professional (Librarian at a college) The high cost of college is based on the demands of students. 30 years ago there was no internet, no smoothy bar equipped fitness rooms, no pretty dorms and everything upscale. No pay increase for staff and I actually went through 3 years with no cost of living increases while the cost for college went up 15%. Then the idea that Public Education in the US is a failure (Totally false narrative except in the cities and other low income communities). So people believe that private investment and legislation will fix things. This just has people coming in making and taking the money from teachers and the community and taking them else where.

marcelluspye 10y ago
>30 years ago there was no internet, no smoothy bar equipped fitness rooms, no pretty dorms and everything upscale. How certain are you that these are the reasons school has gotten more expensive? I'm a student at a school with a decent amount of cushy stuff and a high tuition, and I can't say it looks like that's where the money's going (or even being sunk). Housing got cushy once university policy stopped forcing us to live on campus, at which point it had to compete with surrounding options (before that we were charged through the nose for pretty tight quarters). The school-provided food options are, ... well, the university's definitely making money on those, and if they're not the company they contract out to is. I gotta say, I don't think paying for gym equipment has significantly held up salary increases on the academic side of things. You might know better than I (and we're probably in different places), but looking around campus the things that seem to be sucking money away from paying the professors more is a) the ever-expanding number of administrative offices and b) the constant renovation/replacement/addition of buildings. I'll be the first to tell you my campus is an eyesore, but it feels more and more every year like they did that on purpose in the 70s so they wouldn't face much resistance in sinking a ton of money to replace it with a shinier one now. I'm also not totally unwilling to believe the standard economic argument that gets thrown out all the time: with so many more people leaving school with advanced degrees, finding people to fill teaching positions is a lot easier.

dasmoth 10y ago

“If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it. -- Attributed to J.J. Thomson (although I've not been able to turn up a definitive citation -- anyone know where it comes from?) Does make me think some people have been pondering this for a long while though.

pdimitar 10y ago
I don't know where the quote is from but it represents my only hope for the future of research. Of course, the hardcore bean-counters would not let that middle ground happen either; they are people in power who absolutely positively won't give you almost any leisure. It's still a good idea however. And some managers would embrace it since the notion of "paying somebody for possible benefits 5-10 years in the future" is something which our current form of capitalism hates. But some of them are reasonable people and would accept a middle ground. Another thing I was thinking about in the past is rotational work: given a team of 6 people, have 1 person always working on a research for at least a month with zero responsibility about the money-making activities. After that month -- or two, or six -- expire, return the person back to the capitalistic loop and put somebody else on the research duty. This however is rather clunky because people will have to be constantly caught up with where are the things currently in the money-making job and the research job. Perhaps there are other ideas as well, but I am not aware of them.
- quantum_state 10y ago
  In a university setting, it seems to me that teaching in combination with support from endowment (not research grants) can be one of the scenario for this ... key is still to have fundamentally good people with passion and joy for the endeavor ..

ZeroGravitas 10y ago

This isn't an academic issue, the same is true of almost every white collar profession. To apply standard numerical management techniques you either need to know exactly what a person needs to do as part of the process, or be able to evaluate the output, ideally both, and generally neither are true.

pmilot 10y ago

My wife works as a teacher at one of the largest schoolboards in Québec, and she can tell you that the solution "hire good people, pay them the going rate and tell them to do the job to the best of their ability" doesn't work in a unionized system. Complacency a systemic problem. When your union protects your job, you don't need to actually put effort into doing the job. Of course, that doesn't mean ALL teachers are lazy; but it does mean that a teacher who works hard and gets better results than average is not rewarded for it. It also means that bad teachers cannot be replaced by good ones. Of course, preventing public school teachers from unionizing will simply allow the government to exploit them. Perhaps the solution would be to replace unions with a professional order, similar to how engineers and lawyers are regulated.

wycx 10y ago
An infernal situation. There is no solidarity in a meritocracy, and no reward for merit with solidarity.
- trentnix 10y ago
  On the contrary, consider team sports as a perfect example of a meritocracy which often also promotes solidarity. A good team, led by a good coach, recognizes merit is earned by the performance of the whole. The whole wins and the whole loses together. The best of sport very clearly defines success and failure. It uses competition to spur innovation and the pursuit of excellence. Solidarity is often the consequence of that pursuit. Very little about education, from how teachers are managed to how schools are run, does the same.
  - geodel 10y ago
    For sports one very important difference is that real time performance is monitored by a large number of people. People who do not pay salaries of sports person and do not meddle with working of sports team directly. For any other kind of job performance is measured by insiders. Outsiders may gauge performance on quarterly / annual basis.

golemotron 10y ago

It would be great to see Nassim Nicholas Taleb's response to this. He's been working on a draft called Skin In The Game. Overall, I think that the GP is right but seeing that requires us to consider that nebulous ideas like character and cohesive culture need to be at the forefront more than market reasoning.

kev009 10y ago

I think this is somewhat well understood in things like Sports Psychology. You don't tell a receiver to catch X number of throws per game, you just encourage them perform. Likewise, training that more accurately simulates performance is more effective than working toward some metric. I learned it from Pat McNamara who has a special way with words and delivery: "When we are consumed by the outcome it will sabotage our ability to perform." - http://soldiersystems.net/2012/09/15/gunfighter-moment-pat-mcnamara/ http://soldiersystems.net/2012/09/15/gunfighter-moment-pat-m...

mseebach 10y ago
This works because both sides have a well-understood model of what 'performing' looks like. And if you don't, you'll be shown the door very quickly.
- hn_throwaway_99 10y ago
  I think this is key. In the sports arena, the metrics are VERY clear: if you don't win games, and your stats are poor, you're fired. I understand where academics are coming from, but at the same time it seems like many people want to be isolated from any objective assessment of their performance at all. FWIW, I see this a LOT among software developers as well.
CuriouslyC 10y ago
Current research indicates that the best way to promote performance is not to encourage someone's current performance, but instead encourage their development of skill mastery. In the short term, rewards (like money) produce the best results of all, but they also seem to impair learning.
mason240 10y ago
>You don't tell a receiver to catch X number of throws per game They kind of do though. In the NFL, many contracts are incentive laden with bonuses for (in the case of a wide receiver) X number of catches or Y number of yards.

pjmorris 10y ago

If the blog post seems anecdotal, there's a great, underused book, 'Measuring and Managing Performance in Organizations' by Robert Austin that covers his CMU PhD work, which came to the same conclusion as the quote attributed to Harford: "If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability." It doesn't hurt to make sure they don't have to worry about where to get, e.g., paper clips, lunch, or health care, so some level of administration is helpful.

kesor 10y ago

Back in 2010 there was an attempt to put Deliverology in place at California State University. It basically means that academia will be measured by targets. A horrible idea that made the California Faculty Association meet and decide on almost mutiny. They even invited John Seddon who wrote several books on the subject of how Deliverylogy twisted and perverted systems in the UK. Fortunately for us, the video of his presentation is available on YouTube at https://youtu.be/8EGMlCau6iU https://youtu.be/8EGMlCau6iU for part 1, and has about 5-6 parts. If you liked that, then I would highly recommend watching John Seddon's presentation about how settings targets and implementing broken IT systems to enforce those targets are a horrible idea. https://youtu.be/hbNsQFd8DQw https://youtu.be/hbNsQFd8DQw

kesor 10y ago
The full explanation of CFA Deliverology problem in 2010 is located on their page on the subject: http://www.calfac.org/resource/deliverology-etc-resources http://www.calfac.org/resource/deliverology-etc-resources

__roland__ 10y ago

I think one way to 'treat' this would be to acknowledge that certain things just cannot be optimized easily with outside incentives, and to simply accept that there will be some inevitable waste when you let 'good people do their job as best as they can'. It's ironic that we fret so much about professors that 'retire on the job', but at the same time think it is perfectly fine to steal time from the motivated researchers, by making them playing these silly games too. Who really thinks the next breakthrough will come from a person that was forced into writing another paper, just to pass some arbitrary performance threshold? If you consider research as a 'stochastic race', i.e. many people working on a problem with their own intuitions and approaches, it makes no sense to slow down those that are most likely to come up with a good solution (those that are self-motivated). IMHO, it does not really matter if those that are unlikely to contribute much (retired on the job) get a free pass. After all, even to receive tenure takes a lot of effort (and thus, ambition) nowadays, so at least in fields with a competitive job market (i.e. good salaries outside academia) it makes no sense to strive for a tenured position and then not do what you love (i.e., research). BTW Daniel Lemire's blog has many interesting posts on the more absurd aspects of academia, including research grants (e.g. http://lemire.me/blog/2009/09/15/the-hard-truth-about-research-grants/ http://lemire.me/blog/2009/09/15/the-hard-truth-about-resear...). IIRC one of his suggestions was so fund people (for longer terms), not 'projects'. This may not always be possible (e.g., if you need expensive equipment), but in many fields it would make much more sense. EDIT: language

Lxr 10y ago

If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability. Absolutely, could not agree more. This is the only way to beat gamification of metrics in my opinion. There is a worrying trend of treating staff as equally capable but untrusted cogs in a wheel, and metricising performance. I fundamentally believe that no amount of rules or metrics can turn a bad academic into a good one, an unsafe doctor into a safe one or a crooked politician into an honest public servant. We have to calm down with the metrics, accreditations and regulations, and accept that some people are better than others at certain things, hire the right people and trust them to do a good job.

pg_is_a_butt 10y ago
yeah, white men. you're all idiots.
merpnderp 10y ago
Seems like in IT the spread of different process systems is exactly this. To reduce each job to a set of functions that people with similar levels of training can do the job at the same level of output, interchangeably. So we're left with jobs that are 90% following inane process and 10% producing something useful. All to make sure if we quit, we can be replaced in a few days.
- LanceH 10y ago
  Where do you sit? Let's go to lunch today.
- colechristensen 10y ago
  Seems like if the tedious portion of the job reaches such a high proportion it's significant lack of automation.
- wpietri 10y ago
  The folks at Toyota talk about two different kinds of process: controlling and supportive. Controlling is that sort of BS. And I had seen it so much early in my career that I thought that's all there was. Supportive process is bottom up, done to improve consistency and reliability. E.g., by my door I have checklists for going running and swimming, because I was tired of going out to exercise and realizing I had forgotten something. I say this because I think process can be amazingly helpful if it's done bottom up, if the workers are the ones who design and improve it. But that only happens if managers have another thing that Toyota talks a lot about: "respect for people" [1]. That's what I think is really missing in these metrics-driven, people-are-cogs schemes. [1] https://www.lean.org/womack/DisplayObject.cfm?o=755 https://www.lean.org/womack/DisplayObject.cfm?o=755

kesor 10y ago

“Tell me how you measure me and I will tell you how I will behave. If you measure me in an illogical way… do not complain about illogical behavior…” - Dr. Eliyahu M. Goldratt

Fede_V 10y ago

That list is correct, but it's only half of the problem. The other other half of the problem is supply. In fields like mathematics, physics, computer science, grad students have excellent non-academic options. There is an oversupply of PhD students with respect to the number of available positions, but the excess PhD students have fantastic career avenues. In fields like the humanities, and, to a lesser extent, the molecular biology/ecology, there is a huge excess of PhD students, and there is a very limited number of career options outside of research (for bio, there is pharma, for everything else, eh). This is compounded by the fact that for professors, postdocs are an absolute steal (http://www.sciencemag.org/careers/2017/01/price-doing-postdoc http://www.sciencemag.org/careers/2017/01/price-doing-postdo...). Professors get to pay 50-55k (and that's on the high very high end) a year for a PhD trained researcher who will often work themselves to the bone in the hope of getting the publications that will allow them to become a professor. Thus - the very people who have the power to change the system instead greatly benefit from the status quo. The real reform that has to take place is that PhD programs need to become way more selective. Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis.

jessriedel 10y ago
> Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis. There are lots of industries like this, e.g., corporate law, sports, acting, music. Are they all "in crisis"? http://slatestarcodex.com/2015/07/28/non-dual-awareness/ http://slatestarcodex.com/2015/07/28/non-dual-awareness/ [EDIT] Others: military officers, investment banking, organized crime, tech start-up, trade guilds, etc.
- zepto 10y ago
  Yes
- anoonmoose 10y ago
  Law is in crisis, the other three are all so (relatively speaking) small, unimportant, and gated by the genetic lottery that they're not worth discussing.

maxforce 10y ago

The system is managed perfectly to minimise agendas being undermined. Those in power have worked their entire lives to gain it and will have no issue papering over any accusation that veracity is not their sole priority, let alone a complete farce. "In many scientific fields, results are often difficult to reproduce accurately, being obscured by noise, artefacts, and other extraneous data. That means that even if a scientist does falsify data, they can expect to get away with it – or at least claim innocence if their results conflict with others in the same field. There are no "scientific police" who are trained to fight scientific crimes; all investigations are made by experts in science but amateurs in dealing with criminals. It is relatively easy to cheat although difficult to know exactly how many scientists fabricate data" https://en.wikipedia.org/wiki/Scientific_misconduct https://en.wikipedia.org/wiki/Scientific_misconduct

dkural 10y ago

I had a similar problem with hiring product designers. How do you get & measure great design? What I ultimately decided was, the only way to get great design, is by hiring great designers, and only other great designers recognize who's very promising vs. who's not, with some degree of success. Research is the same. However, due to the inherent entanglement of historical biases with merit, it became very fraught in the U.S. to rely on expert judgment to pick new professors, every school clamoring to find some objective metrics to avoid charges of discrimination. This is coupled with the fact that most departments are filled with mediocre people who are not particularly qualified to pick promising young scientists to begin with. So ironically, discrimination is still there & at times worse, but now there is metricized cover for this bias. "We found that tall white dude X has more publications than Y". Meanwhile, academia right now is almost irrevocably filled with pedantic scientists, drip-feeding papers with maximal verbiage at a steady pace.

gustosoo 10y ago

I would suggest to teach and emphasize the role of science and technology for our future. Social esteem for the role of the scientist and technologist is important to nurture scientific talent. Reading Paul Drucker book about how America respect for the technological man and British respect for science man explains why America took the lead in technology. For young people motivation to solve some very important problem by means of science and technology and giving them freedom, good money and respect is the right combination to make the any field advance.

gustosoo 10y ago

Could we use deep learning to select who is going to do a good job in academia?

dredmorbius 10y ago
DL is a great way to reimplement extant bias whilst obscuring the reimplementation.

gustosoo 10y ago

I think than more than never working in a team is important. But a team must be prepared to accept A people what is not a small feat.

gustosoo 10y ago

There a a lot of comments here and the take-away for me is that managing academia is a very difficult task and nobody knows how to do it. Some people recognize that we could make more harm than good by adopting short-sight metrics, I consider fortunate that people recognize that sometimes we don't know all the answers and that the best we can do is to share information about this complex task.

gustosoo 10y ago

Since here many things are about start-ups, what about a start-up for developing a better academia, a way for great people to expand the frontiers of knowledge? Tim harford Ted talk about error and evolution is about how the only way to manage complex system is about trial and error, perhaps he would subscribe that trial and error is the only way to search for a better way of managing academia. http://www.ted.com/talks/tim_harford/transcript http://www.ted.com/talks/tim_harford/transcript

ThomPete 10y ago

I think we need to rethink what we consider academically important which should be taught at universities and then what should be taught as self studies. I can't help but thinking that one of the real problems is that a lot of academia isn't that valuable anymore as a field of study. Don't get me wrong I consider ex. philosophy as one of the most important fields but I also feel like it doesn't belong as an indiependent discipline anymore. Maybe I am wrong but after postmodernism I really don't see what new revelations are going to be of fundamental value to society. I see plenty of need for applying philosophy in various fields as interpretation of those fields such as neuroscience, quantum mechanics, technology etc. but as a stand alone field I think it gave us what we needed. And even if I am wrong you don't need an academic faculty to support it anymore. I feel the same with something like psychology which to me is even more problematic as it it's based on a premise that you can learn about the human mind by studying humans. Again it should be tied to something like Neuroscience instead. That way the fields are put to good use rather than being self servant. I am aware this might not be a popular opinion and it's not something that I am 100% certain of myself but it seems to be that a lot of academia could be learned without large institutions to support and and thus with the need to measure it the only way large institutions normally can. Thoughts?

trishume 10y ago
It's really easy to make punchy-sounding arguments about why a certain field isn't as important to teach, so we should be rather reluctant to go along with them. For example your argument about psychology could equally applied to almost any field and physics. "I feel the same way about (Computer Science|Biology|Chemistry) which to me is even more problematic as it is based on the premise that we can learn about (computers|life|chemicals) by studying (code|proteins|chemicals). Again it should be tied to something like quantum physics instead." In fact this argument makes no sense to me, why wouldn't studying human behaviour be the best way to learn about human behaviour? Same for the argument about philosophy, it's too easy to assume something valuable a field produces is too amazing to ever be topped. "After [the light bulb] I really don't see what new revelations [about electricity] are going to be of fundamental value to society."
- 10y ago

jccalhoun 10y ago

I'm currently a lecturer at a small midwestern liberal arts university. I don't know what kind of job I will have next year because the Dean decided that there will no longer be one year full time contracts and they won't add any more three year contracts. Theoretically they want to have more tenure track job but in practice our department isn't getting any more. So at worst I will be part time next year which means I won't have insurance. Best case one of the job applications I have pans out and I get a tenure track job. The reason for the change is, of course, money. It is cheaper to hire 5 part time people to teach the load of 3 full time people because the part timers get less pay and no benefits. Of course finding 2 extra people who are qualified doesn't seem to be something they are worried about... Strangely, they haven't suggested switching to part-time administrators...

flamedoge 10y ago
of course that would be outrageous! we should be giving them millions in bonus!

uaaa 10y ago

Management and other myths 1. Complex systems tend to produce complex responses (not solutions) to problems. 2. Great advances are not produced by systems designed to produce great advances. https://en.wikipedia.org/wiki/Systemantics https://en.wikipedia.org/wiki/Systemantics

dredmorbius 10y ago
"The chief cause of problems is solutions." -- Sevaried's Law
dredmorbius 10y ago
Also, thank you so very much for Systemantics. I've run across it before, but very much needed to run across it here and now.

jknoepfler 10y ago

My take on the desire to regulate/manage is that some people would rather have an economy of perverse incentives than tolerate the inevitable abuses caused by lack of accountability. Management in the programming world feels the same way. At first, to many of us, this seems foolish and counter-productive, but then think: how does a university policy maker or a manager explain the rotten eggs (low output academics, unproductive developers) to their "boss," which might be the tax paying public or a budget strapped firm? How does a competitive university distinguish itself in a communicable/verifiable way? Could I be fired, or in the private sector sued for letting smart people fuck up unsupervised? It is difficult to trust others when the worst case outcome exposes us to catastrophic consequences. I personally agree with the author, but I think there's serious work that could be done to mitigate or remove the incentives to manage, and I don't think those are as thin as "tell people to stop trying to control what they can't"

cr0sh 10y ago

This article, and the follow-up on it (link is in the article) is about something very important, and should be more widely known about and discussed by anyone with an interest in science, research, and funding for it all. It points (yet again - as we've all seen the various p-hacking commentary) to the very real problems in scientific research that seemingly threaten to undermine it - especially from the public's point of view. It has been said that "religion poisons everything" - there is probably a corollary of "politics poisons everything" as well; both of these are really just social control and exhibition proxies. Ultimately, human nature rears its head, and the want for more money, power, and prestige (at all levels) leads to these results. ...and society becomes poorer for it. The questions and the proposed ideas aiming at solutions to the problems discussed sound plausible (or workable) on the surface, but I tend to wonder if any and all solutions will just be able to be gamed anyhow? At any rate, I'm glad that this was posted, though I despair at it leading to any solution, as it seems the problems lie within our psychology and society, and are thus nearly intractable.

jeremyt 10y ago

In the most non-trolling way that I can manage, why can this not be generalized to the economy? Why is academia a special case scenario where bureaucracy and regulations create unintended consequences and cause people to expend effort to get around them, but we can't say the same thing about the economy as a whole? Why do we believe that people can't even manage PhD's at a university, but we can take the same people, put them in government, and expect them to, for example, manage the monetary supply or see bubbles in advance and create proper policy to avoid them?

MooMooMilkParty 10y ago
I think one of the hard things about academia (by ways of "pure research"), is measuring impacts. In some positions there are metrics which are more representative of real progress than others. My belief is that measuring an academic's "impact" is a very fuzzy thing in the long term, especially when you bring in people who don't specialize in the same field. That's a reason this can't be generalized fully, though you can probably find corollaries. But, your last sentence to me, seems like a good one. Presumably, that would be an excellent topic to pursue research in, considering we clearly don't have a working theory on it (but that's not my specialty so who knows, maybe there's a bunch of smart people being ignored about it). Thus, it's hard to measure progress in, and we institute perverse incentives per the article.

otempomores 10y ago

You cant put a metric on the future.. In the literal sense..you cant measure whatyou yet dont know..or how valueable it will be for humanity.

bluefox 10y ago

I'll just note that this entry was posted 8 times by different accounts. Someone wants you to adopt an opinion.