7 ms·

The root of the problem is that we allow people with high esteemed credentials to run society. They jumped through a hoop. "You got straight As at 14 - 18 year

The root of the problem is that we allow people with high esteemed credentials to run society. They jumped through a hoop. "You got straight As at 14 - 18 years old and got into an ivy league school as a result? Here run this venture fund." "You got a PhD in Economics with a good publication by P-hacking your secret data? Here take a run at the FED with power over the US Economy. Your Phd shows that you are the man for the job." "You got a PhD in ML by making some incremental improvement on some already existing model and then doing massive hyper parameter tuning? Here, become director of research at this big corporation." Research will never be fully productive in this system, there are too many people who have too much to gain from gaming the publication system.

Something1234 6y ago
Thanks for pointing out the problem with credentials. Name a better indicator then for being competent.
- ktaylora 6y ago
  Industry kinda already has. Products launched. Teams managed. A candidate's publication record is a conversation starter during interviews, but I feel like years of experience at a company working on launching a successful project is a much more valuable currency.
  - saagarjha 6y ago
    For getting a job, maybe…industry isn't everything.
    - nomel 6y ago
      I think his point is that practical experience and a track record of proven, successful, outcomes are more useful for predicting future outcome.
  - throwaway2048 6y ago
    These metrics can just as easily be gamed, see how all the glory at Google goes towards launching exciting projects, that then immediately get put into stasis, then canceled.
  - chrisseaton 6y ago
    A good PhD is launching a successful project.
  - rumanator 6y ago
    > Industry kinda already has. Products launched. This suggestion is so detached from both the problem and the very nature of academia that it's straight-out laughable. I mean, what do product launches have to do building knowledge on the state of the art, identifying a novel idea, doing the iterative work to refine the idea, and finally document it to the public? Product launches at best require you to manage people and expectations. Do you honestly believe that a guy who launched a product is more qualified to drive science forward than a PhD with an outstanding academic track record just because your PM had a knack to cut corners, descope requirements, and pass the buck to underlings? Because that's the bulk of the job of all PMs I ever met, including from FANGs. Your suggestion is the poster child of the old mantra "if the only tool you have is a hammer you tend to see every problem as a nail".
    - nostrebored 6y ago
      The private sector does do research, and for RoI /it actually has to work/. Academia is a cesspit of politics and dishonesty. Forming the most effective cartels, sensationalizing your results, and being well networked with other academics is not aligned with getting results on actual problems.
      - rumanator 6y ago
        > The private sector does do research, and for RoI /it actually has to work/. The private sector does research with researchers, not product managers. The process is exactly the same. It's not a public vs private thing. It's a research vs production thing. Research is open-ended and iterative and exploratory. Product design is close-ended, focused, and with hard requirements. Research has zero to do with product management, and you don't change the nature of the problem by pretending that a scientific discovery is a product expecting to be launched. > Academia is a cesspit of politics and dishonesty. Oh awesome. Have you ever did any corporate work? Because if you believe that academia suffers from this problem but corporations don't then I have a few bridges I'd like to sell you. At least in academia you do need to have your publications to back you up. In corporate environments all you have is the cesspool part.
  - jariel 6y ago
    " Products launched. Teams managed." This definitely has zero to do with quality research being published, and even in startup land, those things are nice to have but still don't prove anything, unless they were the directing agents of those initiatives.
  - jjoonathan 6y ago
    Those are even easier to hack. Here's the result of measuring by projects launched: https://killedbygoogle.com/ https://killedbygoogle.com/
- deleted 6y ago
  [deleted]
- SherlockeHolmes 6y ago
  Allow me to offer some qualities other than competency that are necessary for responsible research. 1. Purpose. what is it that drives one? 2. Integrity. when does one compromise it? 3. Awareness. is one able to disengage when one's ego (fueled by credentials) is activated.
- TrackerFF 6y ago
  For most, credentialism works fine in academia. The problem is that you're incentivized to cheat, because it's either that or your career. Academia suffers from incredible power asymmetry, and an obsession with prestige. It's shit culture that rots from the head and down. Of course, the vast majority of research groups are not like that - but it seems to be a problem which has always been there, but just been swept under the rug. Again, it's difficult to find informants. Groups are small, and most advisors have very few people under their wing. It is probably very easy to identify and black-ball whistleblowers. It gets more difficult, the more you're invested in your work/degree.
sjg007 6y ago
Publications really only look impressive to outsiders and gra d students.
- wjn0 6y ago
  Surely you mean the number of publications, or some other superficial metric (e.g. being the third "supervising" author on a paper with a dozen authors). If I read a paper and have a great appreciation for the ideas, and have a sense that an author contributed significantly to the aspects I appreciate (either from explicit descriptions of the authors' contributions in the publication, or by speaking to them), why would that not be impressive to me?
- rumanator 6y ago
  I beg to differ. Publications are impressive because they are, in the very least, an indicator of the volume of work that a research group does in a domain. Writing a paper requires time and effort and, more importantly, focused work. As anyone who works and ever worked on academia is able to tell you, finding time to do anything of value, whether is exploring a new idea or continue working on any of the ideas you might have floating around, is the biggest challenge.
- killjoywashere 6y ago
  I work with a group that got a pretty ground-breaking tech paper published in a top medical journal. They estimated it generated $100M in investment for the group. Over 2-3 years. I think the original investment for the work that produced the paper was about $1M. FWIW.
throwawayiionqz 6y ago
And the cherry-on-top: Once you are part of a research unit at big corporation, your annual bonus depends on your number of publications. Or why the pursuit of incremental but well marketed results never ends. Or why papers with 6+ authors among which only 1-2 substantially contributed is the new normal.
opportune 6y ago
All three of those examples are missing a step of like 5-20 years in the middle. Nobody is running a venture fund as a 22 year old ivy grad (barring someone who's taking over for their family member, but in that case it has nothing to do with being an ivy grad), nobody is running the FED right out of a PhD, nobody is a director at a big corporation right out of a PhD. Presumably once those people are in such high-power positions, they also have a track record of real accomplishments behind them; it's certainly possible they've lied and cheated their entire career but it's definitely less likely they'll make it that far that way.
- bsder 6y ago
  > it's certainly possible they've lied and cheated their entire career but it's definitely less likely they'll make it that far that way. The number of people who make "correct, active decisions" is vanishingly small. It's less they've cheated than "Did you really make correct decisions or did your coin just come up heads 8 times in a row?" Other people may have been as smart or smarter, but if the coin flip went tails, they get politically hammered. Success has a large part of "survivor bias" to it.
  - opportune 6y ago
    See sibling comment. I agree there is a lot of survivorship bias but the reality is more like the first coin flip has a 25% chance and all the subsequent ones are closer to like 80-90% because there are a lot of positive feedback mechanisms you can take advantage of to maximize early career advantages
- logicslave 6y ago
  Part of the point is that the gap in the middle often doesn't matter. I've hung around the Stanford crowd and as I enter my thirties I am awestruck by the opportunities they have regardless of what they have done since graduation. The credential is almost all that matters, as long as they dont mess up massively. A common path is: Graduate Stanford => Work at Mckinsey => Get hired into VC. Graduate Stanford => Raise 15 Million dollar series A with your buddy => Doesn't matter what happens you will end up rich. It's not a cynical opinion, I have seen it myself and I am doing very well for myself. I worked under an Economics PhD who was the number one in his class (top 5 phd program) and graduated top of his undergrad at UPenn. His incompetence relative to his credentials shattered my respect for the way that we allocate positions of power
cosmodisk 6y ago
Can't agree more. And it's getting worse. In certain fields people are almost untouchable. Couldn't count anymore the number of discussions I had about doctors and how absolutely incompetent some of them are: the usual reaction is like 'No, you don't say! But the selection process ensures only the smartest get selected,so what are you on about?'. The second one is PhDs, and in general all sorts of professors almost elevated to god like level. You almost get a free pass to spread whatever shit you want if the credentials are right. The corporate world is exactly the same. Since my last promotion,I now have a nice title. People now listen to what I have to say. They even take me seriously. I can now go to some guy who's got 20 times more experience and start selling my consulting services(I'm not a consultant).
- logicchains 6y ago
  >Couldn't count anymore the number of discussions I had about doctors and how absolutely incompetent some of them are: the usual reaction is like 'No, you don't say! But the selection process ensures only the smartest get selected,so what are you on about?'. The second one is PhDs, and in general all sorts of professors almost elevated to god like level. You almost get a free pass to spread whatever shit you want if the credentials are right. It's like the old joke: what do you call the worst-performing graduate from medical school (or a PhD program), the one who was at the bottom of their class? You call them "Doctor"!
alexashka 6y ago
I'll trade my credential for a few million any day of the week. Never have any takers, I wonder why.
- weehoo 6y ago
  Credentials are non transferable and non-voidable unless you are later discovered having cheated. I don’t get what point you’re trying to make here.
  - alexashka 6y ago
    > The root of the problem is that we allow people with high esteemed credentials to run society. If high credentials run society, surely somebody would offer me a bribe of a few million dollars by now, in order to make use of my 'society running' abilities.
CannoloBlahnik 6y ago
This is absolutely not the reality of a PhD. On what planet do people with those backgrounds get those positions out of their PhDs? What's the basis for this entire comment?
- logicslave 6y ago
  The basis is spending 15 years in NYC high finance and having multiple close groups of friends who graduated from Stanford, work in tech, raised VC, etc
  - shuckles 6y ago
    You have 15 years of finance experience as you “enter your thirties”?
    - logicslave 6y ago
      You got me, poor wording. My father worked at a very high level in influential investment funds, I grew up around the industry and understand how it works. I will delete this later because its not something I want to advertise
      - woofie11 6y ago
        Isn't there a time limit on deleting / editing comments?
      - shuckles 6y ago
        It sounds like you have a background and exposure to people who might have many enabling success factors. Shiny credentials might be one but social networks are another. Collectively, they might be contributing to the success you suggest is not deserved, but I’m not sure that’s an obvious argument. People of these backgrounds could probably game any gating ritual. Perhaps we should focus on drawing more actual talent rather than excluding the phonies (even at the highest levels, status and influence isn’t zero sum in my experience).
analog31 6y ago
You forgot a step: "Your parents are rich, so you have a chance."
deleted 6y ago
[deleted]

Something1234 6y ago

Thanks for pointing out the problem with credentials. Name a better indicator then for being competent.

ktaylora 6y ago
Industry kinda already has. Products launched. Teams managed. A candidate's publication record is a conversation starter during interviews, but I feel like years of experience at a company working on launching a successful project is a much more valuable currency.
- saagarjha 6y ago
  For getting a job, maybe…industry isn't everything.
  - nomel 6y ago
    I think his point is that practical experience and a track record of proven, successful, outcomes are more useful for predicting future outcome.
- throwaway2048 6y ago
  These metrics can just as easily be gamed, see how all the glory at Google goes towards launching exciting projects, that then immediately get put into stasis, then canceled.
- chrisseaton 6y ago
  A good PhD is launching a successful project.
- rumanator 6y ago
  > Industry kinda already has. Products launched. This suggestion is so detached from both the problem and the very nature of academia that it's straight-out laughable. I mean, what do product launches have to do building knowledge on the state of the art, identifying a novel idea, doing the iterative work to refine the idea, and finally document it to the public? Product launches at best require you to manage people and expectations. Do you honestly believe that a guy who launched a product is more qualified to drive science forward than a PhD with an outstanding academic track record just because your PM had a knack to cut corners, descope requirements, and pass the buck to underlings? Because that's the bulk of the job of all PMs I ever met, including from FANGs. Your suggestion is the poster child of the old mantra "if the only tool you have is a hammer you tend to see every problem as a nail".
  - nostrebored 6y ago
    The private sector does do research, and for RoI /it actually has to work/. Academia is a cesspit of politics and dishonesty. Forming the most effective cartels, sensationalizing your results, and being well networked with other academics is not aligned with getting results on actual problems.
    - rumanator 6y ago
      > The private sector does do research, and for RoI /it actually has to work/. The private sector does research with researchers, not product managers. The process is exactly the same. It's not a public vs private thing. It's a research vs production thing. Research is open-ended and iterative and exploratory. Product design is close-ended, focused, and with hard requirements. Research has zero to do with product management, and you don't change the nature of the problem by pretending that a scientific discovery is a product expecting to be launched. > Academia is a cesspit of politics and dishonesty. Oh awesome. Have you ever did any corporate work? Because if you believe that academia suffers from this problem but corporations don't then I have a few bridges I'd like to sell you. At least in academia you do need to have your publications to back you up. In corporate environments all you have is the cesspool part.
- jariel 6y ago
  " Products launched. Teams managed." This definitely has zero to do with quality research being published, and even in startup land, those things are nice to have but still don't prove anything, unless they were the directing agents of those initiatives.
- jjoonathan 6y ago
  Those are even easier to hack. Here's the result of measuring by projects launched: https://killedbygoogle.com/ https://killedbygoogle.com/
deleted 6y ago
[deleted]
SherlockeHolmes 6y ago
Allow me to offer some qualities other than competency that are necessary for responsible research. 1. Purpose. what is it that drives one? 2. Integrity. when does one compromise it? 3. Awareness. is one able to disengage when one's ego (fueled by credentials) is activated.
TrackerFF 6y ago
For most, credentialism works fine in academia. The problem is that you're incentivized to cheat, because it's either that or your career. Academia suffers from incredible power asymmetry, and an obsession with prestige. It's shit culture that rots from the head and down. Of course, the vast majority of research groups are not like that - but it seems to be a problem which has always been there, but just been swept under the rug. Again, it's difficult to find informants. Groups are small, and most advisors have very few people under their wing. It is probably very easy to identify and black-ball whistleblowers. It gets more difficult, the more you're invested in your work/degree.

sjg007 6y ago

Publications really only look impressive to outsiders and gra d students.

wjn0 6y ago
Surely you mean the number of publications, or some other superficial metric (e.g. being the third "supervising" author on a paper with a dozen authors). If I read a paper and have a great appreciation for the ideas, and have a sense that an author contributed significantly to the aspects I appreciate (either from explicit descriptions of the authors' contributions in the publication, or by speaking to them), why would that not be impressive to me?
rumanator 6y ago
I beg to differ. Publications are impressive because they are, in the very least, an indicator of the volume of work that a research group does in a domain. Writing a paper requires time and effort and, more importantly, focused work. As anyone who works and ever worked on academia is able to tell you, finding time to do anything of value, whether is exploring a new idea or continue working on any of the ideas you might have floating around, is the biggest challenge.
killjoywashere 6y ago
I work with a group that got a pretty ground-breaking tech paper published in a top medical journal. They estimated it generated $100M in investment for the group. Over 2-3 years. I think the original investment for the work that produced the paper was about $1M. FWIW.

throwawayiionqz 6y ago

And the cherry-on-top: Once you are part of a research unit at big corporation, your annual bonus depends on your number of publications. Or why the pursuit of incremental but well marketed results never ends. Or why papers with 6+ authors among which only 1-2 substantially contributed is the new normal.

opportune 6y ago

All three of those examples are missing a step of like 5-20 years in the middle. Nobody is running a venture fund as a 22 year old ivy grad (barring someone who's taking over for their family member, but in that case it has nothing to do with being an ivy grad), nobody is running the FED right out of a PhD, nobody is a director at a big corporation right out of a PhD. Presumably once those people are in such high-power positions, they also have a track record of real accomplishments behind them; it's certainly possible they've lied and cheated their entire career but it's definitely less likely they'll make it that far that way.

bsder 6y ago
> it's certainly possible they've lied and cheated their entire career but it's definitely less likely they'll make it that far that way. The number of people who make "correct, active decisions" is vanishingly small. It's less they've cheated than "Did you really make correct decisions or did your coin just come up heads 8 times in a row?" Other people may have been as smart or smarter, but if the coin flip went tails, they get politically hammered. Success has a large part of "survivor bias" to it.
- opportune 6y ago
  See sibling comment. I agree there is a lot of survivorship bias but the reality is more like the first coin flip has a 25% chance and all the subsequent ones are closer to like 80-90% because there are a lot of positive feedback mechanisms you can take advantage of to maximize early career advantages
logicslave 6y ago
Part of the point is that the gap in the middle often doesn't matter. I've hung around the Stanford crowd and as I enter my thirties I am awestruck by the opportunities they have regardless of what they have done since graduation. The credential is almost all that matters, as long as they dont mess up massively. A common path is: Graduate Stanford => Work at Mckinsey => Get hired into VC. Graduate Stanford => Raise 15 Million dollar series A with your buddy => Doesn't matter what happens you will end up rich. It's not a cynical opinion, I have seen it myself and I am doing very well for myself. I worked under an Economics PhD who was the number one in his class (top 5 phd program) and graduated top of his undergrad at UPenn. His incompetence relative to his credentials shattered my respect for the way that we allocate positions of power

cosmodisk 6y ago

Can't agree more. And it's getting worse. In certain fields people are almost untouchable. Couldn't count anymore the number of discussions I had about doctors and how absolutely incompetent some of them are: the usual reaction is like 'No, you don't say! But the selection process ensures only the smartest get selected,so what are you on about?'. The second one is PhDs, and in general all sorts of professors almost elevated to god like level. You almost get a free pass to spread whatever shit you want if the credentials are right. The corporate world is exactly the same. Since my last promotion,I now have a nice title. People now listen to what I have to say. They even take me seriously. I can now go to some guy who's got 20 times more experience and start selling my consulting services(I'm not a consultant).

logicchains 6y ago
>Couldn't count anymore the number of discussions I had about doctors and how absolutely incompetent some of them are: the usual reaction is like 'No, you don't say! But the selection process ensures only the smartest get selected,so what are you on about?'. The second one is PhDs, and in general all sorts of professors almost elevated to god like level. You almost get a free pass to spread whatever shit you want if the credentials are right. It's like the old joke: what do you call the worst-performing graduate from medical school (or a PhD program), the one who was at the bottom of their class? You call them "Doctor"!

alexashka 6y ago

I'll trade my credential for a few million any day of the week. Never have any takers, I wonder why.

weehoo 6y ago
Credentials are non transferable and non-voidable unless you are later discovered having cheated. I don’t get what point you’re trying to make here.
- alexashka 6y ago
  > The root of the problem is that we allow people with high esteemed credentials to run society. If high credentials run society, surely somebody would offer me a bribe of a few million dollars by now, in order to make use of my 'society running' abilities.

CannoloBlahnik 6y ago

This is absolutely not the reality of a PhD. On what planet do people with those backgrounds get those positions out of their PhDs? What's the basis for this entire comment?

logicslave 6y ago
The basis is spending 15 years in NYC high finance and having multiple close groups of friends who graduated from Stanford, work in tech, raised VC, etc
- shuckles 6y ago
  You have 15 years of finance experience as you “enter your thirties”?
  - logicslave 6y ago
    You got me, poor wording. My father worked at a very high level in influential investment funds, I grew up around the industry and understand how it works. I will delete this later because its not something I want to advertise
    - woofie11 6y ago
      Isn't there a time limit on deleting / editing comments?
    - shuckles 6y ago
      It sounds like you have a background and exposure to people who might have many enabling success factors. Shiny credentials might be one but social networks are another. Collectively, they might be contributing to the success you suggest is not deserved, but I’m not sure that’s an obvious argument. People of these backgrounds could probably game any gating ritual. Perhaps we should focus on drawing more actual talent rather than excluding the phonies (even at the highest levels, status and influence isn’t zero sum in my experience).

analog31 6y ago

You forgot a step: "Your parents are rich, so you have a chance."

deleted 6y ago

[deleted]