8 ms·

I will just offer some unsolicited advice about PhDs, as it came up a few weeks ago also. And the author touches on this. Of course this mostly applies to em

I will just offer some unsolicited advice about PhDs, as it came up a few weeks ago also. And the author touches on this. Of course this mostly applies to empirical sciences, but maybe some theoretical ones too. An effective PhD advisor/thesis isn't wandering the woods to find something. It's a guided coaching exercise, with an outcome in mind. Test whether the PhD advisor you select knows this and has a history of taking this approach with former students. An advisor (and the PhD research he/she guides you on) should be targeting a very conscious choice of incremental versus breakthrough research results. And for most students, you're not going to have a whole lot of mindblowingly field-changing results -- as much as you may think you're a rockstar. So it's just sensible or an insurance policy to make sure you've got solid steady work that you're progressing on. Learn to enable your research to have visible incremental gains on a known path every day, rather than hoping for some breakthrough at the end. Amazing breakthroughs have high risk, and make it highly likely you'll have a crisis when it doesn't happen. Concretely, even if you don't know what the answer is going to be at the end of your research, you must think about, or have an idea about, the format of what that amazing answer is going to be. Write the outline of your thesis and "ghost out" what the major charts will be. Write the intro sentences of each chapter -- what are they? (and I don't just mean the boring review of the field part, but your findings part) You should know what major type of finding, plot, or table your research is going to output. What are the columns and rows of that table, or axes of that plot? How many data points are required? How many of them can already be guessed? Where is the surprise going to be? What is the conclusion going to be? Draw out the answer you're aiming for, now. If you can't even articuate what the answer will look like, you may be in for a bad time, so work on fixing that. It will also push you and your advisor to be specific about what the output of your thesis will be -- and set you up for a much better PhD experience.

legerdemain 6y ago
> Test whether the PhD advisor you select knows this and has a history of taking this approach with former students. How do you "test" for this? Asking potential advisors "do you know this?" is blunt and rude. Asking potential advisors in a mannered way prompts anodyne answers. Current students in the potential advisor's lab are unlikely to offer an intimate assessment of their personal satisfaction and research progress to a stranger, assuming they can even be honest with themselves. My take is that having a good graduate advisor is largely about having a good undergraduate advisor who can offer an insider take and advocate on your behalf.
- eegilbert 6y ago
  This is my syllabus. :-) I tell all my prospective students to talk to my former students.
  - legerdemain 6y ago
    Yes, and it is an enormously appreciated breath of fresh air. But you're not every PhD advisor!
    - supernova87a 6y ago
      I would say that as a student, you would be asking a potential advisor what mix of projects would be an example course of the research. What is "their approach" to the sequence of your grad projects over several years? Examples? What combination of risk/certainty does it have? I think you can tell pretty quickly whether the advisor has this concept even in mind. Former students should have this idea in their mind too, after finishing the program. If they don't, they haven't learned one of the important things about a PhD... And if you feel that it's too rude to ask, or the professor can't handle such a question (or you're hesitant to ask) -- either you probably want to find someone who is comfortable with such questions, or better find out why you are uncomfortable. Better to be awkward up front and get the answers you want, than be polite and suffer for 5 years to discover the wrong answer. I mean, that's a whole nother topic -- this is a huge investment of years of your life. You should ask difficult questions before you make the leap. And the professor should owe you such answers before committing to taking you on. In fact, it's not that different from an awkward pre-nup contract conversation. Only this time you don't have to be swayed by emotion.
      - legerdemain 6y ago
        Well, my story was 15 years ago and in an area where students were mostly expected to bring their own research questions to the program. I had trouble getting the time of day from potential advisors, to say nothing of interviewing them about potential projects and their advising methodology!
n3ur0n 6y ago
As a rising 5th year PhD in ML -- I could not agree with this advice more! I have very hands-off advisors. I spent the first 3 years "wandering the woods to find something". Last year, I really had to sit down and think about how I can finish up my PhD on time. I pretty much did what you outline here. A lot of tools I used were organization tools I learned from my business school/product manager friends. I honestly think the PhD system needs to be overhauled. even at a "top-tier" program like mine, it is amazing to me that at no point do we receive any training regarding practical components of research. 100 years ago, the way you became a physician was to follow around a physician and one day you were ready to be a physician yourself. In the year 2020, this is how PhDs are trained. I do realize that a PhD is not a "professional" degree like MD or JD, however, given that most PhD grads will 1) go to industry 2) go into academia, we need to teach students about project management, planning etc.
- ta1234567890 6y ago
  > we need to teach students about project management, planning etc. This is a problem with education in general. Those things should be taught in middle/high school as essential tools for modern life. Same as personal finances, effective strategies for team/group work, overcoming social anxiety, managing stress and building/maintaining relationships (personal and professional).
  - quineoa 6y ago
    Couldn’t agree more. Instead of learning those things in school. I got lucky and had privilege and learned some of them outside of school, and still need to learn others. But instead, they made sure I knew the name of the boats Columbus sailed in, in 1492. The Niña the Pinta, and the Santa Maria. I was forced to submit to whatever story they told me about the relationship between Columbus and the natives, no matter how untrue it was. But maybe that was the real lesson....
    - hinkley 6y ago
      One of the reasons I recommend volunteer work to people who graduated or are about to graduate with nothing like a plan for getting a job is that this is how I learned a lot of organizational skills (and got my first references). It also makes that, “what have you been doing for the last five months?” question a lot less uncomfortable to answer.
bonoboTP 6y ago
> Concretely, even if you don't know what the answer is going to be at the end of your research, you must think about, or have an idea about, the format of what that amazing answer is going to be. Write the outline of your thesis and "ghost out" what the major charts will be. Write the intro sentences of each chapter -- what are they? (and I don't just mean the boring review of the field part, but your findings part) > You should know what major type of finding, plot, or table your research is going to output. What are the columns and rows of that table, or axes of that plot? How many data points are required? How many of them can already be guessed? Where is the surprise going to be? What is the conclusion going to be? You are absolutely right and it's depressing. The only practical, predictable and steady way to a PhD is to come up with the storyline in advance, do the experiments, bend the interpretation and analysis to support the story you already wanted in advance, then pretend the finding is surprising and novel (when actually you knew you could safely make it work in advance). There is a tension here, on the one hand PhDs have to pretend play that they discovered major novel stuff and innovated a lot (which is only possible through immense risk), on the other hand in reality they must finish in a given number of years and must be very risk averse. Only a major cultural shift towards handing out PhD titles for meticulous, systematic, risky but unfruitful research may help. Forcing people to find flashy new stuff will lead to tons of overinflated findings that are more about storytelling than actual surprising novel things that bring a field forwards. It's all a huge pretend play. ---- Ideally in an alternate universe of different incentive structures that support real science, a person would work in research, use existing approaches in different scenarios, push and prod them in different use cases then perhaps once every year you'd stumble upon some interesting effect you don't understand. Then you'd dive in and prod this specific effect with well designed experiments. Once it's clearer what is going on, you come up with the best way to convey this newly earned knowledge. It is at this point (once you actually know what you want to alert the research community to) that you decide how to best present the knowledge, what tables, sections and paragraphs will succinctly and ergonomically convey the information. If you knew all the story and layout beforehand and you know you will be able to just plug in the numbers and make the paper stand, then you are not doing real science, you are playing the academic game, a cargo cult of science, a game stealing and consuming the prestige of science. Yes, as I said this only works in the alternate universe. Here in this universe we must churn out papers and must overinflate our findings. Of course, we need to eat, we need jobs, we need titles etc. It has driven so many PhD students I know to bitterness and cynicism. Several have given up and went to industry. Others, after the disillusionment, have cynically pushed through with whatever minimal things they knew would be required (plus the right story and presentation) then bailed to consulting. You also need to bullshit in consulting but are at least compensated for it well enough. In the rarest cases do people come out of it as a positive passionate person. Even the successful ones become a hedging, scheming, lukewarm, slimy academic personality. Everything encourages this. The way you get professor positions, the way you get grant money etc. It optimizes and chisels people into becoming the antithesis of curious, open scientific inquiry.
xab31 6y ago
> An effective PhD advisor/thesis isn't wandering the woods to find something. It's a guided coaching exercise, with an outcome in mind. There is a tradeoff here. One of Eric's selling points for his lab is obviously that students get a lot of intellectual freedom to develop their own ideas. This is the "wandering the woods" approach, which was my advisor's as well. The advantage is the freedom, as well as the responsibility and intellectual independence that comes from exercising it. The disadvantage is that it can lead to a lot of wasted time and dead ends. The "guided coaching exercise", when taken to the opposite extreme, results in an advisor that hands a project with clearly defined goals and outcomes to the student. That kind of approach usually, in my experience, leads to more and higher-impact papers. It does not develop the student very well, but it does develop their CV, which is quite important, and once that student becomes a postdoc, it is do-or-die time, and every CV item helps. But if we just view this advice as "how should someone doing a project conceptualize their project", this is good advice. Although you would be astonished how many PIs don't follow it. I've had PIs want to write papers before seeing a single figure. I've had countless collaborators send me data to analyze without any clear idea what sort of results/analysis they want, without a hypothesis even.
- Beldin 6y ago
  The job of a PhD advisor is to guide the student to become an independent researcher. That is: someone able to conceive of research-worthy ideas, separate the good from the bad, execute the research and write it all up in a paper. I have seen very hands-off approaches and very structured approaches. Both have their merits and pitfalls: the first may leave a student stranded in the proverbial forest, while the second may turn into nothing more than a 4 year long series of related tasks to be executed, set at day one, with no input from the student. On the other hand, I've also seen both approaches work well. Basically, you want a daily supervisor that cares, and with whom you have a click.
shrimpx 6y ago
My impression is that things are drastically different if you’re going for a top professorship vs something else, like a job at google. If you’re going for a professorship then basically nothing matters until you strike on an idea that hits it out of the park in some way. And it’s a tricky combination of technical and political fitness. Depending on your work, some schools will like you more than others. If you’re just looking for a job in some research lab then a basic “get it done” process on an incremental result works fine.

legerdemain 6y ago

> Test whether the PhD advisor you select knows this and has a history of taking this approach with former students. How do you "test" for this? Asking potential advisors "do you know this?" is blunt and rude. Asking potential advisors in a mannered way prompts anodyne answers. Current students in the potential advisor's lab are unlikely to offer an intimate assessment of their personal satisfaction and research progress to a stranger, assuming they can even be honest with themselves. My take is that having a good graduate advisor is largely about having a good undergraduate advisor who can offer an insider take and advocate on your behalf.

eegilbert 6y ago
This is my syllabus. :-) I tell all my prospective students to talk to my former students.
- legerdemain 6y ago
  Yes, and it is an enormously appreciated breath of fresh air. But you're not every PhD advisor!
  - supernova87a 6y ago
    I would say that as a student, you would be asking a potential advisor what mix of projects would be an example course of the research. What is "their approach" to the sequence of your grad projects over several years? Examples? What combination of risk/certainty does it have? I think you can tell pretty quickly whether the advisor has this concept even in mind. Former students should have this idea in their mind too, after finishing the program. If they don't, they haven't learned one of the important things about a PhD... And if you feel that it's too rude to ask, or the professor can't handle such a question (or you're hesitant to ask) -- either you probably want to find someone who is comfortable with such questions, or better find out why you are uncomfortable. Better to be awkward up front and get the answers you want, than be polite and suffer for 5 years to discover the wrong answer. I mean, that's a whole nother topic -- this is a huge investment of years of your life. You should ask difficult questions before you make the leap. And the professor should owe you such answers before committing to taking you on. In fact, it's not that different from an awkward pre-nup contract conversation. Only this time you don't have to be swayed by emotion.
    - legerdemain 6y ago
      Well, my story was 15 years ago and in an area where students were mostly expected to bring their own research questions to the program. I had trouble getting the time of day from potential advisors, to say nothing of interviewing them about potential projects and their advising methodology!

n3ur0n 6y ago

As a rising 5th year PhD in ML -- I could not agree with this advice more! I have very hands-off advisors. I spent the first 3 years "wandering the woods to find something". Last year, I really had to sit down and think about how I can finish up my PhD on time. I pretty much did what you outline here. A lot of tools I used were organization tools I learned from my business school/product manager friends. I honestly think the PhD system needs to be overhauled. even at a "top-tier" program like mine, it is amazing to me that at no point do we receive any training regarding practical components of research. 100 years ago, the way you became a physician was to follow around a physician and one day you were ready to be a physician yourself. In the year 2020, this is how PhDs are trained. I do realize that a PhD is not a "professional" degree like MD or JD, however, given that most PhD grads will 1) go to industry 2) go into academia, we need to teach students about project management, planning etc.

ta1234567890 6y ago
> we need to teach students about project management, planning etc. This is a problem with education in general. Those things should be taught in middle/high school as essential tools for modern life. Same as personal finances, effective strategies for team/group work, overcoming social anxiety, managing stress and building/maintaining relationships (personal and professional).
- quineoa 6y ago
  Couldn’t agree more. Instead of learning those things in school. I got lucky and had privilege and learned some of them outside of school, and still need to learn others. But instead, they made sure I knew the name of the boats Columbus sailed in, in 1492. The Niña the Pinta, and the Santa Maria. I was forced to submit to whatever story they told me about the relationship between Columbus and the natives, no matter how untrue it was. But maybe that was the real lesson....
  - hinkley 6y ago
    One of the reasons I recommend volunteer work to people who graduated or are about to graduate with nothing like a plan for getting a job is that this is how I learned a lot of organizational skills (and got my first references). It also makes that, “what have you been doing for the last five months?” question a lot less uncomfortable to answer.

bonoboTP 6y ago

> Concretely, even if you don't know what the answer is going to be at the end of your research, you must think about, or have an idea about, the format of what that amazing answer is going to be. Write the outline of your thesis and "ghost out" what the major charts will be. Write the intro sentences of each chapter -- what are they? (and I don't just mean the boring review of the field part, but your findings part) > You should know what major type of finding, plot, or table your research is going to output. What are the columns and rows of that table, or axes of that plot? How many data points are required? How many of them can already be guessed? Where is the surprise going to be? What is the conclusion going to be? You are absolutely right and it's depressing. The only practical, predictable and steady way to a PhD is to come up with the storyline in advance, do the experiments, bend the interpretation and analysis to support the story you already wanted in advance, then pretend the finding is surprising and novel (when actually you knew you could safely make it work in advance). There is a tension here, on the one hand PhDs have to pretend play that they discovered major novel stuff and innovated a lot (which is only possible through immense risk), on the other hand in reality they must finish in a given number of years and must be very risk averse. Only a major cultural shift towards handing out PhD titles for meticulous, systematic, risky but unfruitful research may help. Forcing people to find flashy new stuff will lead to tons of overinflated findings that are more about storytelling than actual surprising novel things that bring a field forwards. It's all a huge pretend play. ---- Ideally in an alternate universe of different incentive structures that support real science, a person would work in research, use existing approaches in different scenarios, push and prod them in different use cases then perhaps once every year you'd stumble upon some interesting effect you don't understand. Then you'd dive in and prod this specific effect with well designed experiments. Once it's clearer what is going on, you come up with the best way to convey this newly earned knowledge. It is at this point (once you actually know what you want to alert the research community to) that you decide how to best present the knowledge, what tables, sections and paragraphs will succinctly and ergonomically convey the information. If you knew all the story and layout beforehand and you know you will be able to just plug in the numbers and make the paper stand, then you are not doing real science, you are playing the academic game, a cargo cult of science, a game stealing and consuming the prestige of science. Yes, as I said this only works in the alternate universe. Here in this universe we must churn out papers and must overinflate our findings. Of course, we need to eat, we need jobs, we need titles etc. It has driven so many PhD students I know to bitterness and cynicism. Several have given up and went to industry. Others, after the disillusionment, have cynically pushed through with whatever minimal things they knew would be required (plus the right story and presentation) then bailed to consulting. You also need to bullshit in consulting but are at least compensated for it well enough. In the rarest cases do people come out of it as a positive passionate person. Even the successful ones become a hedging, scheming, lukewarm, slimy academic personality. Everything encourages this. The way you get professor positions, the way you get grant money etc. It optimizes and chisels people into becoming the antithesis of curious, open scientific inquiry.

xab31 6y ago

> An effective PhD advisor/thesis isn't wandering the woods to find something. It's a guided coaching exercise, with an outcome in mind. There is a tradeoff here. One of Eric's selling points for his lab is obviously that students get a lot of intellectual freedom to develop their own ideas. This is the "wandering the woods" approach, which was my advisor's as well. The advantage is the freedom, as well as the responsibility and intellectual independence that comes from exercising it. The disadvantage is that it can lead to a lot of wasted time and dead ends. The "guided coaching exercise", when taken to the opposite extreme, results in an advisor that hands a project with clearly defined goals and outcomes to the student. That kind of approach usually, in my experience, leads to more and higher-impact papers. It does not develop the student very well, but it does develop their CV, which is quite important, and once that student becomes a postdoc, it is do-or-die time, and every CV item helps. But if we just view this advice as "how should someone doing a project conceptualize their project", this is good advice. Although you would be astonished how many PIs don't follow it. I've had PIs want to write papers before seeing a single figure. I've had countless collaborators send me data to analyze without any clear idea what sort of results/analysis they want, without a hypothesis even.

Beldin 6y ago
The job of a PhD advisor is to guide the student to become an independent researcher. That is: someone able to conceive of research-worthy ideas, separate the good from the bad, execute the research and write it all up in a paper. I have seen very hands-off approaches and very structured approaches. Both have their merits and pitfalls: the first may leave a student stranded in the proverbial forest, while the second may turn into nothing more than a 4 year long series of related tasks to be executed, set at day one, with no input from the student. On the other hand, I've also seen both approaches work well. Basically, you want a daily supervisor that cares, and with whom you have a click.

shrimpx 6y ago

My impression is that things are drastically different if you’re going for a top professorship vs something else, like a job at google. If you’re going for a professorship then basically nothing matters until you strike on an idea that hits it out of the park in some way. And it’s a tricky combination of technical and political fitness. Depending on your work, some schools will like you more than others. If you’re just looking for a job in some research lab then a basic “get it done” process on an incremental result works fine.