3 ms·

Ask HN: Advice on how to tackle a summer research project

2 points by SandB0x 16y ago

I'm about to start a three month project for my masters, in the field of computer vision and machine learning. Of course I have an advisor, but mostly this is going to be a period of fairly open ended independent work, and at the end I need to have something to show for my time. It's a bit daunting, and I'd appreciate receiving some wisdom from you on how to get good research done in such a short time. Some specific points that come to mind are: > Ideas for testing techniques in a simplified form, to avoid wasting weeks reaching dead ends. > Avoiding feature creep - I get distracted by every new paper I read and want to apply it to my problem. > How to keep track of my work, so that the final write-up is as painless as possible. Any other thoughts most welcome. Thanks in advance

_delirium 16y ago
Hmm, it's a kind of broad question. What kind of project roughly do you have in mind? Is it something along the lines of: compare/benchmark existing techniques for [some problem]? Or: improve current widely used technique by using [some ideas]? Or: propose new technique based on some idea of yours, your advisor's, or from the literature? The risks/etc. of the different approaches are pretty different. As a time-limited project, one way of reducing risk is to think up front about your hypotheses, and stick to testing the hypotheses and investigating explanations for your results, rather than engaging in a more open-ended quest like "improve performance on [problem]". That is, your successful end state is: I confirmed or refuted this hypothesis, and did a good job explaining every step of the way. For example, based on background knowledge/etc., you could come up with an argument for why it's plausible that some algorithms could be improved in some particular ways. Perhaps you're proposing an improvement that was used successfully in a different domain, or on a related algorithm, but hasn't been tried in this one; or you have an argument for why it should work based on some sort of theoretical analysis. Then, implement and run tests, and try some variations based on those results. Then report whether your improvements worked or didn't. If your initial hypothesis is reasonable, even a negative result can be fine, especially if you can, after the fact, explain why this plausible-seeming technique or improvement turned out not to work (i.e. what was wrong about the initial intuition, or what differs about your domain where it didn't work from other cases where something similar was applied successfully). The biggest risk is probably proposing to come up with a solution to a problem where no decent solutions currently exist. Then it's hard to promise that you can get one in one summer. It's possible you might, or it's possible you'll just find 5 ways that didn't work. If you do plan to go that way, you should make sure your advisor would accept a "progress towards solution" writeup in the event you don't come up with something. That kind of thing is common in math and theoretical CS, where it's quite possible even a PhD student will fail to prove some important theorem, but after working on it for a few years, will write up a thesis that's more or less: here are some approaches to Important Theorem, and what did and didn't work about them. It's a bit less common in ML, but if you can explain why the problem is interesting, and demonstrate that the obvious solutions don't work and lay out the parameters of what would have to be done to overcome those problem they have, I could see that being a good thesis. --- A short version: come up early on with some idea of what your hypothesis will be, and what sorts of positive or negative writeups you could write, depending on how experiments come out. What you don't want to do is just spend 11 weeks working on random things, then one frantic week trying to figure out what out of that you can write up.
- SandB0x 16y ago
  Thanks for taking the time - this is all really helpful. You seem to have understood the situation well (unless I'm reading this like a horoscope) - there are currently no decent, general solutions to my problem. Most recent papers use techniques that work well on images, apply them to video, show a couple of limited results and of course the authors don't publish any code. Conceptually, everyone can see that this class of techniques is not suitable, and whenever a paper is re-implemented the results are extremely poor. Basically, it looks like a dead end and I think even a tentative first step in a new direction would be counted as a success. But yes I will make sure to clarify this. So based on your advice, over the next few days I will nail down the techniques to try, write a short hypothesis for why I think each one can work, and plan my time so that I am able to run with a promising approach or combination of approaches for the last month or so. Thanks again. If you're interested, drop me an email and if I have something fun I will send you a copy of my writeup later this year.

2 points by SandB0x 16y ago

_delirium 16y ago

Hmm, it's a kind of broad question. What kind of project roughly do you have in mind? Is it something along the lines of: compare/benchmark existing techniques for [some problem]? Or: improve current widely used technique by using [some ideas]? Or: propose new technique based on some idea of yours, your advisor's, or from the literature? The risks/etc. of the different approaches are pretty different. As a time-limited project, one way of reducing risk is to think up front about your hypotheses, and stick to testing the hypotheses and investigating explanations for your results, rather than engaging in a more open-ended quest like "improve performance on [problem]". That is, your successful end state is: I confirmed or refuted this hypothesis, and did a good job explaining every step of the way. For example, based on background knowledge/etc., you could come up with an argument for why it's plausible that some algorithms could be improved in some particular ways. Perhaps you're proposing an improvement that was used successfully in a different domain, or on a related algorithm, but hasn't been tried in this one; or you have an argument for why it should work based on some sort of theoretical analysis. Then, implement and run tests, and try some variations based on those results. Then report whether your improvements worked or didn't. If your initial hypothesis is reasonable, even a negative result can be fine, especially if you can, after the fact, explain why this plausible-seeming technique or improvement turned out not to work (i.e. what was wrong about the initial intuition, or what differs about your domain where it didn't work from other cases where something similar was applied successfully). The biggest risk is probably proposing to come up with a solution to a problem where no decent solutions currently exist. Then it's hard to promise that you can get one in one summer. It's possible you might, or it's possible you'll just find 5 ways that didn't work. If you do plan to go that way, you should make sure your advisor would accept a "progress towards solution" writeup in the event you don't come up with something. That kind of thing is common in math and theoretical CS, where it's quite possible even a PhD student will fail to prove some important theorem, but after working on it for a few years, will write up a thesis that's more or less: here are some approaches to Important Theorem, and what did and didn't work about them. It's a bit less common in ML, but if you can explain why the problem is interesting, and demonstrate that the obvious solutions don't work and lay out the parameters of what would have to be done to overcome those problem they have, I could see that being a good thesis. --- A short version: come up early on with some idea of what your hypothesis will be, and what sorts of positive or negative writeups you could write, depending on how experiments come out. What you don't want to do is just spend 11 weeks working on random things, then one frantic week trying to figure out what out of that you can write up.

SandB0x 16y ago
Thanks for taking the time - this is all really helpful. You seem to have understood the situation well (unless I'm reading this like a horoscope) - there are currently no decent, general solutions to my problem. Most recent papers use techniques that work well on images, apply them to video, show a couple of limited results and of course the authors don't publish any code. Conceptually, everyone can see that this class of techniques is not suitable, and whenever a paper is re-implemented the results are extremely poor. Basically, it looks like a dead end and I think even a tentative first step in a new direction would be counted as a success. But yes I will make sure to clarify this. So based on your advice, over the next few days I will nail down the techniques to try, write a short hypothesis for why I think each one can work, and plan my time so that I am able to run with a promising approach or combination of approaches for the last month or so. Thanks again. If you're interested, drop me an email and if I have something fun I will send you a copy of my writeup later this year.