4 ms·

Building a State of the Art Bacterial Classifier with Fast.ai and Paperspace

112 points by hsikka 7y ago

sp332 7y ago
Very cool. I'm surprised ResNet50 had so much of an advantage over ResNet34 with only 660 samples.
- hsikka 7y ago
  Yes, it shocked me too! In this project I didn't get a chance to dive deep into some of the decisions the Fast.ai Library did, but I'm hoping to see if there's some inherent gain based on training. I'm also very curious what the performance of some of the newer architectures, i.e. capsulenets would look like.
  - sp332 7y ago
    I know this is a simplistic comment, but do you suppose it was overfitting? I know the tools try to avoid this, but with so many parameters in ResNet50 maybe it's harder to avoid.
    - dhairya 7y ago
      Deep neural nets have a natural tendency to overfit. Ideally, you have a held-out test which the model hasn't seen and only used after model has trained and tuned on the dev and validation sets. Often bad experimental models will repeatedly use the test set in fine-tuning an existing model which may result in your model learning about the test set rendering the test set useless. In the practice real world results may vary as your training and test data may not represent the actual distribution of the real world data.
    - jph00 7y ago
      fastai uses held-out data for reporting accuracy by default, which was done in this article.
cntrlaltdlt 7y ago
Man I had such a problem using paper space's notebook system, that I just gave up. In retrospect though it was more to do with me not understanding how to use generator functions to better control memory bloating.
- Edmar 7y ago
  Have you tried Google Collab? I tend to use it a lot for running tutorials.
Edmar 7y ago
Anyone that liked this tutorial should check this course: https://course.fast.ai/ https://course.fast.ai/ 1) It's free 2) Jeremmy is a good teacher and one the library creators.
- hsikka 7y ago
  Second this! The course is solid and drops you right in to practice, which is great. You could actually replicate all the work in this post after just watching the first lesson, that's how fast you start learning.
hsikka 7y ago
Hey HN, author here. This post was the result of a small set of experiments that came out of the Paperspace Advanced Technologies Group. We've been working on some pretty ambitious research projects at the intersection of systems, ML, and HCI, and we were evaluating tools and libraries (i.e. Keras and Fast.ai) that would allow us to prototype concepts quickly. (More on our research approach and project structure coming soon). We found this interesting classification task and used it as a testbed for some small scale testing and the results were pretty cool!
- emilwallner 7y ago
  Solid work - hats off!
  - hsikka 7y ago
    Thanks my friend, hope all is going well with you! I've been working on some cool stuff, will definitely share with you soon.
    - emilwallner 7y ago
      Awesome, looking forward to it!
carbocation 7y ago
Since 'hsikka is here in the thread - wondering if you can clear up a question that I have from your article. In the FastAI course, they use the descending limb of the one-cycle curve to set learning rate cutoffs. So I was expecting to see your final model fit with learning rates between ~3e-4 and ~5e-2. However, you used 1e-6 and 1e-4, basically on the flat portion of the curve. That seemed to work just fine for you. Am I correct that you didn't follow the standard recommendations for the learning rate from one-cycle, or is there some mismatch with the figure?
jcims 7y ago
This is very cool. I've taken an interest in cytotoxins and cytotoxic therapies lately, and there seems to be a use here to help identify and measure impact of various treatments on the cell lifecycle. One example is the following which shows the difference in cell lifecycle between untreated cancer cells and those being treated with a 200KHz electromagnetic field ('TTFields'): https://www.youtube.com/watch?v=voVa7Pj2xUg https://www.youtube.com/watch?v=voVa7Pj2xUg Presently videos like the above are manually reviewed. However, the timescales and noise of some of these observations seem to stretch human attention quite a bit. The unblinking eye of a neural network might help inform the process, if for no other reason than to help direct human attention to anomalous behavior. For example, in the above video 'blebbing' is seen as an indicator that the cell has started apoptosis and will subsequently die. This is supported by the overall lack of growth in the culture and the mechanism of action is attributed to tubulin disruption. However, later analysis shows that the cells actually recover from this state but tend to have corrupted mitosis in the future. This in turn indicated that the mechanism of action may be something else.

112 points by hsikka 7y ago

sp332 7y ago

Very cool. I'm surprised ResNet50 had so much of an advantage over ResNet34 with only 660 samples.

hsikka 7y ago
Yes, it shocked me too! In this project I didn't get a chance to dive deep into some of the decisions the Fast.ai Library did, but I'm hoping to see if there's some inherent gain based on training. I'm also very curious what the performance of some of the newer architectures, i.e. capsulenets would look like.
- sp332 7y ago
  I know this is a simplistic comment, but do you suppose it was overfitting? I know the tools try to avoid this, but with so many parameters in ResNet50 maybe it's harder to avoid.
  - dhairya 7y ago
    Deep neural nets have a natural tendency to overfit. Ideally, you have a held-out test which the model hasn't seen and only used after model has trained and tuned on the dev and validation sets. Often bad experimental models will repeatedly use the test set in fine-tuning an existing model which may result in your model learning about the test set rendering the test set useless. In the practice real world results may vary as your training and test data may not represent the actual distribution of the real world data.
  - jph00 7y ago
    fastai uses held-out data for reporting accuracy by default, which was done in this article.

cntrlaltdlt 7y ago

Man I had such a problem using paper space's notebook system, that I just gave up. In retrospect though it was more to do with me not understanding how to use generator functions to better control memory bloating.

Edmar 7y ago
Have you tried Google Collab? I tend to use it a lot for running tutorials.

Edmar 7y ago

Anyone that liked this tutorial should check this course: https://course.fast.ai/ https://course.fast.ai/ 1) It's free 2) Jeremmy is a good teacher and one the library creators.

hsikka 7y ago
Second this! The course is solid and drops you right in to practice, which is great. You could actually replicate all the work in this post after just watching the first lesson, that's how fast you start learning.

hsikka 7y ago

Hey HN, author here. This post was the result of a small set of experiments that came out of the Paperspace Advanced Technologies Group. We've been working on some pretty ambitious research projects at the intersection of systems, ML, and HCI, and we were evaluating tools and libraries (i.e. Keras and Fast.ai) that would allow us to prototype concepts quickly. (More on our research approach and project structure coming soon). We found this interesting classification task and used it as a testbed for some small scale testing and the results were pretty cool!

emilwallner 7y ago
Solid work - hats off!
- hsikka 7y ago
  Thanks my friend, hope all is going well with you! I've been working on some cool stuff, will definitely share with you soon.
  - emilwallner 7y ago
    Awesome, looking forward to it!

carbocation 7y ago

Since 'hsikka is here in the thread - wondering if you can clear up a question that I have from your article. In the FastAI course, they use the descending limb of the one-cycle curve to set learning rate cutoffs. So I was expecting to see your final model fit with learning rates between ~3e-4 and ~5e-2. However, you used 1e-6 and 1e-4, basically on the flat portion of the curve. That seemed to work just fine for you. Am I correct that you didn't follow the standard recommendations for the learning rate from one-cycle, or is there some mismatch with the figure?

jcims 7y ago

This is very cool. I've taken an interest in cytotoxins and cytotoxic therapies lately, and there seems to be a use here to help identify and measure impact of various treatments on the cell lifecycle. One example is the following which shows the difference in cell lifecycle between untreated cancer cells and those being treated with a 200KHz electromagnetic field ('TTFields'): https://www.youtube.com/watch?v=voVa7Pj2xUg https://www.youtube.com/watch?v=voVa7Pj2xUg Presently videos like the above are manually reviewed. However, the timescales and noise of some of these observations seem to stretch human attention quite a bit. The unblinking eye of a neural network might help inform the process, if for no other reason than to help direct human attention to anomalous behavior. For example, in the above video 'blebbing' is seen as an indicator that the cell has started apoptosis and will subsequently die. This is supported by the overall lack of growth in the culture and the mechanism of action is attributed to tubulin disruption. However, later analysis shows that the cells actually recover from this state but tend to have corrupted mitosis in the future. This in turn indicated that the mechanism of action may be something else.