8 ms·

Fastai for PyTorch: Fast and accurate neural nets using modern best practices

279 points by stablemap 8y ago

jph00 8y ago
Jeremy from fast.ai here - I'm sitting here at the PyTorch developer conference, listening to all the really great new stuff being announced. :) Happy to answer any questions about the fastai library release here. It's a rewrite from scratch compared to v0.7. Docs here: http://docs.fast.ai http://docs.fast.ai . GitHub repo here: https://github.com/fastai/fastai https://github.com/fastai/fastai . It's also available now on Google Cloud Platform, including example notebooks and datasets (Viacheslav Kovalevskyi from Google has posted a walk-thru here: https://blog.kovalevskyi.com/google-compute-engine-now-has-images-with-pytorch-1-0-0-and-fastai-1-0-2-57c49efd74bb https://blog.kovalevskyi.com/google-compute-engine-now-has-i... ). AWS support coming soon.
- deathcalibur 8y ago
  Thanks for all your hard work Jeremy. In Cleveland, we are running a study group with about 30 people on the 2018 version of the course. Because of logistics, we aren't planning on rescheduling to match the new course, but its still nice knowing that the library is being invested into.
- maykr 8y ago
  Hello and thank you for the great work! Hopefully, the new version will work on MacOS with Anaconda, because 0.7 failed there due to the unsatisfied dependency on PIL. Apparently, the default Anaconda channels doesn't include the python 3.x-compiliant PIL package for Darwin. That blocked my progress on ML course.
  - jacquesm 8y ago
    You can always get a server with a GPU instance in the cloud somewhere. That might cost a few $ but then again, so does a GPU.
  - jph00 8y ago
    MacOS is really hard to work with for deep learning. Try https://salamander.ai/ https://salamander.ai/ . $0.19/hour for a server with NVIDIA GPU, fastai v1 pre-installed with notebooks and datasets. Or try Google Cloud preemptible instances - similar cost IIRC, but a little more setup to do.
    - ashtonsix 8y ago
      Oh, it's $0.36/hour for salamander. the equivalent on gcp (n1-16-standard with attached k80) is $0.29/hour. though as you said, Salamander is easier to get started with (just 1 click after signing up!) ps: i'm the creator of salamander
  - kalefranz 8y ago
    Are you sure you didn’t want pillow, which provides the PIL import and is the follow-on from the original PIL?
- fuddle 8y ago
  I was just listening to the the podcast when I saw this post. It's great to hear you can make a lot of progress on ML without a PHD. :)
  - ScoutOrgo 8y ago
    What podcast?
    - jph00 8y ago
      I assume they are referring to the TWiMLAI podcast that was posted this morning https://twimlai.com/twiml-talk-186-the-fastai-v1-deep-learning-framework-with-jeremy-howard/ https://twimlai.com/twiml-talk-186-the-fastai-v1-deep-learni...
  - shas3 8y ago
    When someone has a very high IQ, like Jeremy Howard obviously does, PhD becomes just a superfluous credential.
- pitchups 8y ago
  Great - can't wait to try out the new version! I am running fast.ai in a virtual environment. How does one upgrade to the latest version for both Fast.ai and PyTorch. Are the new versions backwards comaptible with the earlier version - would it break any code? If not, would you recommend I clone the current install and upgrade that to be on the safe side. Thanks!
  - jph00 8y ago
    None of fastai is backwards compatible with v0.7 - although the API has a similar structure, so it shouldn't be hard to rewrite. PyTorch v1.0 also has some big differences to the version we used in the course. I suggest you create a new conda environment and then follow the instructions in the readme to install both.
- exploreml 8y ago
  Blog announcement here: https://cloud.google.com/blog/products/ai-machine-learning/introducing-pytorch-across-google-cloud https://cloud.google.com/blog/products/ai-machine-learning/i... For a little latitude, I believe there are PyTorch VMs available on the other cloud providers as well.
- ericd 8y ago
  Hey Jeremy, just wanted to say thanks very much for making the course and the library, both have been invaluable for us in getting started with deep learning (and achieving useful results) quickly. Both parts of the 2018 course have been a really great tour through the different areas of deep learning, and provided good jumping off points for digging deeper on the parts we got interested in.
- bcheung 8y ago
  I have a gaming system (Ryzen 1950X w/ GTX 1080Ti). Is there an easy way to create a bootable external media with most of the stuff preconfigured already? I remember it being a pain to get all the drivers working and there was a lot of version incompatibility in the Python ecosystem. I'd rather not mess with the bootloader and just have an external drive I can select to boot to in the BIOS. For some reason Windows wiped out GRUB and I can't boot my old Ubuntu distro anymore.
  - jph00 8y ago
    The story on python is much better nowadays. You can install cuda and drivers from nvidia, and then install anaconda, then just run: conda install -c pytorch -c fastai fastai pytorch-nightly cuda92 If you don't use conda, things can get more complicated, depending on what libs you have installed.
  - maykr 8y ago
    You have a couple of options to consider: 1. Install a modern Linux distribution on the bare metal, get https://github.com/NVIDIA/nvidia-docker https://github.com/NVIDIA/nvidia-docker and run the GPU-accelerated container https://github.com/Paperspace/fastai-docker https://github.com/Paperspace/fastai-docker 2. Try running the Ubuntu Hyper-V image provided by Canonical/Microsoft P.S. I'm considering building a DL box based on Ryzen 2600 w/ GTX 1070Ti, so could you please share your experience with Ryzen and Linux.
    - JimmyAustin 8y ago
      My DL box has a Ryzen 1800x and a GTX 1080ti running Ubuntu 18.04. The Ryzen has given me very few issues, Nvidia gave me some setup issues, but now that it's all configured its rock solid.
    - bcheung 8y ago
      A Hyper-V image would be great if it can take advantage of the GPU and not cause driver issues. Didn't know they had one for DL. Or is it just pure Ubuntu you mean?
  - 8y ago
- sweezyjeezy 8y ago
  Why do you hate variable names longer than 4 characters so much? :b
- montenegrohugo 8y ago
  Hi Jeremy, not a question just a short thank you note. I think the value you produce with fast.ai is absolutely fantastic, and I really want to communicate my gratefulness for all of the excellent content and courses and blog posts. Especially having access to them for free. Personally, it has helped me to learn so much and to actually implement these techniques in a workplace setting. Again, thanks. Regards
- ipsum2 8y ago
  Hey Jeremy, will you update your courses to reflect the new API changes? I just started on your deep learning course, using pip to install the fastai library.
  - claytonjy 8y ago
    The live version of the course, updated for fastai v1, is beginning later this month, with publicly available videos to follow in early 2019.
- kuu 8y ago
  Hi Jeremy! What is the main difference with Keras+TF, I've only worked with them (still not mastering them), why should I consider fastai? Thank you!
  - yboris 8y ago
    Possibly addressed in a passage from this post: http://www.fast.ai/2018/10/02/fastai-ai/ http://www.fast.ai/2018/10/02/fastai-ai/ > It’s important to understand that these improved results over Keras in no way suggest that Keras isn’t an excellent piece of software. Quite the contrary! If you tried to complete this task with almost any other library, you would need to write far more code, and would be unlikely to see better speed or accuracy than Keras. That’s why we’re showing Keras in this comparison - because we’re admirers of it, and it’s the strongest benchmark we know of!
- mcintyre1994 8y ago
  Hey Jeremy, are you going to be recommending GCP/AWS hosted fastai as the best way to get started now over Paperspace etc?
pitchups 8y ago
This is great news! Here has been my experience using Fast.ai : I had been training a deep learning network using Keras with Tensorflow, for diagnosing medical images - and it took me several months of hard work - tweaking parameters, training and testing to get acceptable levels of accuracy for our models. And then last month, I switched to Fast.ai (their pre-release version) and I was blown away - my models trained faster, and I matched and finally exceeded accuracy levels acheived with my earlier models. And I accomplished what had taken several months in Keras, in just a few days! And the biggest reasons for it were in my view, fast.ai's learning rate finder, the differential learning rates, and Test Time augmentation - all which are advanced features built into fast.ai. And the other great thing is that fast.ai uses the best defaults automatically, and it trains much, much faster than Keras / TF for some reason. So I can't wait to try the new release out. I think Fast.ai has set a new bar for deep learning frameworks in terms of speed and ease of use. Thank you for all your great work!
- infinitone 8y ago
  Do you know if you can do object detection on images right out of the box? Or is it just classification?
  - pitchups 8y ago
    I am using it for classification right now - but pretty sure you should be able to do object detection quite easily as well.
    - jph00 8y ago
      It supports segmentation right now, but object detection won't be supported "out of the box" until 1.1 (in a month or two). You can certainly train a YOLO or SSD pytorch model with fastai, however.
      - lelima 8y ago
        Hello Jeremy, One question, there is any best practice to transform video into [n] frames to then use model.predict(n) to make a "live classification / object detection"? Kind regards from Dublin!
qwerty456127 8y ago
Is it meant to be a better alternative to Keras?
- jph00 8y ago
  Keras doesn't support PyTorch, so it's not exactly an alternative. But it would be fair to call it an inspiration for fastai.
yazr 8y ago
Can this be used / intended for production ? Is it fast/efficient enough on common hw ? We are not really doing cutting edge or research stuff. Just developing a big dumb resnet and hoping to scale up to 10s of GPUs over 2019. I was really happy to read a comment here about how this framework was used to reproduce a 6 month project in 2 weeks :)
- jph00 8y ago
  It's very fast indeed: http://www.fast.ai/2018/08/10/fastai-diu-imagenet/ http://www.fast.ai/2018/08/10/fastai-diu-imagenet/ For inference, new features being announced today at the conference will help a lot too.
  - songhuicheng 8y ago
    Will serving be as fast as TensorFlow? I assume PyTorch's dynamic network is inherently slower than TensorFlow's compute graph approach
imaginenore 8y ago
What would be the reason to learn PyTorch when Tensorflow and Keras have more market share? Like what's the real advantage here? https://deepsense.ai/wp-content/uploads/2018/06/arvix_graph_cut.png https://deepsense.ai/wp-content/uploads/2018/06/arvix_graph_...
binalpatel 8y ago
I'm a big fan - I've been using their old version of their library for NLP on medical text, but the course and library certainly removed a lot of the "mystique" around deep learning for me.
screye 8y ago
I haven't used fast.ai yet, but extensively use pytorch for anything to do with deeplearning. (it is so much better than tensorflow) It's great to see a keras like higher level abstraction library for deep learning. A few questions: 1. what is the benefit of using fast.ai to someone well acquainted with pytorch (for academic use) 2. how well does fast.ai interface with pytorch itself ? Can parts of program be in fast ai and other parts be in pytorch ? 3. Am I correct in assuming that despite being very fast, fast.ai is still slower (even if marginally so) than pytorch itself ?
- jph00 8y ago
  1. Less boilerplate, so you can focus on your algorithm; best practices built in, so you might be surprised at how your speed and accuracy may improve 2. Very close integration with PyTorch. fastai is designed to extend PyTorch, not hide it. E.g. fastai uses standard PyTorch Datasets for data, but then provides a number of pre-defined Datasets for common tasks 3. fastai is not slower than PyTorch, since PyTorch is handling all the computation. It'll othen be faster than your handwritten PyTorch however, since we went to a lot of effort to use performant algorithms.
  - screye 8y ago
    Thanks for replying. From appearance alone, everything about fast.ai seems to be positive with no obvious negatives over native pytorch. I often require to train baselines for my projects. That will be the perfect place to start using fast.ai. I am sold. I have heard a few people lament the lack of keras in pytorch. Fast.ai also appears to take care of that. Thanks for the great work.
- poppingtonic 8y ago
  1. You get a state-of-the art training loop that includes the latest research findings such as 1-cycle. 2. Yes. You can make everything project-specific if you want to. e.g. Image segmentation datasets are just slightly modified Dataset classes. 3. fastai extends PyTorch in a very Pythonic OO sense, so I think the only speed issues could come from that, and maybe maintaining a few extra dicts in memory. If it's about i/o, probably not, if it's about parallelization (for nlp), definitely not. In fact, I can't think of a way in which there's a significant speed penalty.
montenegrohugo 8y ago
Cannot recommend fast.ai enough. How they and Jeremy especially communicate is absolutely outstanding, and I guarantee you will learn a lot by the results-driven approach that they have.
jph00 8y ago
ZDNet just published an article with some more background about this BTW : https://www.zdnet.com/google-amp/article/fast-ais-new-software-could-radically-democratize-ai https://www.zdnet.com/google-amp/article/fast-ais-new-softwa...
synaesthesisx 8y ago
Is there a guide available for image segmentation? I've only really used TF in the past
- jph00 8y ago
  There's a development notebook for segmentation on the Carvana dataset. https://github.com/fastai/fastai_docs/blob/master/dev_nb/006_carvana.ipynb https://github.com/fastai/fastai_docs/blob/master/dev_nb/006... Most of the functions in there are pre defined in fastai so you can remove much of the code there in practice. We'll be adding a proper segmentation example soon.
mkolodny 8y ago
This is awesome! I'd love to use Fastai, but my company's pretty committed to Tensorflow right now. Is there any work being done to port Fastai over to Tensorflow?
zeptomu 8y ago
The name fastai sounds suspicious. There is "fast" and "AI" in it, so I fear that it makes the easy things easier and the hard things harder. It just sounds like too much fluff. We obviously don't have AI yet (but pretty convincing machine learning models) and we won't get it in the next years (not fast) - so if software, a company or an organization puts "ai" in its name (or chooses it as top-level domain) I suspect they just want to ride the hype train and am very skeptical ... @Topic: What is the difference to Keras, PyTorch, etc.? They are already pretty high-level and the basic models for common tasks are available in all libraries at that point.
- deleted 8y ago
  [deleted]
- pure-awesome 8y ago
  I mean, _just_ basing your impression off of the name is not going to get you very far. As a first impression, sure, but if a bunch of comments are saying they really like it, maybe it's worth doing a bit more exploration. And AI is commonly-used as basically a synonym for Machine Learning in the ML community. Yes, the word has multiple definitions and connotations, and some people prefer to avoid using the term AI at all because of that. But that doesn't meant that someone using the term AI to refer to ML is wrong or up to something. Furthermore, just because non-legitimate organizations want to ride the hype train, it doesn't mean legitimate ones shouldn't benefit from that effect as well. fast.ai is an organization actually writing software for doing machine learning, as opposed to a completely unrelated company (like an electric toothbrush company) claiming their latest product "uses AI" whatever that means. The funniest part is that the slogan for fast.ai "Making neural nets uncool again". If that isn't explicitly going against the hype and saying this is _actual_ neural nets and not just fluff, then I don't know what is.
  - zeptomu 8y ago
    > I mean, _just_ basing your impression off of the name is not going to get you very far. There are so many neural networks libraries now and I don't find the time to test all of them - dismissing libraries that use .ai in their website's TLD (or in their name) served me very well so far. Sure if it really makes neural networks that much simpler to train, I will take a look, but for now it does not seem to be more high-level than Keras or PyTorch. It does not matter if you can describe your network in 50, 100 or 200 lines of code as the hard part is to make it learn, choose hyper parameters, change loss functions, etc. - this is possible in all frameworks. And I don't think AI is used as a synonym for machine learning in the community, but is used to describe AGI.
denfromufa 8y ago
@jph00 where do I find lstm/gru/seq2seq layers for time-series sequence predictions (not text)? Also interested in autoencoder implementations. The fast.ai docs search does not really work for this. What do you think about other notable APIs built on top of pytorch such as Pyro and AllenNLP?
sharcerer 8y ago
Hi Jeremy, I am a beginner. Going to start your ML course,then follow up with DL- 1 & 2. My question is when will these 3 be updated to the new version of fastai. I am aware of the fellowship for DL-1, but there's a 12 hour difference. Also, I want to start with ML,so, I want to learn with the new version. Thanks On the side, I am also taking Andrew's coursera course for a theoretical grounding , Bengio and Goodfellow's Machine Learning Book ,and Hands-On Machine Learning with Tensorflow and Sci-kit( O'reilly book).

279 points by stablemap 8y ago

Jeremy from fast.ai here - I'm sitting here at the PyTorch developer conference, listening to all the really great new stuff being announced. :) Happy to answer any questions about the fastai library release here. It's a rewrite from scratch compared to v0.7. Docs here: http://docs.fast.ai http://docs.fast.ai . GitHub repo here: https://github.com/fastai/fastai https://github.com/fastai/fastai . It's also available now on Google Cloud Platform, including example notebooks and datasets (Viacheslav Kovalevskyi from Google has posted a walk-thru here: https://blog.kovalevskyi.com/google-compute-engine-now-has-images-with-pytorch-1-0-0-and-fastai-1-0-2-57c49efd74bb https://blog.kovalevskyi.com/google-compute-engine-now-has-i... ). AWS support coming soon.

deathcalibur 8y ago
Thanks for all your hard work Jeremy. In Cleveland, we are running a study group with about 30 people on the 2018 version of the course. Because of logistics, we aren't planning on rescheduling to match the new course, but its still nice knowing that the library is being invested into.
maykr 8y ago
Hello and thank you for the great work! Hopefully, the new version will work on MacOS with Anaconda, because 0.7 failed there due to the unsatisfied dependency on PIL. Apparently, the default Anaconda channels doesn't include the python 3.x-compiliant PIL package for Darwin. That blocked my progress on ML course.
- jacquesm 8y ago
  You can always get a server with a GPU instance in the cloud somewhere. That might cost a few $ but then again, so does a GPU.
- jph00 8y ago
  MacOS is really hard to work with for deep learning. Try https://salamander.ai/ https://salamander.ai/ . $0.19/hour for a server with NVIDIA GPU, fastai v1 pre-installed with notebooks and datasets. Or try Google Cloud preemptible instances - similar cost IIRC, but a little more setup to do.
  - ashtonsix 8y ago
    Oh, it's $0.36/hour for salamander. the equivalent on gcp (n1-16-standard with attached k80) is $0.29/hour. though as you said, Salamander is easier to get started with (just 1 click after signing up!) ps: i'm the creator of salamander
- kalefranz 8y ago
  Are you sure you didn’t want pillow, which provides the PIL import and is the follow-on from the original PIL?
fuddle 8y ago
I was just listening to the the podcast when I saw this post. It's great to hear you can make a lot of progress on ML without a PHD. :)
- ScoutOrgo 8y ago
  What podcast?
  - jph00 8y ago
    I assume they are referring to the TWiMLAI podcast that was posted this morning https://twimlai.com/twiml-talk-186-the-fastai-v1-deep-learning-framework-with-jeremy-howard/ https://twimlai.com/twiml-talk-186-the-fastai-v1-deep-learni...
- shas3 8y ago
  When someone has a very high IQ, like Jeremy Howard obviously does, PhD becomes just a superfluous credential.
pitchups 8y ago
Great - can't wait to try out the new version! I am running fast.ai in a virtual environment. How does one upgrade to the latest version for both Fast.ai and PyTorch. Are the new versions backwards comaptible with the earlier version - would it break any code? If not, would you recommend I clone the current install and upgrade that to be on the safe side. Thanks!
- jph00 8y ago
  None of fastai is backwards compatible with v0.7 - although the API has a similar structure, so it shouldn't be hard to rewrite. PyTorch v1.0 also has some big differences to the version we used in the course. I suggest you create a new conda environment and then follow the instructions in the readme to install both.
exploreml 8y ago
Blog announcement here: https://cloud.google.com/blog/products/ai-machine-learning/introducing-pytorch-across-google-cloud https://cloud.google.com/blog/products/ai-machine-learning/i... For a little latitude, I believe there are PyTorch VMs available on the other cloud providers as well.
ericd 8y ago
Hey Jeremy, just wanted to say thanks very much for making the course and the library, both have been invaluable for us in getting started with deep learning (and achieving useful results) quickly. Both parts of the 2018 course have been a really great tour through the different areas of deep learning, and provided good jumping off points for digging deeper on the parts we got interested in.
bcheung 8y ago
I have a gaming system (Ryzen 1950X w/ GTX 1080Ti). Is there an easy way to create a bootable external media with most of the stuff preconfigured already? I remember it being a pain to get all the drivers working and there was a lot of version incompatibility in the Python ecosystem. I'd rather not mess with the bootloader and just have an external drive I can select to boot to in the BIOS. For some reason Windows wiped out GRUB and I can't boot my old Ubuntu distro anymore.
- jph00 8y ago
  The story on python is much better nowadays. You can install cuda and drivers from nvidia, and then install anaconda, then just run: conda install -c pytorch -c fastai fastai pytorch-nightly cuda92 If you don't use conda, things can get more complicated, depending on what libs you have installed.
- maykr 8y ago
  You have a couple of options to consider: 1. Install a modern Linux distribution on the bare metal, get https://github.com/NVIDIA/nvidia-docker https://github.com/NVIDIA/nvidia-docker and run the GPU-accelerated container https://github.com/Paperspace/fastai-docker https://github.com/Paperspace/fastai-docker 2. Try running the Ubuntu Hyper-V image provided by Canonical/Microsoft P.S. I'm considering building a DL box based on Ryzen 2600 w/ GTX 1070Ti, so could you please share your experience with Ryzen and Linux.
  - JimmyAustin 8y ago
    My DL box has a Ryzen 1800x and a GTX 1080ti running Ubuntu 18.04. The Ryzen has given me very few issues, Nvidia gave me some setup issues, but now that it's all configured its rock solid.
  - bcheung 8y ago
    A Hyper-V image would be great if it can take advantage of the GPU and not cause driver issues. Didn't know they had one for DL. Or is it just pure Ubuntu you mean?
- 8y ago
sweezyjeezy 8y ago
Why do you hate variable names longer than 4 characters so much? :b
montenegrohugo 8y ago
Hi Jeremy, not a question just a short thank you note. I think the value you produce with fast.ai is absolutely fantastic, and I really want to communicate my gratefulness for all of the excellent content and courses and blog posts. Especially having access to them for free. Personally, it has helped me to learn so much and to actually implement these techniques in a workplace setting. Again, thanks. Regards
ipsum2 8y ago
Hey Jeremy, will you update your courses to reflect the new API changes? I just started on your deep learning course, using pip to install the fastai library.
- claytonjy 8y ago
  The live version of the course, updated for fastai v1, is beginning later this month, with publicly available videos to follow in early 2019.
kuu 8y ago
Hi Jeremy! What is the main difference with Keras+TF, I've only worked with them (still not mastering them), why should I consider fastai? Thank you!
- yboris 8y ago
  Possibly addressed in a passage from this post: http://www.fast.ai/2018/10/02/fastai-ai/ http://www.fast.ai/2018/10/02/fastai-ai/ > It’s important to understand that these improved results over Keras in no way suggest that Keras isn’t an excellent piece of software. Quite the contrary! If you tried to complete this task with almost any other library, you would need to write far more code, and would be unlikely to see better speed or accuracy than Keras. That’s why we’re showing Keras in this comparison - because we’re admirers of it, and it’s the strongest benchmark we know of!
mcintyre1994 8y ago
Hey Jeremy, are you going to be recommending GCP/AWS hosted fastai as the best way to get started now over Paperspace etc?

pitchups 8y ago

This is great news! Here has been my experience using Fast.ai : I had been training a deep learning network using Keras with Tensorflow, for diagnosing medical images - and it took me several months of hard work - tweaking parameters, training and testing to get acceptable levels of accuracy for our models. And then last month, I switched to Fast.ai (their pre-release version) and I was blown away - my models trained faster, and I matched and finally exceeded accuracy levels acheived with my earlier models. And I accomplished what had taken several months in Keras, in just a few days! And the biggest reasons for it were in my view, fast.ai's learning rate finder, the differential learning rates, and Test Time augmentation - all which are advanced features built into fast.ai. And the other great thing is that fast.ai uses the best defaults automatically, and it trains much, much faster than Keras / TF for some reason. So I can't wait to try the new release out. I think Fast.ai has set a new bar for deep learning frameworks in terms of speed and ease of use. Thank you for all your great work!

infinitone 8y ago
Do you know if you can do object detection on images right out of the box? Or is it just classification?
- pitchups 8y ago
  I am using it for classification right now - but pretty sure you should be able to do object detection quite easily as well.
  - jph00 8y ago
    It supports segmentation right now, but object detection won't be supported "out of the box" until 1.1 (in a month or two). You can certainly train a YOLO or SSD pytorch model with fastai, however.
    - lelima 8y ago
      Hello Jeremy, One question, there is any best practice to transform video into [n] frames to then use model.predict(n) to make a "live classification / object detection"? Kind regards from Dublin!

qwerty456127 8y ago

Is it meant to be a better alternative to Keras?

jph00 8y ago
Keras doesn't support PyTorch, so it's not exactly an alternative. But it would be fair to call it an inspiration for fastai.

yazr 8y ago

Can this be used / intended for production ? Is it fast/efficient enough on common hw ? We are not really doing cutting edge or research stuff. Just developing a big dumb resnet and hoping to scale up to 10s of GPUs over 2019. I was really happy to read a comment here about how this framework was used to reproduce a 6 month project in 2 weeks :)

jph00 8y ago
It's very fast indeed: http://www.fast.ai/2018/08/10/fastai-diu-imagenet/ http://www.fast.ai/2018/08/10/fastai-diu-imagenet/ For inference, new features being announced today at the conference will help a lot too.
- songhuicheng 8y ago
  Will serving be as fast as TensorFlow? I assume PyTorch's dynamic network is inherently slower than TensorFlow's compute graph approach

imaginenore 8y ago

What would be the reason to learn PyTorch when Tensorflow and Keras have more market share? Like what's the real advantage here? https://deepsense.ai/wp-content/uploads/2018/06/arvix_graph_cut.png https://deepsense.ai/wp-content/uploads/2018/06/arvix_graph_...

binalpatel 8y ago

I'm a big fan - I've been using their old version of their library for NLP on medical text, but the course and library certainly removed a lot of the "mystique" around deep learning for me.

screye 8y ago

I haven't used fast.ai yet, but extensively use pytorch for anything to do with deeplearning. (it is so much better than tensorflow) It's great to see a keras like higher level abstraction library for deep learning. A few questions: 1. what is the benefit of using fast.ai to someone well acquainted with pytorch (for academic use) 2. how well does fast.ai interface with pytorch itself ? Can parts of program be in fast ai and other parts be in pytorch ? 3. Am I correct in assuming that despite being very fast, fast.ai is still slower (even if marginally so) than pytorch itself ?

jph00 8y ago
1. Less boilerplate, so you can focus on your algorithm; best practices built in, so you might be surprised at how your speed and accuracy may improve 2. Very close integration with PyTorch. fastai is designed to extend PyTorch, not hide it. E.g. fastai uses standard PyTorch Datasets for data, but then provides a number of pre-defined Datasets for common tasks 3. fastai is not slower than PyTorch, since PyTorch is handling all the computation. It'll othen be faster than your handwritten PyTorch however, since we went to a lot of effort to use performant algorithms.
- screye 8y ago
  Thanks for replying. From appearance alone, everything about fast.ai seems to be positive with no obvious negatives over native pytorch. I often require to train baselines for my projects. That will be the perfect place to start using fast.ai. I am sold. I have heard a few people lament the lack of keras in pytorch. Fast.ai also appears to take care of that. Thanks for the great work.
poppingtonic 8y ago
1. You get a state-of-the art training loop that includes the latest research findings such as 1-cycle. 2. Yes. You can make everything project-specific if you want to. e.g. Image segmentation datasets are just slightly modified Dataset classes. 3. fastai extends PyTorch in a very Pythonic OO sense, so I think the only speed issues could come from that, and maybe maintaining a few extra dicts in memory. If it's about i/o, probably not, if it's about parallelization (for nlp), definitely not. In fact, I can't think of a way in which there's a significant speed penalty.

montenegrohugo 8y ago

Cannot recommend fast.ai enough. How they and Jeremy especially communicate is absolutely outstanding, and I guarantee you will learn a lot by the results-driven approach that they have.

jph00 8y ago

ZDNet just published an article with some more background about this BTW : https://www.zdnet.com/google-amp/article/fast-ais-new-software-could-radically-democratize-ai https://www.zdnet.com/google-amp/article/fast-ais-new-softwa...

synaesthesisx 8y ago

Is there a guide available for image segmentation? I've only really used TF in the past

jph00 8y ago
There's a development notebook for segmentation on the Carvana dataset. https://github.com/fastai/fastai_docs/blob/master/dev_nb/006_carvana.ipynb https://github.com/fastai/fastai_docs/blob/master/dev_nb/006... Most of the functions in there are pre defined in fastai so you can remove much of the code there in practice. We'll be adding a proper segmentation example soon.

mkolodny 8y ago

This is awesome! I'd love to use Fastai, but my company's pretty committed to Tensorflow right now. Is there any work being done to port Fastai over to Tensorflow?

zeptomu 8y ago

The name fastai sounds suspicious. There is "fast" and "AI" in it, so I fear that it makes the easy things easier and the hard things harder. It just sounds like too much fluff. We obviously don't have AI yet (but pretty convincing machine learning models) and we won't get it in the next years (not fast) - so if software, a company or an organization puts "ai" in its name (or chooses it as top-level domain) I suspect they just want to ride the hype train and am very skeptical ... @Topic: What is the difference to Keras, PyTorch, etc.? They are already pretty high-level and the basic models for common tasks are available in all libraries at that point.

deleted 8y ago
[deleted]
pure-awesome 8y ago
I mean, _just_ basing your impression off of the name is not going to get you very far. As a first impression, sure, but if a bunch of comments are saying they really like it, maybe it's worth doing a bit more exploration. And AI is commonly-used as basically a synonym for Machine Learning in the ML community. Yes, the word has multiple definitions and connotations, and some people prefer to avoid using the term AI at all because of that. But that doesn't meant that someone using the term AI to refer to ML is wrong or up to something. Furthermore, just because non-legitimate organizations want to ride the hype train, it doesn't mean legitimate ones shouldn't benefit from that effect as well. fast.ai is an organization actually writing software for doing machine learning, as opposed to a completely unrelated company (like an electric toothbrush company) claiming their latest product "uses AI" whatever that means. The funniest part is that the slogan for fast.ai "Making neural nets uncool again". If that isn't explicitly going against the hype and saying this is _actual_ neural nets and not just fluff, then I don't know what is.
- zeptomu 8y ago
  > I mean, _just_ basing your impression off of the name is not going to get you very far. There are so many neural networks libraries now and I don't find the time to test all of them - dismissing libraries that use .ai in their website's TLD (or in their name) served me very well so far. Sure if it really makes neural networks that much simpler to train, I will take a look, but for now it does not seem to be more high-level than Keras or PyTorch. It does not matter if you can describe your network in 50, 100 or 200 lines of code as the hard part is to make it learn, choose hyper parameters, change loss functions, etc. - this is possible in all frameworks. And I don't think AI is used as a synonym for machine learning in the community, but is used to describe AGI.

denfromufa 8y ago

@jph00 where do I find lstm/gru/seq2seq layers for time-series sequence predictions (not text)? Also interested in autoencoder implementations. The fast.ai docs search does not really work for this. What do you think about other notable APIs built on top of pytorch such as Pyro and AllenNLP?

sharcerer 8y ago

Hi Jeremy, I am a beginner. Going to start your ML course,then follow up with DL- 1 & 2. My question is when will these 3 be updated to the new version of fastai. I am aware of the fellowship for DL-1, but there's a 12 hour difference. Also, I want to start with ML,so, I want to learn with the new version. Thanks On the side, I am also taking Andrew's coursera course for a theoretical grounding , Bengio and Goodfellow's Machine Learning Book ,and Hands-On Machine Learning with Tensorflow and Sci-kit( O'reilly book).