4 ms·

A super inexperienced observer here, so just a very basic question - how does Caffe(2) differ from TF/Theano/Torch etc.? What are the obvious upsides and potent

by drej 10y ago

A super inexperienced observer here, so just a very basic question - how does Caffe(2) differ from TF/Theano/Torch etc.? What are the obvious upsides and potential downsides? A tweet-sized answer will do, I'm just curious as to what the high level differences are. Cheers and congrats on the release!

smhx 10y ago
http://caffe2.ai/docs/caffe-migration.html#null__how-is-caffe2-different-from-pytorch http://caffe2.ai/docs/caffe-migration.html#null__how-is-caff... > Caffe2 is built to excel at mobile and at large scale deployments. While it is new in Caffe2 to support multi-GPU, bringing Torch and Caffe2 together with the same level of GPU support, Caffe2 is built to excel at utilizing both multiple GPUs on a single-host and multiple hosts with GPUs. PyTorch is great for research, experimentation and trying out exotic neural networks, while Caffe2 is headed towards supporting more industrial-strength applications with a heavy focus on mobile. This is not to say that PyTorch doesn’t do mobile or doesn’t scale or that you can’t use Caffe2 with some awesome new paradigm of neural network, we’re just highlighting some of the current characteristics and directions for these two projects. We plan to have plenty of interoperability and methods of converting back and forth so you can experience the best of both worlds.
- mtw 10y ago
  does it mean PyTorch or Tensorflow are better for research, experimentation and Caffe2 was more designed for production? Or it actually means Caffe2 was designed for both research and production
  - jiayq84 10y ago
    PyTorch definitely makes experimentation much better. For example, if you want to train some system that is highly dynamic (reinforcement learning, for example), you might want to use a real scripting language which is Python, and PyTorch makes that really sweet. Sometimes the line gets a bit blurred - for research that are focusing on relatively fixed patterns, such as Mask RCNN, both PyTorch and caffe2 are working great. In fact, Mask RCNN is trained in Caffe2, and that also makes things much easy when we put it on mobile - what our CTO Mike Schroepfer showed in his keynote is a Mask RCNN model trained and then deployed onto mobile with Caffe2.
    - binaryzeitgeist 10y ago
      When are you planning to release the Mask-RCNN code? I'm trying to implement the RoIAlign layer in Tensorflow and I've a few doubts and having the author's code would definitely help in implementing it.
- anand_MSFT 10y ago
  We launched a Data Science VM supporting caffe2 - https://blogs.technet.microsoft.com/machinelearning/2017/04/18/deep-learning-with-caffe2-on-the-azure-data-science-virtual-machine/ https://blogs.technet.microsoft.com/machinelearning/2017/04/...
danielmorozoff 10y ago
Would also love some information on performance. How does Caffe2 stack up against TF1.0+? When is MKL support planned for ?
- jiayq84 10y ago
  MKL is going to be merged very soon - a full MKL pull request is here if you'd like to track it: https://github.com/caffe2/caffe2/pull/270 https://github.com/caffe2/caffe2/pull/270 And thanks so much for the awesome work from Intel!
- jiayq84 10y ago
  RE the performance - many frameworks nowadays enjoy the high-performance libraries such as CuDNN for optimized runtime on different platforms. For example, we've been collaborating with NVidia on optimized distributed training on GPUs, and you can check out more details here: https://blogs.nvidia.com/blog/2017/04/18/caffe2/ https://blogs.nvidia.com/blog/2017/04/18/caffe2/
  - danielmorozoff 10y ago
    Thanks for the link. Are there any public specifics comparing distributed tf to Caffe2's distributed implementation?
    - jiayq84 10y ago
      I am not aware of that right now. Internally we measure the performance by looking at the theoretical peak possible (like the scaling efficiency when you use distributed training). C2 has been doing pretty well; if you find performance degradations, send an issue to us and we are usually very good to figure out the perf.

by drej 10y ago

smhx 10y ago

http://caffe2.ai/docs/caffe-migration.html#null__how-is-caffe2-different-from-pytorch http://caffe2.ai/docs/caffe-migration.html#null__how-is-caff... > Caffe2 is built to excel at mobile and at large scale deployments. While it is new in Caffe2 to support multi-GPU, bringing Torch and Caffe2 together with the same level of GPU support, Caffe2 is built to excel at utilizing both multiple GPUs on a single-host and multiple hosts with GPUs. PyTorch is great for research, experimentation and trying out exotic neural networks, while Caffe2 is headed towards supporting more industrial-strength applications with a heavy focus on mobile. This is not to say that PyTorch doesn’t do mobile or doesn’t scale or that you can’t use Caffe2 with some awesome new paradigm of neural network, we’re just highlighting some of the current characteristics and directions for these two projects. We plan to have plenty of interoperability and methods of converting back and forth so you can experience the best of both worlds.

mtw 10y ago
does it mean PyTorch or Tensorflow are better for research, experimentation and Caffe2 was more designed for production? Or it actually means Caffe2 was designed for both research and production
- jiayq84 10y ago
  PyTorch definitely makes experimentation much better. For example, if you want to train some system that is highly dynamic (reinforcement learning, for example), you might want to use a real scripting language which is Python, and PyTorch makes that really sweet. Sometimes the line gets a bit blurred - for research that are focusing on relatively fixed patterns, such as Mask RCNN, both PyTorch and caffe2 are working great. In fact, Mask RCNN is trained in Caffe2, and that also makes things much easy when we put it on mobile - what our CTO Mike Schroepfer showed in his keynote is a Mask RCNN model trained and then deployed onto mobile with Caffe2.
  - binaryzeitgeist 10y ago
    When are you planning to release the Mask-RCNN code? I'm trying to implement the RoIAlign layer in Tensorflow and I've a few doubts and having the author's code would definitely help in implementing it.
anand_MSFT 10y ago
We launched a Data Science VM supporting caffe2 - https://blogs.technet.microsoft.com/machinelearning/2017/04/18/deep-learning-with-caffe2-on-the-azure-data-science-virtual-machine/ https://blogs.technet.microsoft.com/machinelearning/2017/04/...

danielmorozoff 10y ago

Would also love some information on performance. How does Caffe2 stack up against TF1.0+? When is MKL support planned for ?

jiayq84 10y ago
MKL is going to be merged very soon - a full MKL pull request is here if you'd like to track it: https://github.com/caffe2/caffe2/pull/270 https://github.com/caffe2/caffe2/pull/270 And thanks so much for the awesome work from Intel!
jiayq84 10y ago
RE the performance - many frameworks nowadays enjoy the high-performance libraries such as CuDNN for optimized runtime on different platforms. For example, we've been collaborating with NVidia on optimized distributed training on GPUs, and you can check out more details here: https://blogs.nvidia.com/blog/2017/04/18/caffe2/ https://blogs.nvidia.com/blog/2017/04/18/caffe2/
- danielmorozoff 10y ago
  Thanks for the link. Are there any public specifics comparing distributed tf to Caffe2's distributed implementation?
  - jiayq84 10y ago
    I am not aware of that right now. Internally we measure the performance by looking at the theoretical peak possible (like the scaling efficiency when you use distributed training). C2 has been doing pretty well; if you find performance degradations, send an issue to us and we are usually very good to figure out the perf.