4 ms·
Mandatory plug: do consider using PyTorch instead. It's far easier to pick up and work with. Easy things are easy, hard things are possible.
by m0zg 6y ago
Mandatory plug: do consider using PyTorch instead. It's far easier to pick up and work with. Easy things are easy, hard things are possible.
- osipov 6y agoAgreed. I used TensorFlow 1.x before PyTorch matured. Today, all of my deep learning is PyTorch based because TensorFlow 2.x didn't fix the core TensorFlow 1.x issues (e.g., poor reproducibility, unintuitive API design) and PyTorch performs better than Keras w/TensorFlow backend.
- jiggunjer 6y agoDoes it train just as fast on gpus?
- comicjk 6y agoIn my experience, not quite as fast for fully-tuned code, but the difference is small - and given the same project deadline, the PyTorch version will probably be faster.
- m0zg 6y agoProperly tuned (DistributedDataParallel + mixed precision) it will train faster, and consume a lot less RAM, allowing you to use larger batches, and higher LR.
- ausjke 6y agoI read somewhere saying TF2 is good for production while Pytorch is good for research(and papers), is this true? I'm more interested in putting one of them into real products, esp standalone embedded devices.
- m0zg 6y agoIt used to be true, but nowadays there are several options to deploy Pytorch models: 1. PyTorch C++ API which can trivially load and execute your models exported via JIT 2. ONNX export and inference in TensorRT (highest performance inference option) 3. Or just deploy straight up PyTorch Python code - it'll run fine in "production". One place where PyTorch is weaker than TF is mobile. TFLite is a lot more mature and has all sorts of acceleration support (GPU, DSP). So if that's what you need, at this point there's really no other good choice IMO.
- 0-_-0 6y agoFor what it's worth, I've found Pytorch to be much more rigid than TF. Maybe I just haven't found the easy way to do things. For example here's a function that applies an N×N box filter to all but the first 2 dimensions of a tensor (apologies to mobile users): def boxfilter(image, N=3): shape = image.shape image = tf.reshape(image, [1, shape[0], shape[1], -1]) C = image.shape[-1] conv = tf.nn.conv2d(image, tf.eye(C, C, [N, N]), 1, "SAME")[0] return tf.reshape(conv, shape) / (N * N) Is there a simple way to do this in Pytorch? Preferably without having to inherit from the base class for convolution. It seems to me that Pytorch is like Keras and Tensorflow is like Numpy.
- apl 6y agoYou can almost 1:1 translate this by swapping "tf" and "torch". No need to use nn.Conv2d -- there's a functional API for all these layers: https://pytorch.org/docs/master/nn.functional.html#conv2d https://pytorch.org/docs/master/nn.functional.html#conv2d Torch doesn't have "same" padding, so you have to manually calculate the correct padding value for your input/output shapes.
- 0-_-0 6y agoThat looks great, thanks!