4 ms·

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the g

by belval 3y ago

There is a first mover handicap there though. TF1.0 included a bunch of things that were harder to understand like tf.Session(). PyTorch was inspired from the good parts and "we will eager-everything". Internally I'm sure there was a lot of debate in the TF team that culminated with TF2.0, but by that time the damage was done and people saw PyTorch as easier.

bitL 3y ago
I think the main problem was debugging tensors on the fly, impossible with TF/Keras, but completely natural to PyTorch. Most researchers needed to sequentially observe what is going on in tensors (histograms etc.) and even doing backprop for their newly constructed layers by hand and that was difficult with TF.
- mirker 3y ago
  Nah, TF has had dynamic execution since TF2 and it’s still losing users, it seems. The execution model and API is simply more complicated. What’s a session, placeholder, constant, tensor, …? PyTorch was sold as numpy with GPU support and it is pretty close to that. JAX is an attempt to approach language simplicity and purity.
disgruntledphd2 3y ago
Nope, Pytorch was inspired by the Lua version of Torch which well predates Tensor flow. To be fair, basically every other DL framework made the same mistake though. Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted.
- andyjohnson0 3y ago
  > Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted. I think this is a very important point, and I remember sweating blood trying to build a standalone tf environment (admittedly on windows) in the past. I'm impressed by how much simpler and smoother the process has recently become. I do prefer Keras to Pytorch though - but thats just me
- throwaway2037 3y ago
  tensorflow was a total nightmare to install while Pytorch was pretty straightforward Hat tip for this comment. On HN, I read some great commentary about "time to achieve first HTTP 200 with your REST API". Regarding installed software libraries, lower friction to achieve "Hello, World!" is important.
- mirker 3y ago
  PyTorch examples were also cleaner. torchvision had ResNet training batteries included while TF had role your own or clone some weird Keras repository.

by belval 3y ago

bitL 3y ago

I think the main problem was debugging tensors on the fly, impossible with TF/Keras, but completely natural to PyTorch. Most researchers needed to sequentially observe what is going on in tensors (histograms etc.) and even doing backprop for their newly constructed layers by hand and that was difficult with TF.

mirker 3y ago
Nah, TF has had dynamic execution since TF2 and it’s still losing users, it seems. The execution model and API is simply more complicated. What’s a session, placeholder, constant, tensor, …? PyTorch was sold as numpy with GPU support and it is pretty close to that. JAX is an attempt to approach language simplicity and purity.

disgruntledphd2 3y ago

Nope, Pytorch was inspired by the Lua version of Torch which well predates Tensor flow. To be fair, basically every other DL framework made the same mistake though. Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted.

andyjohnson0 3y ago
> Also, tensorflow was a total nightmare to install while Pytorch was pretty straightforward, which definitely shouldn't be discounted. I think this is a very important point, and I remember sweating blood trying to build a standalone tf environment (admittedly on windows) in the past. I'm impressed by how much simpler and smoother the process has recently become. I do prefer Keras to Pytorch though - but thats just me
throwaway2037 3y ago
tensorflow was a total nightmare to install while Pytorch was pretty straightforward Hat tip for this comment. On HN, I read some great commentary about "time to achieve first HTTP 200 with your REST API". Regarding installed software libraries, lower friction to achieve "Hello, World!" is important.
mirker 3y ago
PyTorch examples were also cleaner. torchvision had ResNet training batteries included while TF had role your own or clone some weird Keras repository.