9 ms·
As someone who uses tensorflow a lot, I predict an enormous clusterfuck of a transition. Tensorflow has turned into a multiheaded monster, supporting many thing
by 317070 7y ago
As someone who uses tensorflow a lot, I predict an enormous clusterfuck of a transition. Tensorflow has turned into a multiheaded monster, supporting many things and approaches but none of them very well.
I mean, when relying on third party code, things like `tf.enable_control_flow_v2() and tf.disable_control_flow_v2()` can and will go horribly wrong. It looks like some operations are changing behaviour depending on a global flag being set. And not just some operations, but control flow operations! That will lead to some very hard to figure out bugs.
In my opinion there are some architectural problems with TF, which have not been adressed in this update. There is still global state in TF2. There is still a difference in behaviour between eager and non-eager mode. There is still the control flow as a second class citizen.
If you need to transition from TF1 to TF2, consider doing the TF1 to pytorch transition instead.
- akhilcacharya 7y agoI'm still not sure what the difference between tf.keras and the other keras repo is! And that's before Sonnet, the Estimator API (?) and TFLearn (I'm probably forgetting a bunch).
- minimaxir 7y agoThe intent of having separate repos for tf.keras and keras was to support third-party platforms in the latter. But since the supported third-party platforms (CNTK and Theano) have stopped development due to TensorFlow, well...
- ghop02 7y ago> This is also the last major release of multi-backend Keras. Going forward, we recommend that users consider switching their Keras code to tf.keras in TensorFlow 2.0 https://github.com/keras-team/keras/releases/tag/2.3.0 https://github.com/keras-team/keras/releases/tag/2.3.0
- jlebar 7y ago> If you need to transition from TF1 to TF2, consider doing the TF1 to pytorch transition instead. Or consider https://github.com/google/jax https://github.com/google/jax !
- applecrazy 7y agoJust looked at Jax. At first glance, it seems to be a GPU/TPU based NumPy? The thing is, TF has more than tensor ops. It has pre-defined NN layers, data loading/serialization, distributed training, metrics, and model serving. It seems like a bit of a step backwards, that's all. Edit: "matrix ops" -> "tensor ops"
- jlebar 7y ago> At first glance, it seems to be a GPU/TPU based NumPy? Yes, with a compiler to make this fast. > The thing is, TF has more than tensor ops. It has pre-defined NN layers, data loading/serialization, distributed training, metrics, and model serving. Yes, it is a simpler and smaller API. For things like data loading, you can use the tool of your choice -- TF, pytorch, whatever. For pre-defined NN layers, there are libraries that build this as a very thin wrapper around JAX's low-level API, see e.g. lax, which is include in JAX.
- nmca 7y agoHave you ever used any of those features of tensorflow though? They're all, er, idiosyncratic. If you're a decent software engineer and are following the mafs well from a book, I couldn't reccomend Jax highly enough. (I work on big tf RL projects every day).
- applecrazy 7y agoThat’s true. I’m still a student and haven’t really shipped a large ML project. Most of my ETL is done using scrapers, manual Pandas transformations, and storage in flat files. But yeah, I see your point, and frankly I don’t think there’s too big of a user base around these specialized features. Not to mention that some can be, erm, difficult to use. A friend tried using the YouTube-8M dataset in TFRecords format for a project and was extremely annoyed at the complexity.
- option 7y ago»If you need to transition from TF1 to TF2, consider doing the TF1 to pytorch transition instead.» - that’s exactly what we did and we don’t regret that decision
- applecrazy 7y agoI'm actually looking to learn PyTorch. Where's the best place to start?
- travisporter 7y agoPytorch website has a “blitz” tutorial that was fantastic.
- rm_-rf_slash 7y agoIf you have any existing ML/DL experience, picking up PyTorch is a breeze. You could get a pretty solid understanding of the framework with an MNIST handwritten digit recognition model over the course of an afternoon, so don’t sweat looking for the “right” tutorial.
- applecrazy 7y agoAwesome, I've done some ML before. I'll just dive into the docs.
- mitchellgoffpc 7y agoHighly recommend it! I love pytorch so much, it's basically numpy with automatic backprop and CUDA support. It evaluates eagerly by default, which makes debugging a lot easier since you can just print your tensors, and IMO it's much simpler to jump between high-level and low-level details in pytorch than in tensorflow+keras. Just as one example, activation functions in pytorch are applied by calling a python function on your layer, instead of passing a string argument with the function name to the layer constructor, so you write layer = F.relu(Linear()(input)) instead of layer = Dense(activation_fn='relu')(input) As a result, it's a lot more straightforward to try out custom activation functions in PyTorch.
- mitchellgoffpc 7y agoAgreed. Tensorflow kinda reminds me of OpenGL in that its dependence on global flags causes some really annoying bugs, especially when you're using third-party libraries or pre-trained models. `enable_eager_execution`, `enable_tensor_equality`, and `enable_v2_tensorshape` have all completely broken my code at one point or another.
- StreamBright 7y agoSorry maybe a stupid question but would not be Mxnet better for what you do in TF?
- alfalfasprout 7y agoAgreed, MxNet is faster and considerably easier to use than TF.
- solidasparagus 7y agoThere's absolutely no evidence that MXNet is faster than TF. At the high end, all three (TF/PyTorch/MXNet) are similarly performant. The reality is that implementation matters more than framework when you are talking about performance.
- StreamBright 7y agoMy question was more about usability or interchangeability. Could TF get replaced by Mxnet in a typical deep learning project?
- solidasparagus 7y agoIn terms of features and functionality, they are very much interchangeable. A new project could be written in any of the three major frameworks and be equally good. The only standout feature I'm aware of is that TF has the best support for doing inference on devices, but that won't be true forever. In terms of actually migrating a codebase from one to the other, the APIs are different enough in small ways that it would be a large amount of effort.
- XCSme 7y agoNot only upgrading is hard, but also installation (on Windows at least). For each Tensorflow version you need a specific python version, a specific CUDA version, specific tensorflow-gpu version, and many other easy to get wrong things. The problem is not the requirements, but that it's very hard to know what versions are compatible. There are endless threads on Github of people trying to use Tensorflow but failing after spending days trying to install it.
- nicklarsennz 7y agoI've been using containers for this, and mounting the source code.
- jacquesm 7y agoI've been using Anaconda and it finally made Python work without endless rounds of 'incompatible libary bingo'. It also works well for machine learning projects.
- tudelo 7y ago+1 for Anaconda, worth trying for anyone who has a problem with package versions.
- echelon 7y agoAs a software engineer and non-data scientist, I hate Anaconda because it feels like it's a tool that tries to be the be-all, end-all package management tool for everyone in the data science field, yet it feels like a sloppily built, bloated whale. It's even managed to overwrite PATH on some of my Linux machines, which is where I drew the line. I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead. Sadly, many of the ML models I investigate on Github don't even have their package requirements frozen. It's an uphill battle...
- 7y ago
- rch 7y agoIs the friction a result of trying to support eager execution?
- vl 7y agoThe problem is that tensorflow is an umbrella name for bunch of related technologies: it's a matrix calculation engine, graph definition language, distributed graph calculation engine, ML algorithmic libraries, ML training libraries. On top of that it's extremely poorly documented. At the end of the day when you use it anything beyond most trivial stuff turns out to be incompatible with each other (this operation is not implemented for TPUs or GPUs, this API doesn't work with this API) and most of development cut-n-paste trial and error. Then you go to read it at source, but creative Python renaming and importing leads you to multi-hour wild goose chase. If you switch to PyTorch, what are you going to use for prod deployment? Is there any way to use TPUs?
- smhx 7y ago> If you switch to PyTorch, what are you going to use for prod deployment? Is there any way to use TPUs? PyTorch has an optional XLA device, that let's you use TPUs: https://github.com/pytorch/xla https://github.com/pytorch/xla
- tastyminerals 7y agoCannot but agree. Above that 1.14 documentation was simply deleted from the tensorflow website and now we are scratching our heads at what to do when it comes to model maintenance. We serve our models via TF Java API since our system is Java/Scala based. We can't even update existing TF Java API because it is incompatible with anything prior to 1.15. It's an utter mess.