14 ms·
Tensorflow 2.0: models migration and new design
- qnsi 8y agoFeels bad man. All tutorials will be outdated.
- friendshaver 8y agoThis is a great development for both new and experienced users. Most importantly, it seems to be an effort to introduce a canonical way to build a given network in Tensorflow, as opposed to the previous era where any given TF repo might implement the same network in one of several completely different ways (tf.contrib.slim, tf.layers, etc). Hopefully, defining a standard way to build models will also help accelerate the standardization of train/test scripts and data pipelines, putting to rest the era of models being deeply infected with the execution structure built around them. Overall, it looks like Pytorch is forcing TF devs to focus more on users and usability, and I'm excited to see how they continue to spur each other's growth.
- p1esk 8y agoeffort to introduce a canonical way I'm pretty sure there will be plenty of incompatible API changes in every 2.* release, and a completely new "canonical" way introduced in TF 3.0.
- adev_ 8y agoBy the sake of god, please try to add also a more friendly build system too. Bazel is a disaster that would almost make maven looks lightweight, npm reliable and autotools userfriendly.
- Judgmentality 8y agoWhat do you dislike about bazel? Not championing it, just curious.
- adev_ 8y agoThe list would be too long but out of order : - Using a JVM to compile mainly C++ and python, implying to deploy a JDK/JRE just for that. - Awful RAM consumption, compiling tensorflow takes 16GB of RAM to create a python wheel - Try to download the world and compile everything without allowing to specify external dependencies. - Compile time that are crazy long due to previous reason. - Invasive. Make very hard to integrate with anything else that does not build with Bazel - Make very hard, or sometimes, impossible to tune compiler flags - Just not reproducible. - Unstable, try to build a recent Google package with a 6 month old Bazel and good luck. I strongly advise you to watch this video of the last FOSDEM : https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers_cry/ https://archive.fosdem.org/2018/schedule/event/how_to_make_p...
- haberman 8y agoI strongly agree with you on the first one. Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times. I do see Bazel seems to support depending on a prebuilt .so, though I have not tried this: https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#... > Invasive. Make very hard to integrate with anything else that does not build with Bazel I think your main options here are: 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#...) 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/master/bazel/third_party https://github.com/googlecartographer/cartographer/tree/mast... > Make very hard, or sometimes, impossible to tune compiler flags "bazel --copt=<your options here> :target"? > Just not reproducible. What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there. Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need.
- solomatov 8y agoBazel is the best system which I have ever used.
- adev_ 8y agoLucky you, I never met a user outside of Google that like it.
- solomatov 8y agoI am not a Google employee. There're several reasons why it worked so well for us: - It can take advantage of many cores which you have in your computers. - It's polyglot. We have several languages in our project, and mixing several build systems were a nightmare. - It's correct. Very rarely you really need to run clean builds. It has its learning curve, but once you got it, you start to love it.
- curiousgal 8y ago>it can take advantage of many cores which you have in your computers. Which is a bad thing when you're on a shared server. I remember it wasn't straightforward to limit the number of cores it utilizes and it even ignored the flag I passed it. I eventually resorted to waiting until the weekend when no one in the lab was running jobs.
- maccam94 8y agoOuch. For a workaround in the future, maybe you can lower the priority of your bazel server via nice/renice?
- grandmczeb 8y agoDoes “--jobs N” not work?
- jmmv 8y agoUnfortunately not. "--jobs N" limits the parallelism during the execution phase (when Bazel executes compilers, etc.) but doesn't limit the parallelism of the analysis phase (when Bazel loads BUILD files and constructs the build graph). For the latter there is a "--loading_phase_threads N" flag. We are actively trying to make Bazel better use the local resources, and also clean up the current mess that the flags are in this area. See the discussion at https://groups.google.com/d/msg/bazel-dev/7pcrJY6Xo78/JDe256-tBQAJ https://groups.google.com/d/msg/bazel-dev/7pcrJY6Xo78/JDe256... for some context.
- nshm 8y agoHow about making it more stable? Kind of tired to close 10 issues a day on github because tensorflow developers decided to rename a variable. Seriously looking on pytorch.
- juanuys 8y agoStable, and complete: https://github.com/tensorflow/tensorflow/issues/14248 https://github.com/tensorflow/tensorflow/issues/14248
- amelius 8y agoIt still uses dataflow graphs :( As a programmer, I don't want to think in terms of dataflow graphs. That is what compilers are for!
- me2too 8y agoAt your contrary, I love the graph. However, migrating to a keras-like approach, you can work thinking about "objects with variables inside" and then the graph will be built for you by the abstraction introduced by `tf.keras.Model`. However, for automatic differentiation, graph is always required (as you can see from the example that uses eager)
- amelius 8y ago> At your contrary, I love the graph. Can you explain what you like so much about manipulating graphs, over just writing statements like z= x⋆y, where ⋆ is some tensor operation? Really, the graph makes me feel like I'm stacking Lego bricks with chopsticks, rather than with my bare hands ;)
- me2too 8y agoThe fact that I have all the computation described in a coherent manner, in something that's agnostic to the language I'm using. The same description (the graph) can be taken and used in every other language. I can move a trained model to production just picking a single file (a `tf.train.SavedModel` IIRC), give it a tag and I'm ready to go, with different models with a native support for the "tagging" (hence the model versioning).
- amelius 8y agoWhether that's an advantage depends on your perspective, because what has really happened is that you have now created a new language inside the original language.
- CMCDragonkai 8y agoA very common compiler architecture for functional languages is to use a combinatory language (dataflow like graphs) as the IR, and have a more usable frontend language that has variables and binders.
- maaaats 8y agoNot trying to start a language war, but I would like the old or the new api better if it had static types. Right now it's thousands of functions that can take thousands of different objects, and I would never know what to use except the few I randomly learned in a tutorial.
- me2too 8y agoBut the API is strictly statically typed in reality. The core is written in C++ and every operation requires a well-known type in input. As you can see from the error raised in this example: https://pgaleone.eu/tensorflow/go/2017/05/29/understanding-tensorflow-using-go/#question-time-1 https://pgaleone.eu/tensorflow/go/2017/05/29/understanding-t...
- halflings 8y agoThen you'll be happy to learn about Swift for Tensorflow: https://www.tensorflow.org/swift/ https://www.tensorflow.org/swift/ And no.. it's not simply a Swift wrapper for tensorflow, it's literally Tensorflow (or generally speaking: differentiable calculation graphs) built into the Swift language.
- lostmsu 8y agoMy company is working on a full API .NET binding. Leave your email if you're interested in trying the first technical preview in about 2 months.
- kodablah 8y ago> Support for more platforms and languages, and improved compatibility and parity between these components via standardization on exchange formats and alignment of APIs. Does this mean we can get a pure C API for training and running? Right now, IIRC you can only train in Python and C++ which ignores a large part of the programming community. If not a pure C FFI API, what is the approach being taken?
- nine_k 8y agoHow hard would it be to provide and maintain a C++-to-pure-C API adapter? Serious question.
- kodablah 8y agoI am not sure as I haven't done it, but this has been attempted at a generic level in many languages. E.g. there are multiple cpp-to-rust bridges. That's essentially how most Qt bindings work. Most of the trouble is probably around inheritance, visibility, etc. Well, that and the "provide and maintain [...] adapter" outside of the core supporting library authors has its own upkeep challenges.
- danieldk 8y agoThe C API permits training and running. But you have to define and serialize the graph in Python (or C++). (Of course, you could also write Tensorflow protobuf directly, but that would be tedious.)
- kodablah 8y agoPardon my naivete, so at some point when not reusing others' graphs/models, to get the most out of TF you are basically forced to use one of those two languages? With TF 2 claiming more platforms/languages, will this no longer be the case?
- CretinDesAlpes 8y agoI've used tensorflow in 2016-2018 and switched to Pytorch a few months ago. My theory is that TF is used mainly because it is supported by Google, even if it is really badly designed for practitioners. In fact, TF was not meant to be public but was at first some internal tool at Google. This would explain why people at Google itself started to develop Keras or Sonnet (deepmind) at some point. For anyone interested in research in deep learning with a python-numpy like environment, I can only recommend to switch.
- me2too 8y agoBut why? I do use Tensorflow form the 0.x release and I'm still using it right now, migrating all the codebase to the 2.x version. I can do everything in both pytorch and tensorflow and when I have to define really efficient input pipelines (tf.data is a great thing), parallelize and distribute the training and export a trained model to production... With tensorflow everything is easier. Moreover, pytorch in 1.x will have static-graph too, exactly like Tensorflow. Both frameworks are converging to something really similar. I don't see a reason to switch (right now)
- CretinDesAlpes 8y agoAt least 1) debugging in a numpy-like env and 2) multi GPU training on a single machine with multiple GPU (last time I was using TF it was such a nightmare that people had to use horovod, a wrapper library developed by... Uber).
- me2too 8y agoPoint 1 is a good point, although tfdebug exists and it works. For point 2: I don't see the problem. I use almost daily a wrapper (defined inside tf.contrib, that in version 2.x will go in core [I hope]) around the optimizer that in 2 lines allows me to distribute the training on multiple GPUs on the same machine
- lostmsu 8y ago
- ericfriday 8y agoThis reminds me Keras's creator wrote a blog on how to design APIs for the best user experience: https://blog.keras.io/user-experience-design-for-apis.html https://blog.keras.io/user-experience-design-for-apis.html
- LeanderK 8y ago> Tensorflow 2.0 will be a major milestone for the most popular machine learning framework: lots of changes are coming, and all with the aim of making ML accessible to everyone. These changes, however, requires for the old users to completely re-learn how to use the framework I never felt like I was not re-learning tensorflow! A constant series of breackage, deprecations, new apis etc.
- sebazzz 8y agoYou should probably employ machine learning with TensorFlow to automatically update your broken code and remove usage of deprecated APIs.
- aerovistae 8y agopriceless.
- FridgeSeal 8y agoNot to mention terrible documentation. I think that’s a property of Google though, because there’s never been a google service for which I’ve read the documentation and not been like “I do not understand what’s going on at all” for a non-trivial amount of time. And all their examples for Python are just like “run this magic-code-ridden python file, congrats you did the tutorial”.
- davmar 8y agoas if it wasn't already confusing enough. should i use tensorflow hub? TF slim? keras applications that import tensorflow? should i still use those outdates TF slim models that are for some reason kept in models/inception? please, tensorflow team, make this stuff easier for us non-Phds.
- deleted 8y ago[deleted]
- etaioinshrdlu 8y agoPhds have as much trouble as you.
- breatheoften 8y agoI think they should really consider making a clean JavaScript/typescript api first — use that and test for awhile - then port that design to python for a 2.0 of the python api... The worst thing about tensorflow is being locked into having to use python for graph description and training (in my opinion) ...
- mark_l_watson 8y agoTry TensorFlow.js - very nice, lots of examples included, easy to set up.
- tmulc18 8y ago2.5 years of learning tensorflow now gone to waste :(
- beefsack 8y agoTangential, but as someone in the southern hemisphere it's frustrating when people use seasons for timelines. "Q2 2019" is immediately more understandable instead of having to do some mental gymnastics to convert it.
- sytelus 8y agoThis: https://twitter.com/fchollet/status/1052228463300493312 https://twitter.com/fchollet/status/1052228463300493312
- xvilka 8y agoHopefully this time it will support AMD opensource drivers. It is awful TF now tied up to proprietary NVIDIA drivers.
- shimnasuresh 8y agoTensorflow 2.0 will be a major milestone for the most popular machine learning framework: lots of changes are coming, and all with the aim of making ML accessible to everyone. https://www.yarddiant.com/wordpress-development/ https://www.yarddiant.com/wordpress-development/ https://plus.google.com/+Yarddiant-Kozhikode https://plus.google.com/+Yarddiant-Kozhikode https://medium.com/@yarddiant https://medium.com/@yarddiant