3 ms·

Wait, (s)he’s arguing that the code isn’t imperative enough, but the punchline is they don’t like having to type session.run? I don’t understand their vendetta

Wait, (s)he’s arguing that the code isn’t imperative enough, but the punchline is they don’t like having to type session.run? I don’t understand their vendetta against the graph, which is a powerful abstraction that lets you choose different backend, and let’s tensorboard show you an awesome view of your computation. Session.run isn’t hard to type and it takes at most a few days to grok that everything is lazily evaluated. Would (s)he like eager apis? Does (s)he want better c++ apis? Or does the author just want to hate on tensorflow because it’s been hyped so much?

ffriend 9y ago
> I don’t understand their vendetta against the graph, which is a powerful abstraction that lets you choose different backend [...] You don't really need a graph to support different backends. One popular approach is to have different array implementations (e.g. CPU and GPU arrays). > [...] and let’s tensorboard show you an awesome view of your computation At the end of the post the author shows his API that lets you do the same things as Tensorboard, but for whatever framework you like. All in all, expression graphs like these used in TF and Theano are great for symbolic differentiation of a loss function and further expression optimization (e.g. simplification, operation fusion, etc.). But TF goes further and makes everything a node in a graph. Even things that are not algebraic expressions such as variable initialization or objective optimization.
- KirinDave 9y ago
  > You don't really need a graph to support different backends. One popular approach is to have different array implementations (e.g. CPU and GPU arrays). And now you can (waves arms) write it twice! Alternatively, you can make the interfaces between various impls be exactly the same but rename them so they're purpose-named. Then you've written Graph, for the most part.
  - candiodari 9y ago
    Except your graph is symbolic, and good luck getting a breakpoint to fire when the calculation is happening ... Or if you don't like debugging, the problem manifests itself with merely printing values too.
cjalmeida 9y ago
I believe the issue is that the code only actually runs when you calls session.run this makes debugging a nightmare. Sure TF can improve message, but it's not there yet. IMO, TF2 should make dynamic execution first class and switch easily to static graph when you need to deploy stuff.
- rasmi 9y ago
  TensorFlow is introducing eager evaluation: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/eager https://github.com/tensorflow/tensorflow/tree/master/tensorf... https://twitter.com/oshtim/status/879453382326353920 https://twitter.com/oshtim/status/879453382326353920
dnautics 9y ago
You don't even have to invoke session.run for frontends that can JIT the graph. Tensor flow in Julia is not declarative, for example.

ffriend 9y ago

> I don’t understand their vendetta against the graph, which is a powerful abstraction that lets you choose different backend [...] You don't really need a graph to support different backends. One popular approach is to have different array implementations (e.g. CPU and GPU arrays). > [...] and let’s tensorboard show you an awesome view of your computation At the end of the post the author shows his API that lets you do the same things as Tensorboard, but for whatever framework you like. All in all, expression graphs like these used in TF and Theano are great for symbolic differentiation of a loss function and further expression optimization (e.g. simplification, operation fusion, etc.). But TF goes further and makes everything a node in a graph. Even things that are not algebraic expressions such as variable initialization or objective optimization.

KirinDave 9y ago
> You don't really need a graph to support different backends. One popular approach is to have different array implementations (e.g. CPU and GPU arrays). And now you can (waves arms) write it twice! Alternatively, you can make the interfaces between various impls be exactly the same but rename them so they're purpose-named. Then you've written Graph, for the most part.
- candiodari 9y ago
  Except your graph is symbolic, and good luck getting a breakpoint to fire when the calculation is happening ... Or if you don't like debugging, the problem manifests itself with merely printing values too.

cjalmeida 9y ago

I believe the issue is that the code only actually runs when you calls session.run this makes debugging a nightmare. Sure TF can improve message, but it's not there yet. IMO, TF2 should make dynamic execution first class and switch easily to static graph when you need to deploy stuff.

rasmi 9y ago
TensorFlow is introducing eager evaluation: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/eager https://github.com/tensorflow/tensorflow/tree/master/tensorf... https://twitter.com/oshtim/status/879453382326353920 https://twitter.com/oshtim/status/879453382326353920

dnautics 9y ago

You don't even have to invoke session.run for frontends that can JIT the graph. Tensor flow in Julia is not declarative, for example.