3 ms·
> You can create graphs, compute losses, and backpropagate them on the fly Sure, but you'd have to define the weight variables for your conv layer first outsid
by bonoboTP 7y ago
> You can create graphs, compute losses, and backpropagate them on the fly
Sure, but you'd have to define the weight variables for your conv layer first outside the train loop, then use them within the train loop. With TF I can just create the weights and the ops in one line.
Also, constantly doing .cuda() and .cpu() is not much nicer than feed dicts and fetches.
The real advantage of pytorch is the debuggability, you can set a breakpoint and see every value.
- briggers 7y agoI think I understand your point about declaring and then later using your layers. Are you aware of the Sequential module? It allows you to chain together layers into a single variable, making this repetition disappear into a single forward/__call__ on the Sequential.
- bonoboTP 7y agoThanks, that does improve it for simple cases of pure chains of layers without any skip connections etc. Unfortunately it's most an issue with more complicated architectures. When testing different designs I often forget to modify it in both places and sure it's not a huge deal, but having one place or two does feel like a qualitative difference in cognitive burden. For example if I introduce a bool flag which decides whether I do a particular alternative design, I have to chech the flag and do an if/else in two places: once to create the layers, once to use them.