3 ms·
For Gorgonia it's the other way around. You build the graph by manually unrolling it. It'll be a big goddamn graph, which is why you subgraph it and run it. so
by chewxy 10y ago
For Gorgonia it's the other way around. You build the graph by manually unrolling it. It'll be a big goddamn graph, which is why you subgraph it and run it.
source: actually have various LSTMs running. Some with attention, some with ADHD
- PeterisP 10y agoIs performance acceptable if you're required to re-build/adjust the graph for every single minibatch, because the number of unrolled items is different every time? That is what I mean by "Manual unrolling fails as soon as the number of repetitions is not fixed"; you can do manual unrolling in every framework but generally repeated graph building makes it unusably slow if you're unable to do the loop "in the system" with a runtime-chosen number of repetitions. Padding is also not really a solution, since your average sequence length is likely 5 or 10 times less than the maximum sequence length that you want to support, so just padding to a fixed size will mean 5-10 times slower processing.