That all sounds reasonable. I was surprised at how much the batch size can matter on different hardware. Maybe you've seen this post: http://svail.github.io/rn

That all sounds reasonable. I was surprised at how much the batch size can matter on different hardware. Maybe you've seen this post: http://svail.github.io/rnn_perf/ http://svail.github.io/rnn_perf/ It's primarily RNN-focused, but the discussion about batch sizes on GPUs is interesting.