> - Memory use aside, forward pass only is potentially a big training speed increase. Forward-pass only doesn't mean it's faster. It converges much slower than

> - Memory use aside, forward pass only is potentially a big training speed increase. Forward-pass only doesn't mean it's faster. It converges much slower than even SGD. It is a memory-time tradeoff, but that's it.