Note from the open review of that paper. > As discussed in the paper and pointed out by the reviewer, the growing memory module is non-differentiable, and so i

Note from the open review of that paper. > As discussed in the paper and pointed out by the reviewer, the growing memory module is non-differentiable, and so it cannot be trained directly by SGD. We acknowledge this observation. Two stack FSA/RNN are interesting, but as of now, not usable in practice.