Usually if you properly use GPU the real limit is on the order of 20k. I think the graph in his post is old as crap, actually (if you sparsify connections and

Usually if you properly use GPU the real limit is on the order of 20k. I think the graph in his post is old as crap, actually (if you sparsify connections and do some tricks with sparsification sky's the limit, O(n * small multiplier over 1) with memory) Hopfield nets don't do memory like LSTMs do. LSTM's already sparsify features and memorize them a long time. There's a medium size literature on adding more shit (NTM, Pointer net, attention) but nearly all of it is not that good (it gets tested on bullshit)