3 ms·
The number of neurons Hopfield networks can use seems like a serious limitation. I think the max number in his graphs was 300? Does anyone know if this is a rea
by epberry 10y ago
The number of neurons Hopfield networks can use seems like a serious limitation. I think the max number in his graphs was 300? Does anyone know if this is a real problem or not?
On another note, what is the state of the art in "'memory networks". I'm thinking beyond LSTMs to a network that might be able to remember high dimensional features across a large span of time.
- curuinor 10y agoUsually if you properly use GPU the real limit is on the order of 20k. I think the graph in his post is old as crap, actually (if you sparsify connections and do some tricks with sparsification sky's the limit, O(n * small multiplier over 1) with memory) Hopfield nets don't do memory like LSTMs do. LSTM's already sparsify features and memorize them a long time. There's a medium size literature on adding more shit (NTM, Pointer net, attention) but nearly all of it is not that good (it gets tested on bullshit)