There are only a few organizational principles that allow the number of groups (K) to scale beyond the number of neurons (N). Polychronization (http://www.izhi

There are only a few organizational principles that allow the number of groups (K) to scale beyond the number of neurons (N). Polychronization (http://www.izhikevich.org/publications/spnet.pdf http://www.izhikevich.org/publications/spnet.pdf). This is e.g. different from synfire chains in which the number of groups equals the number of neurons. I don't remember much about the capacity analysis of standard reservoir computing methods (ESN, LSM, ELM), but I thought it was also limited to N. I'm looking forward to more research in this direction! We as humans build up such long-term memories. This must be a driving factor in AI. A lot of prediction tasks hinge on being able to handle this huge temporal scale. Currently I'm trying to see if I can formulate chains of transition matrices in MCMC that allow simultaneous search on multiple scales. I feel like the math from multiple angles is converging, exciting! We're gonna solve this in our lifetimes!