5 ms·
Ask HN: New Neural Network Primitives?
Hey Hackernews,
I’m completing my ms in cs and also am doing some ml research, and somebody recently told me an investigation into neural network primitives, I.e different cell types like LSTM cells, would be an interesting PhD dissertation.
What are some seminal papers and work done in examining different neural network primitives and cell types? Also has there been any work in treating neural networks as hypergraphs?
- p1esk 7y agoLook into capsules, also [1-3] can give you an idea of what can be explored. Evolving NNs [4] would probably benefit the most from NN primitives research. IMO, the best source of ideas for novel primitives is neocortex [5, 6]. [1] https://arxiv.org/abs/1904.03955 https://arxiv.org/abs/1904.03955 [2] https://arxiv.org/abs/1811.11168 https://arxiv.org/abs/1811.11168 [3] https://arxiv.org/abs/1905.02961 https://arxiv.org/abs/1905.02961 [4] https://www.nature.com/articles/s42256-018-0006-z https://www.nature.com/articles/s42256-018-0006-z [5] https://arxiv.org/abs/1610.00161 https://arxiv.org/abs/1610.00161 [6] https://arxiv.org/abs/1511.00083 https://arxiv.org/abs/1511.00083
- hsikka 7y agoThis is enormously helpful, thank you!