3 ms·
I believe the other big thing is it allowed you to parallelize the training as well, so you could split those colossal training sets across many machines in a w
by saynay 3y ago
I believe the other big thing is it allowed you to parallelize the training as well, so you could split those colossal training sets across many machines in a way you could not do with LSTMs.
- throwawaymaths 3y agoI don't really believe this because in theory it's possible to split up training of arbitrary neural nets using decoupled neural interfaces (not that it's easy mind you) but at the point where you have openAI scale resources the bottleneck to achieving that is the difficulty of Python.
- golly_ned 3y agoWhat’s a neural interface?
- throwawaymaths 3y agohttps://arxiv.org/abs/1608.05343 https://arxiv.org/abs/1608.05343