The openai paper presents a way to learn sparsity - the block-sparse structure with blocks bigger than 1x1 has been chosen for efficiency reasons. You may as w

The openai paper presents a way to learn sparsity - the block-sparse structure with blocks bigger than 1x1 has been chosen for efficiency reasons. You may as well learn block-sparse architecture with 1x1 blocks, effectively doing what MIT was doing, but without two phases.