3 ms·
I'll go a little against Sutton here, though I mostly agree with him. Sometimes restriction in architecture has benefits. Not all dimensions are equal.
by godelski 4y ago
I'll go a little against Sutton here, though I mostly agree with him. Sometimes restriction in architecture has benefits. Not all dimensions are equal.
- panabee 4y agodino v2 reflipped me, reasoning that text supervision artificially constrains image learning: https://arxiv.org/abs/2304.07193 https://arxiv.org/abs/2304.07193 if unified modeling isn't necessary, or even hurtful, a key advantage of transformers goes away.