4 ms·
Did they figure out how to scale them? Because if not, we are working that out.
by adamnemecek 2y ago
Did they figure out how to scale them? Because if not, we are working that out.
- throwaway314155 2y agoMy suspicion is that they will work great but mostly need time for compute scaling to kick in (as was basically the case for the self-attention). Computers will need to get asymptotically faster before they work out. Having said that, it's not my area and i'm mostly an uninformed enthusiast.
- adamnemecek 2y agoWell we are working on scaling them.