My suspicion is that they will work great but mostly need time for compute scaling to kick in (as was basically the case for the self-attention). Computers will

My suspicion is that they will work great but mostly need time for compute scaling to kick in (as was basically the case for the self-attention). Computers will need to get asymptotically faster before they work out. Having said that, it's not my area and i'm mostly an uninformed enthusiast.