3 ms·
Yes, I’ve tested this out. It does train, but the scaling doesn’t seem to pan out. It’ll perform slightly better than the number of trainable parameters, but ne
by buildbot 2y ago
Yes, I’ve tested this out. It does train, but the scaling doesn’t seem to pan out. It’ll perform slightly better than the number of trainable parameters, but never improves as you scale, so for now there’s no benefit.