3 ms·
They’re not, but the difference between the architecture outputting random garbage and impressive text is all in the weights.
by dartos 3y ago
They’re not, but the difference between the architecture outputting random garbage and impressive text is all in the weights.
- Nevermark 3y agoAnd a fitting architecture can save orders of magnitude in numbers of weight and training time. There isn’t one that’s more important than the other. There are no weights at all without an architecture. And any architecture improvements that pay off have a multiplier effect on efficiency. Nobody on Earth had the resources to get the performance they are getting on these big models with just a default fully connected layer architecture.