And a fitting architecture can save orders of magnitude in numbers of weight and training time. There isn’t one that’s more important than the other. There are

And a fitting architecture can save orders of magnitude in numbers of weight and training time. There isn’t one that’s more important than the other. There are no weights at all without an architecture. And any architecture improvements that pay off have a multiplier effect on efficiency. Nobody on Earth had the resources to get the performance they are getting on these big models with just a default fully connected layer architecture.