2 ms·
You are largely wrong, to be honest. Mostly, you can simply stack more transformers and (as long as you have layer norm), it can train on more data (of fixed s
by whimsicalism 4y ago
You are largely wrong, to be honest.
Mostly, you can simply stack more transformers and (as long as you have layer norm), it can train on more data (of fixed sequence length).
There are some tricks & tips, but the vast majority lie outside of the architecture improvements.
- f6v 4y agoI’m not an expert, but I assume there’s a limit where «simply adding stuff” stops to provide improvements when you take training time into account. And my gut feeling is that true reasoning requires different architectures.