3 ms·
But to your point - that is how I feel about graph nns vs transformers or the fully connected set (GPUs are so good at transformers and fully connected nns, eve
by wrsh07 2y ago
But to your point - that is how I feel about graph nns vs transformers or the fully connected set (GPUs are so good at transformers and fully connected nns, even if there is a structure that makes sense we don't have the hardware to have it make sense.... Unless grok makes it cheap??)
- sdenton4 2y agoPerhaps; in a lot of cases the architecture barely matters. Transformers took a lot of extra tricks to get working well; the ConvNext paper showed that applying those same tricks to convolutional networks can fully close the gap. https://arxiv.org/abs/2201.03545 https://arxiv.org/abs/2201.03545