3 ms·
Transformers are basically CNNs with fixed-sized convolutions per layer (called heads in the lingo). Whereas CNNs have different-sized convolutions at each lay
by lil_dispaches 5y ago
Transformers are basically CNNs with fixed-sized convolutions per layer (called heads in the lingo). Whereas CNNs have different-sized convolutions at each layer.
Instead of looking at things through many lenses, like a CNN, the Transformer cross-examines everything through the same lense (aka attention or focus).
Transformers-in-Transformers (TiTs) is, in effect, adding more CNN-like layers to each regular transformer layer.