4 ms·
Hi! Are these positional embeddings literally made by concatenating the patch embedding with a number, then passing that through the next layer, as suggested by
by plonk 4y ago
Hi! Are these positional embeddings literally made by concatenating the patch embedding with a number, then passing that through the next layer, as suggested by the figure under "Images to Patch Embeddings"?
It's the most confusing part of transformers for me. How do we train the module that creates these embeddings?