3 ms·
It actually behaves a lot better than non periodic functions in some scenarios. It seems that period functions enable a network to encode positional information
by PartiallyTyped 4y ago
It actually behaves a lot better than non periodic functions in some scenarios.
It seems that period functions enable a network to encode positional information within [-1,1]. This is supported by the fact that positional encoding is needed in transformers to make them work.