3 ms·
? I did not say that NNs can't represent periodic functions. Show me a NN calculator with >99% accuracy if you wanna refute me.
by SemanticStrengh 5y ago
? I did not say that NNs can't represent periodic functions.
Show me a NN calculator with >99% accuracy if you wanna refute me.
- PartiallyTyped 5y ago> and then your NN can't represent anything else than periodic functions.. This is a quote from the parent comment where you state that NNs with periodic activations can't represent non periodic functions. Siren (what I linked) uses periodic activations and is able to represent non periodic functions.
- SemanticStrengh 5y agoOh OK, thanks yes that's what I was looking for! However if this NN can represent non Periodic functions, what cost does the Periodic function support incur on non Periodic function accuracy, if any?
- PartiallyTyped 5y agoIt actually behaves a lot better than non periodic functions in some scenarios. It seems that period functions enable a network to encode positional information within [-1,1]. This is supported by the fact that positional encoding is needed in transformers to make them work.