3 ms·

And then your NN can't represent anything else than periodic functions.. If we had to build separate programs for each product requirements variations.. program

And then your NN can't represent anything else than periodic functions.. If we had to build separate programs for each product requirements variations.. programming would not be viable. More generally neural networks can't even imitate a dumb calculator without throwing absurd errors despite the rules of calculus being trivial and well defined. And matching a calculator is a task order of magnitudes easier than the semantic Causal reasoning abilities of human NLU that is involved in argumentation, inferences and understanding. But people are pathetically fooled by the fallacy of it being a uNiVeRsAl aPpRoXiMaTor.

telchar 5y ago
And yet neural networks can solve symbolic integral and derivative problems and differential equations better than other computer algebra programs. Sure one network might fail to compute sin(x) numerically, but another could easily tell you its derivative is cos(x). Turns out they are pretty flexible. Do they need to do everything?
- ABeeSea 5y ago
  Source? I would be very surprised if there was a neural symbolic PDE solver better than what’s in wolfram mathematica.
  - SemanticStrengh 5y ago
    Related https://arxiv.org/abs/1806.07366 https://arxiv.org/abs/1806.07366
- mhh__ 5y ago
  At the cost of how many parameters?
viraptor 5y ago
> And then your NN can't represent anything else than periodic functions Significant parts of x+sin are close to linear. You don't need to use this activation as the result layer either. Why would we lose anything? > If we had to build separate programs for each product requirements variations We pretty much do? We use both extremely generic frameworks both in technical sense (.net) and organisational (sap). But we also have software written to specific requirements where needed (there's millions of very specific ways to invoice someone, companies get invoicing platforms written just for them from scratch). There's space for both approaches.
PartiallyTyped 5y ago
You are wrong. https://www.vincentsitzmann.com/siren https://www.vincentsitzmann.com/siren
- SemanticStrengh 5y ago
  ? I did not say that NNs can't represent periodic functions. Show me a NN calculator with >99% accuracy if you wanna refute me.
  - PartiallyTyped 5y ago
    > and then your NN can't represent anything else than periodic functions.. This is a quote from the parent comment where you state that NNs with periodic activations can't represent non periodic functions. Siren (what I linked) uses periodic activations and is able to represent non periodic functions.
    - SemanticStrengh 5y ago
      Oh OK, thanks yes that's what I was looking for! However if this NN can represent non Periodic functions, what cost does the Periodic function support incur on non Periodic function accuracy, if any?
      - PartiallyTyped 5y ago
        It actually behaves a lot better than non periodic functions in some scenarios. It seems that period functions enable a network to encode positional information within [-1,1]. This is supported by the fact that positional encoding is needed in transformers to make them work.

telchar 5y ago

And yet neural networks can solve symbolic integral and derivative problems and differential equations better than other computer algebra programs. Sure one network might fail to compute sin(x) numerically, but another could easily tell you its derivative is cos(x). Turns out they are pretty flexible. Do they need to do everything?

ABeeSea 5y ago
Source? I would be very surprised if there was a neural symbolic PDE solver better than what’s in wolfram mathematica.
- SemanticStrengh 5y ago
  Related https://arxiv.org/abs/1806.07366 https://arxiv.org/abs/1806.07366
mhh__ 5y ago
At the cost of how many parameters?

viraptor 5y ago

> And then your NN can't represent anything else than periodic functions Significant parts of x+sin are close to linear. You don't need to use this activation as the result layer either. Why would we lose anything? > If we had to build separate programs for each product requirements variations We pretty much do? We use both extremely generic frameworks both in technical sense (.net) and organisational (sap). But we also have software written to specific requirements where needed (there's millions of very specific ways to invoice someone, companies get invoicing platforms written just for them from scratch). There's space for both approaches.

PartiallyTyped 5y ago

You are wrong. https://www.vincentsitzmann.com/siren https://www.vincentsitzmann.com/siren

SemanticStrengh 5y ago
? I did not say that NNs can't represent periodic functions. Show me a NN calculator with >99% accuracy if you wanna refute me.
- PartiallyTyped 5y ago
  > and then your NN can't represent anything else than periodic functions.. This is a quote from the parent comment where you state that NNs with periodic activations can't represent non periodic functions. Siren (what I linked) uses periodic activations and is able to represent non periodic functions.
  - SemanticStrengh 5y ago
    Oh OK, thanks yes that's what I was looking for! However if this NN can represent non Periodic functions, what cost does the Periodic function support incur on non Periodic function accuracy, if any?
    - PartiallyTyped 5y ago
      It actually behaves a lot better than non periodic functions in some scenarios. It seems that period functions enable a network to encode positional information within [-1,1]. This is supported by the fact that positional encoding is needed in transformers to make them work.