5 ms·

It's not cheating considering it's not even possible otherwise.

• deleted 5y ago
  [deleted]
• j16sdiz 5y ago
  This is obviously true for non-linear neurons. Who need a proof for that?
  • civilized 5y ago
    Why do you think it's obvious? Can you spell it out? I hope it's more than "presumably, non-linear neurons can approximate any non-linear function since they both have non-linear in the name".
    • programmer_dude 5y ago
      All functions can be approximated by the Fourier transform. Look up the equation to see why it applies here (Hint: it is an integral of f(t)e^(t)).
      • civilized 5y ago
        This is about as far from a proof as we are from the Andromeda Galaxy.
        • kovacs_x 5y ago
          very fitting- https://earthsky.org/astronomy-essentials/earths-night-sky-milky-way-andromeda-merge/#:~:text=The%20Andromeda%20galaxy%20is%20currently,our%20galaxies%20is%20already%20underway https://earthsky.org/astronomy-essentials/earths-night-sky-m....
    • Dylan16807 5y ago
      The really simple proof I'd use is: 1. A function can be approximately implemented as a lookup table. 2. It's trivial to make a neural network act like a lookup table. Which seems to resemble the article but it's much simpler. Point 2 assumes the neurons are normal non-linear ones. I'm not saying that's cheating, but I do agree with it being pretty obvious, at least from the right angle.
      • civilized 5y ago
        If you fleshed out the "trivial" point 2 as a proof, I think the result would be essentially the same as the article. The only way you can make it substantially simpler is if you use a neuron whose nonlinearity makes it essentially a restatement of another result. For example, if the neurons are basically just Haar wavelets.
        • Dylan16807 5y ago
          "Divide x into a bunch of buckets. Make two neurons tied to Input that activate very sharply at the bottom and top edge of each bucket. Use those to make a neuron that activates when Input is in the bucket. Weight it so it adds f(x) to Output." That's over 100 times shorter than the article. The method isn't as elegant since it needs two internal layers but I think it's pretty clear. Is it wrong to say that the logical leap from "a neuron can go from 0 to 1 at a specific input value" to "neurons can make a lookup table" is trivial? Oh well. (With "go from 0 to 1 at a specific input value" being the nonlinear part.)
          • civilized 5y ago
            I encourage you to search for "lookup table" in the article.
            • Dylan16807 5y ago
              Why? Yes, it says that. It's also 6000 words long. I'm saying it's not that hard. I'm not saying the article is wrong or anything, I'm saying you can get to the same result MUCH faster. "You can turn a neural net into a lookup table" should be easily understood by anyone that knows both of those concepts. Edit: Like, isn't triggering specific outputs on specific input conditions the first thing that's usually shown about neural nets? If not a full lookup table, that's at least 90% of one and you just need to combine the outputs.
• anothernewdude 5y ago
  My proof that you can compute any function with a single neuron: 1. Use the function as the activation function.