11 ms·
If you fleshed out the "trivial" point 2 as a proof, I think the result would be essentially the same as the article. The only way you can make it substantiall
by civilized 5y ago
If you fleshed out the "trivial" point 2 as a proof, I think the result would be essentially the same as the article.
The only way you can make it substantially simpler is if you use a neuron whose nonlinearity makes it essentially a restatement of another result. For example, if the neurons are basically just Haar wavelets.
- Dylan16807 5y ago"Divide x into a bunch of buckets. Make two neurons tied to Input that activate very sharply at the bottom and top edge of each bucket. Use those to make a neuron that activates when Input is in the bucket. Weight it so it adds f(x) to Output." That's over 100 times shorter than the article. The method isn't as elegant since it needs two internal layers but I think it's pretty clear. Is it wrong to say that the logical leap from "a neuron can go from 0 to 1 at a specific input value" to "neurons can make a lookup table" is trivial? Oh well. (With "go from 0 to 1 at a specific input value" being the nonlinear part.)
- civilized 5y agoI encourage you to search for "lookup table" in the article.
- Dylan16807 5y agoWhy? Yes, it says that. It's also 6000 words long. I'm saying it's not that hard. I'm not saying the article is wrong or anything, I'm saying you can get to the same result MUCH faster. "You can turn a neural net into a lookup table" should be easily understood by anyone that knows both of those concepts. Edit: Like, isn't triggering specific outputs on specific input conditions the first thing that's usually shown about neural nets? If not a full lookup table, that's at least 90% of one and you just need to combine the outputs.