4 ms·
I don't think this always works if the function in question has infinitely many discontinuities...
by drdeca 11y ago
I don't think this always works if the function in question has infinitely many discontinuities...
- p1esk 11y agoWhat would be an example of a practical task that needs to be modeled with such a function?
- gizmo686 11y agoRounding
- brobdingnagian 11y agoI am a neural network, and I can round.
- rspeer 11y agoIf you lived in the 1800's, would you have described yourself as an automaton made of gears?
- brobdingnagian 11y agoPerhaps - it's not a bad analogy. But calling myself a neural network is not an analogy - it's literal.
- brobdingnagian 11y agoI find it very interesting that I'm being downvoted based on the difference between my self-concept and whoever is doing the downvoting (they don't think they are a neural network). I'm pretty sure that's not what downvotes are supposed to be used for. Perhaps those who are doing the downvoting would be interested in reading the textbook dedicated to the notion that human beings are neural networks, called Computational Cognitive Neuroscience: https://grey.colorado.edu/CompCogNeuro https://grey.colorado.edu/CompCogNeuro
- p1esk 11y agoI think you might be getting downvotes because the neural networks discussed in the article have nothing to do with the neural structures in a human brain, and it appears that you're oblivious to this fact.
- DanBC 11y agoStop talking about how downvotes should be used. It's tedious, and wrong. You aren't just getting downvotes from people who disagree with you (if that's why you're getting downvotes); you are failing to gain upvotes from anyone who either agrees with you or thinks your downvotes are unfair. People downvote to disagree - and there's nothing in any site faqs or guidelines to say this is wrong - but those votes are balanced by people upvoting unfairly downvoted posts.
- cLeEOGPw 11y agoRounding is something humans can't do either. how easily can you round a stick that is almost exactly 1.5 meters long? You would be ever sure that you are correct. And as for rounding in mathematical way, that wouldn't be a problem for neural network, since it is a large distance from 1.4999... and 1.5000... from neural network's perspective, just like it is for you.
- p1esk 11y agoGood example, but in this case it's actually very easy to find weights for the neural network that would do rounding with any desired precision (look for the "stack of towers" method in the article).
- im3w1l 11y agoWorks for all numbers except those ending in .5 And they are quite important to get right.
- gizmo686 11y agoYeah, I think I picked rounding because the article primed me to think about it (the proof presented is essentially based on the ability to create steps, and rounding is my go-to example of a step function). However, any attempt to construct such a network would have a finite amount of steps in it, while the actual step function would have infinite. In practice, if we needed such a network, we would probably have a restricted domain, so there is only a finite number of discontinuous, and there would be a sufficiently large network that could solve it to an arbitrary precision. We would not be able to do this with more exotic functions that are densly discontinues, such as the function: f(x) = 0 iff x is rational f(x) = 1 iff x is irrational There is no way a neural network can reasonably model that function.
- mistaken 11y agoAlso a single layer is not enough to approximate functions in higher dimensions.
- w0000t 11y agoThis is also addressed in the article. Reading the title is not enough, you have to read the article.
- w0000t 11y agoYou missed this paragraph, which occurs in the first 10% of the article. The second caveat is that the class of functions which can be approximated in the way described are the continuous functions. If a function is discontinuous, i.e., makes sudden, sharp jumps, then it won't in general be possible to approximate using a neural net.