6 ms·
This is kind of silly. Neural networks are universal function approximators, meaning any function "is" (more accurately, "can be represented by") a neural netw
by cochne 5y ago
This is kind of silly. Neural networks are universal function approximators, meaning any function "is" (more accurately, "can be represented by") a neural network. In the DFT case, it is a linear function so we could get an exact representation. Though there is also nothing stopping you from just saying, f(x) is a neural network, because I can choose my activation function to be f.
- TheRealPomax 5y agoIf silly but interesting things weren't allowed, there would be no point to having a website like this. The article is a neat exploration of two things that many, many people don't realise are related because they simply don't know the maths involved in either of the two subjects.
- vletal 5y agoMaybe, if the title wan not so overblown the pushback againt the articles premise would not be so hard.
- TheRealPomax 5y agoSorry, have you seen on the internet? There is no requirement of professionalism: write a catchy title, get people to read what you had to say. HN isn't an academic publishing track, it's a reflection of what the world actually posts on the internet. You're free to not like the titles some people use to get eyeballs on their content, but this was clearly not clickbait warranting removal, and it's certainly not HN's job to police page titles on the wider web: we just upvote the cool content, and if enough people like it, it hits the front page. So really, you're complaining that not enough people on HN mind catchy titles enough to downvote a post based on its cover, rather than the book it links to =)
- volta83 5y ago> Neural networks are universal function approximators, Do we have tight error bounds proofs for neural networks as approximators ?
- ska 5y agohttps://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- volta83 5y agoThat just says that it is possible to approximate functions well with neural networks of "sufficient depth" and "suitable weights". It doesn't tell you whether for a particular function f: - a particular network structure is suitable, - a trained network with particular weight value is suitable, and it obviously doesn't answer the most useful question: - given this network with this weights, what's the largest approximation error for this function f? There are many approximation methods in approximation theory that can answer this last question. Do we have these answers for neural networks?
- ska 5y ago> Do we have these answers for neural networks? Not in general that I'm aware of. I agree the result is not practical, but you asked what is known. There have been some refinements (the citations on that page give a reasonable flavor, fwiw).
- visarga 5y ago> Neural networks are universal function approximators What about discontinuous functions?
- estebarb 5y agoEvery function can be made continuous if you add one dimension in it's output for defined/not defined.
- hyperbovine 5y agoContinuous functions are dense in <pick your favorite function space> so yes.