3 ms·

I've been wondering this since machine learning first became blew up in the 2010s and I guess this is as good a place ask to ask as any: is there some sort of l

I've been wondering this since machine learning first became blew up in the 2010s and I guess this is as good a place ask to ask as any: is there some sort of law/theory/conjecture/whatever that states that neural networks are algebraically incapable of modeling physical phenomena in general, or at least within the boundary conditions implied by the training data? For example, if the physical calculation includes imaginary numbers or some other exotic function. I'm thinking specifically of linear motors [1] which are capable of sub-micron positioning as long as the rotor and stators can be manufactured with enough precision. Is there some fundamental problem preventing us from using an interferometric laser encoder to train a neural network to control each linear motor coming off the assembly line, thus eliminating the much of the precision manufacturing? [1] https://en.wikipedia.org/wiki/Linear_motor https://en.wikipedia.org/wiki/Linear_motor

kilotaras 4y ago
Universal approximation theorem stays that neural nets can approximate any function[0] (size of that network may need to be arbitrarily big though). I'm not aware of any restrictions/limits on that. [0] https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- Retric 4y ago
  I believe it’s any continuous function, and training has it’s own limitations.
  - hansvm 4y ago
    Given that they can represent continuous functions you can lightly modify them to produce all sorts of interesting behavior like piecewise continuity as well. The training thing is important though. There are problems which are arbitrarily likely to have all training algorithms fail, not just in a practical sense, but as a fundamental limitation.
voldacar 4y ago
They can approximate any continuous map between finite dimensional vector spaces. Complex numbers are a finite dimensional vector space.
nonameiguess 4y ago
Neural networks can approximate the function fine. The issue here is reinforcement learning works by initially random or largely random action followed by feedback to get better. This means you fail over and over, possibly millions, possibly billions of time before you ever get good. If you're learning to play chess or go or Starcraft, that's fine. If you're learning to perform high-precision industrial process control, I'm not sure you can afford that much initial waste, and it may take centuries to get through enough iterations since the process does not take place entirely inside of a computer.

kilotaras 4y ago

Universal approximation theorem stays that neural nets can approximate any function[0] (size of that network may need to be arbitrarily big though). I'm not aware of any restrictions/limits on that. [0] https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...

Retric 4y ago
I believe it’s any continuous function, and training has it’s own limitations.
- hansvm 4y ago
  Given that they can represent continuous functions you can lightly modify them to produce all sorts of interesting behavior like piecewise continuity as well. The training thing is important though. There are problems which are arbitrarily likely to have all training algorithms fail, not just in a practical sense, but as a fundamental limitation.

voldacar 4y ago

They can approximate any continuous map between finite dimensional vector spaces. Complex numbers are a finite dimensional vector space.

nonameiguess 4y ago

Neural networks can approximate the function fine. The issue here is reinforcement learning works by initially random or largely random action followed by feedback to get better. This means you fail over and over, possibly millions, possibly billions of time before you ever get good. If you're learning to play chess or go or Starcraft, that's fine. If you're learning to perform high-precision industrial process control, I'm not sure you can afford that much initial waste, and it may take centuries to get through enough iterations since the process does not take place entirely inside of a computer.