3 ms·
We did (and engineered) chemistry long before we understood why chemistry works the way it does; we built airplanes long before understanding how a wing actuall
by PeterisP 10y ago
We did (and engineered) chemistry long before we understood why chemistry works the way it does; we built airplanes long before understanding how a wing actually works (I mean, wing and propeller shapes were derived experimentally in wind tubes, not because of solid reasoning why).
If there is proven science, sure, an engineer should base the design on it. But if there isn't, that shouldn't prevent trying to get a workable solution before you can get proven science - if anything, the practical experience of your working (and non-working) solutions might be required to make the "why" questions answerable.
I mean, right now we have a boom of building things with neural networks despite the lack of proven science on guarantees and reasons why a deep NN (often with much more parameters/degrees of freedom than training samples) must generalize. We have "rules of thumb", algorithms and tricks that tend to result in state of art results, we have practical comparisons between them, but little in the way of proofs stating that this method must be better than the other one and stating why. We'd have some proofs where method A has better guaranteed lower bounds than method B, but the guaranteed bounds are so far from real world results that you'd ignore those proofs and pick method B anyway because it seems to works better in practice for everyone.
And yes, exactly as you say, this means that new systems/NN architectures can't be built reliably, they are designed and tested empirically and some degree of luck is involved; still, that doesn't mean you can't design useful things with it. Making those designs becomes hard engineering work, especially because of the lack of solid rules.