3 ms·

Interesting project. I question the implications of using ML data shaping to define models. A sufficiently optimized model may be missing fundamental inputs & c

Interesting project. I question the implications of using ML data shaping to define models. A sufficiently optimized model may be missing fundamental inputs & constraints, yet still perform better than a less optimized model with the correct inputs & constraints. This would effectively create a local maxima of understanding. It seems, while the predictive capabilities of complex systems may increase in some cases, the practice of fundamental understanding & articulation of the system will decrease (i.e. I don't know how it works, the magic black box has all the answers). Perhaps there are ways to mitigate these implications in the scientific process.

ddragon 7y ago
In this case it might be the exact opposite. Regardless of the structure (as long as it has at least one hidden layer and enough parameters), neural networks are universal approximators, which is why they are effectively black boxes that can learn any model. So using more custom structure does not increase the theoretical capability of the network to find more complex models, but instead you're effectively adding more a priori information (by embedding information of the problem within it's structure), which can heavily restrict the search space and therefore making it somewhat less of a magic black box. An example might be attention models, which while being more complex than the previous models it gives you more information about what the model does (by allowing you to visualize what inputs the following layer is using).