6 ms·

I've been wondering if neural nets will able able to find solutions to PDEs with sparse inputs such as this one. Intuitively it makes sense that fast solvers wo

I've been wondering if neural nets will able able to find solutions to PDEs with sparse inputs such as this one. Intuitively it makes sense that fast solvers would exist for any PDE where the input data is either very sparse or highly symmetrical. A lot of the PDE domain can be "meshed" advantageously with portions of the domain either reusing computation or being wiped out as insignificant contributions. A neural net can learn to represent a PDE domain in terms such regions. I am really curious if this is what happens here and if there is something vastly beyond what current PDE solvers are able to do. But I also imagine as the input size grows within the orders of the hidden layers it will lose the advantage.

kuiooullkre 7y ago
But how are you going to trust it? If the neural net gives you a solution, is there any cheap way to confirm it?
- shoo 7y ago
  Yeah. Guarantees about robustness are pretty useful. That said, even if you didn't trust it, it might be possible to use the approximate alleged solution spat out by the neural net as an initial solution to a trusted solver with guarantees. In some cases you might be able to get a guaranteed result from the proper solver with the neural net solution used to accelerate convergence
- TheEzEzz 7y ago
  Depends on the problems, but in many situations yes. For PDE's you can plug in the proposed solution and calculate the error. Usually you'd do this in a discretized version of the PDE, but since neural nets are symbolically differentiable you could instead approximate (by sampling) the average error from the real PDE itself.
  - xiphias2 7y ago
    Sounds like a good way to train an adversial network
- ska 7y ago
  "trust, but verify". You can get decent error estimates for lots of these sorts of systems, with a manageable amount of extra computation.
cossatot 7y ago
Some earthquake researchers I know have started using neural net approximations of PDEs for doing viscoelastic calculations [1] (simulating how the Earth's crust and upper mantle responds during and after an earthquake). The have found pretty huge speedups (5000% on CPU - 50,000% on GPU) using neural nets with good accuracy, but I am not sure that the PDE meshing was super highly optimized. [1]: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017GL072716 https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017... edit: Whether this NN approximations will enable us going well beyond what PDE solvers can do is a tough question, especially because PDE solvers are generally needed to build the training sets as far as I understand (i.e. they are trained on synthetic, not observational, data). It's possible that one could strategically and judiciously train smaller components of a NN system and then assemble the system to do things that the PDE solvers couldn't do, perhaps, in the way that one can often write unit tests for scientific software but not really test the final results directly (because there is nothing to test against). However I think that in general the lack of training data for the more complex system will always be a major obstacle, whereas it's not really for PDEs.

kuiooullkre 7y ago

But how are you going to trust it? If the neural net gives you a solution, is there any cheap way to confirm it?

shoo 7y ago
Yeah. Guarantees about robustness are pretty useful. That said, even if you didn't trust it, it might be possible to use the approximate alleged solution spat out by the neural net as an initial solution to a trusted solver with guarantees. In some cases you might be able to get a guaranteed result from the proper solver with the neural net solution used to accelerate convergence
TheEzEzz 7y ago
Depends on the problems, but in many situations yes. For PDE's you can plug in the proposed solution and calculate the error. Usually you'd do this in a discretized version of the PDE, but since neural nets are symbolically differentiable you could instead approximate (by sampling) the average error from the real PDE itself.
- xiphias2 7y ago
  Sounds like a good way to train an adversial network
ska 7y ago
"trust, but verify". You can get decent error estimates for lots of these sorts of systems, with a manageable amount of extra computation.

cossatot 7y ago

Some earthquake researchers I know have started using neural net approximations of PDEs for doing viscoelastic calculations [1] (simulating how the Earth's crust and upper mantle responds during and after an earthquake). The have found pretty huge speedups (5000% on CPU - 50,000% on GPU) using neural nets with good accuracy, but I am not sure that the PDE meshing was super highly optimized. [1]: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017GL072716 https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017... edit: Whether this NN approximations will enable us going well beyond what PDE solvers can do is a tough question, especially because PDE solvers are generally needed to build the training sets as far as I understand (i.e. they are trained on synthetic, not observational, data). It's possible that one could strategically and judiciously train smaller components of a NN system and then assemble the system to do things that the PDE solvers couldn't do, perhaps, in the way that one can often write unit tests for scientific software but not really test the final results directly (because there is nothing to test against). However I think that in general the lack of training data for the more complex system will always be a major obstacle, whereas it's not really for PDEs.