5 ms·

When I was a grad student, another student was trying to simulate a particular stochastic differential equation. He spent a few weeks writing a parallelized sol

by gaze 6y ago

When I was a grad student, another student was trying to simulate a particular stochastic differential equation. He spent a few weeks writing a parallelized solver, learned to use MPI, and distributed the computation over our school’s cluster, got the results he needed and moved on with his life. When I had encountered the same problem later after studying SDEs in a bit more detail, I found that I could reduce the problem into something I could solve on my laptop in about 10 minutes. We can argue about how much time he and I put into the problem from ground zero. The thing I’m arguing is that one can spend a lot of time misusing the building blocks if one doesn’t understand them. Existence and uniqueness seem hoity toity until you try to numerically solve an equation over a domain for which a solution doesn’t exist, or you try to solve it and you get one of many solutions and think you’re done. It’s hard to measure the value of knowing the fundamentals... especially 10 years out when it seems like you’re reusing the same tiny subset of your toolset over and over again. The fundamentals though exist as a little voice in the back of your head reminding you not to do something that won’t work.

pirocks 6y ago
Mathematica has a function called NDsolve which attempts to reduce/simplify/solve fully symbolically and then falls back on numeric methods. Never actually used it for anything real, but could have potentially avoided this situation.
- lowkeynthorough 6y ago
  Grad student here. I recently did the obvious NDSolve thing but Mathematica kept beachballing, and after heating up my computer tremendously returned an empty set. Turns out, my problem was really simple and I could solve it by hand.
- gaze 6y ago
  These are SDEs. Mathematica doesn’t solve them. In the end I needed a way for a computer to solve them faster, as an analytical solution wasn’t really possible. Mathematica wi never provide such insights. Maybe one day it will?
  - phonon 6y ago
    https://reference.wolfram.com/language/guide/StochasticDifferentialEquationProcesses.html https://reference.wolfram.com/language/guide/StochasticDiffe... isn't bad...
- yters 6y ago
  Symbolic reduction is undecidable.
heimatau 6y ago
> It’s hard to measure the value of knowing the fundamentals... especially 10 years out when it seems like you’re reusing the same tiny subset of your toolset over and over again. The fundamentals though exist as a little voice in the back of your head reminding you not to do something that won’t work. I think this is why the US govt has been pushing STEM majors because there is a serious lack of people that understand 'advanced' math. I'm learning some fundamentals as we speak and it means nothing if you can't connect the dots. Understanding these concepts is the start but using them and experiencing them is where the real 'magic' happens. Except not many get that far. Keep up the great work gaze!
laingc 6y ago
I completely agree with this. I have a constant battle with some (not all) of my team to get them to think through a problem from its fundamentals, and try to find the best one or two or three ways to pose the problem. Conveniently-posed problems lead to both convenient solutions, as well as a variety of potential ancillary benefits. One practical application of this, for those who are more business-minded, is that I have set a very clear and tight tech strategy for our team. There have been quite a few instances where posing a problem differently has allowed us to better align with that strategy by enabling us to pursue solutions that make use of our existing technology stack and in-house tooling, rather than requiring us to introduce more dependencies and spread ourselves even thinner.
- yters 6y ago
  Exactly. We can try brute force counting all the permutations of 1000 bits, which will exhaust the lifespan and size of the universe. Or we can use exponentiation and solve by hand in a second.
petermcneeley 6y ago
My Anecdotal experience is the inverse. Students and even professors struggling to solve problems analytically that are trivial to simulate.
- gaze 6y ago
  Differential equations are “trivial” to simulate until they aren’t. Some differential equations feature conserved quantities or invariants that will blow up over long times if you don’t build their conservation into the numerical solver. Some equations can’t be solved with variable time step without careful consideration. Some problems are so huge you can’t solve them without taking some kind of approximation. For instance you might need to find the low dimensional sub space to project onto, and then write the equations of motion for the sub space. You might be solving a finite element problem and need to find the right way to discretize. For electromagnetism this requires writing your elements to be irrotational for the E field. Furthermore, a lot can be gleaned from understanding the _structure_ of a differential equation. Writing the equation for the RG flow of a field theory and then understanding the fixed points of the flow is crucial. A numerical solution to the RG flow isn’t really that useful. Basically, once you’ve simulated it—-assuming you’ve done it right—-what actually have you gained? Often less than you want, and generally less insight than you could have gained had you really analyzed it. What I’m saying is that “trivial to simulate” is only sometimes true. Ansys’s business exists because solving Maxwell’s equations are not “trivial,” and they’re linear!!
  - petermcneeley 6y ago
    Thanks for your detailed reply. While, perhaps not an expert, I am not completely unfamiliar with the difficulties you raise. I have done graduate computational physics and have worked in realtime simulation professionally as well as offline simulation on my own. "Basically, once you’ve simulated it—-assuming you’ve done it right—-what actually have you gained? " Usually you now have a computational model that can give answer for any valid input. This may not help your discipline but it certainly helps others. As you suggested with Ansys, there is always room for improvement on these computational models. I would guess unless there was something really wrong with the approach these improvements are going to be algorithmic in nature. Im not saying algorithms dont potentially have some good mathematical analysis behind them but something like say Barnes–Hut feels far more like a computer science data structure than some symbolic mathematical representation.
sjg007 6y ago
And then there’s deep neural networks..,
jofer 6y ago
This is a great description of scientific computing in a nutshell. It's rarely getting an algorithmic speedup. It's far more often stepping back and solving a simplified or reformulated problem. It's often more complex in practice, but still, it's easy to focus too much on solving the given problem efficiently efficiently, rather than finding a simpler solution to the underlying mathematics.
- sgc 6y ago
  I have often said more philosophically that the most important thing about intelligence is what you think about, not how good you are at thinking about it. I say it with all due respect for analytics, which are fundamental to my mindset. But many extremely intelligent people reach the point of basically squandering their lives by never asking the right questions.
spaetzleesser 6y ago
I used to work with a guy from East Germany who really understood math. It was amazing how he often could reduce problems I would solve with enormous amounts of code and computing power with a few simple equations. It's a lot of work to get to that level of math understanding but it's definitely a hidden superpower.

by gaze 6y ago

pirocks 6y ago

Mathematica has a function called NDsolve which attempts to reduce/simplify/solve fully symbolically and then falls back on numeric methods. Never actually used it for anything real, but could have potentially avoided this situation.

lowkeynthorough 6y ago
Grad student here. I recently did the obvious NDSolve thing but Mathematica kept beachballing, and after heating up my computer tremendously returned an empty set. Turns out, my problem was really simple and I could solve it by hand.
gaze 6y ago
These are SDEs. Mathematica doesn’t solve them. In the end I needed a way for a computer to solve them faster, as an analytical solution wasn’t really possible. Mathematica wi never provide such insights. Maybe one day it will?
- phonon 6y ago
  https://reference.wolfram.com/language/guide/StochasticDifferentialEquationProcesses.html https://reference.wolfram.com/language/guide/StochasticDiffe... isn't bad...
yters 6y ago
Symbolic reduction is undecidable.

heimatau 6y ago

> It’s hard to measure the value of knowing the fundamentals... especially 10 years out when it seems like you’re reusing the same tiny subset of your toolset over and over again. The fundamentals though exist as a little voice in the back of your head reminding you not to do something that won’t work. I think this is why the US govt has been pushing STEM majors because there is a serious lack of people that understand 'advanced' math. I'm learning some fundamentals as we speak and it means nothing if you can't connect the dots. Understanding these concepts is the start but using them and experiencing them is where the real 'magic' happens. Except not many get that far. Keep up the great work gaze!

laingc 6y ago

I completely agree with this. I have a constant battle with some (not all) of my team to get them to think through a problem from its fundamentals, and try to find the best one or two or three ways to pose the problem. Conveniently-posed problems lead to both convenient solutions, as well as a variety of potential ancillary benefits. One practical application of this, for those who are more business-minded, is that I have set a very clear and tight tech strategy for our team. There have been quite a few instances where posing a problem differently has allowed us to better align with that strategy by enabling us to pursue solutions that make use of our existing technology stack and in-house tooling, rather than requiring us to introduce more dependencies and spread ourselves even thinner.

yters 6y ago
Exactly. We can try brute force counting all the permutations of 1000 bits, which will exhaust the lifespan and size of the universe. Or we can use exponentiation and solve by hand in a second.

petermcneeley 6y ago

My Anecdotal experience is the inverse. Students and even professors struggling to solve problems analytically that are trivial to simulate.

gaze 6y ago
Differential equations are “trivial” to simulate until they aren’t. Some differential equations feature conserved quantities or invariants that will blow up over long times if you don’t build their conservation into the numerical solver. Some equations can’t be solved with variable time step without careful consideration. Some problems are so huge you can’t solve them without taking some kind of approximation. For instance you might need to find the low dimensional sub space to project onto, and then write the equations of motion for the sub space. You might be solving a finite element problem and need to find the right way to discretize. For electromagnetism this requires writing your elements to be irrotational for the E field. Furthermore, a lot can be gleaned from understanding the _structure_ of a differential equation. Writing the equation for the RG flow of a field theory and then understanding the fixed points of the flow is crucial. A numerical solution to the RG flow isn’t really that useful. Basically, once you’ve simulated it—-assuming you’ve done it right—-what actually have you gained? Often less than you want, and generally less insight than you could have gained had you really analyzed it. What I’m saying is that “trivial to simulate” is only sometimes true. Ansys’s business exists because solving Maxwell’s equations are not “trivial,” and they’re linear!!
- petermcneeley 6y ago
  Thanks for your detailed reply. While, perhaps not an expert, I am not completely unfamiliar with the difficulties you raise. I have done graduate computational physics and have worked in realtime simulation professionally as well as offline simulation on my own. "Basically, once you’ve simulated it—-assuming you’ve done it right—-what actually have you gained? " Usually you now have a computational model that can give answer for any valid input. This may not help your discipline but it certainly helps others. As you suggested with Ansys, there is always room for improvement on these computational models. I would guess unless there was something really wrong with the approach these improvements are going to be algorithmic in nature. Im not saying algorithms dont potentially have some good mathematical analysis behind them but something like say Barnes–Hut feels far more like a computer science data structure than some symbolic mathematical representation.

sjg007 6y ago

And then there’s deep neural networks..,

jofer 6y ago

This is a great description of scientific computing in a nutshell. It's rarely getting an algorithmic speedup. It's far more often stepping back and solving a simplified or reformulated problem. It's often more complex in practice, but still, it's easy to focus too much on solving the given problem efficiently efficiently, rather than finding a simpler solution to the underlying mathematics.

sgc 6y ago
I have often said more philosophically that the most important thing about intelligence is what you think about, not how good you are at thinking about it. I say it with all due respect for analytics, which are fundamental to my mindset. But many extremely intelligent people reach the point of basically squandering their lives by never asking the right questions.

spaetzleesser 6y ago

I used to work with a guy from East Germany who really understood math. It was amazing how he often could reduce problems I would solve with enormous amounts of code and computing power with a few simple equations. It's a lot of work to get to that level of math understanding but it's definitely a hidden superpower.