4 ms·

Each time I talk about any problem with our signal theory expert he says: "we could use a Kalman filter" (in jest). That's like saying "hey we could use a matri

Each time I talk about any problem with our signal theory expert he says: "we could use a Kalman filter" (in jest). That's like saying "hey we could use a matrix".

aDfbrtVt 4y ago
I mean yes and no? Kalman filters fuses information about known system behaviour along with observations in an optimal predictor (for certain types of noise). It uses a matrix in the same way that almost every operation on a computer uses a matrix. Edit: I missed the quip about the Kalman comment being in jest, my reply probably wasn’t necessary but I’ll leave it up.
blululu 4y ago
Richard Hamming has a story where he was shocked/appalled that some folks wanted to include him on a paper after he suggested using Linear Regression in an obvious application of linear regression. The upshot is, if it were obvious it would have been done already. So many times people have done everything they can to avoid using a matrix decomposition or a proper filter.
- cushychicken 4y ago
  A lot of expertise is really: 1. Knowing when to use the right tool, and 2. Being in the right place at the right time to offer the insight. What’s obvious to the expert is frequently intractable to the outsider. In many cases, I wouldn’t call it so much “avoidance” as “not knowing the solution exists at all”.
thearn4 4y ago
In my area, every controls engineer says "we can just use a PID". 99% of the time it seems they're not wrong, despite control engineering being a pretty large area of research.
- elevation 4y ago
  I'm nominally familiar with PID loops but am not a control engineer. What other tools would you commonly bring to bear when a PID is not applicable?
- sitkack 4y ago
  And a simple PID controller built with op-amps looks an awful lot like a simple neural network. http://www.ecircuitcenter.com/Circuits/op_pid/op_pid.htm http://www.ecircuitcenter.com/Circuits/op_pid/op_pid.htm https://www.nutsvolts.com/magazine/article/the_perceptron_circuit https://www.nutsvolts.com/magazine/article/the_perceptron_ci...
  - mordae 4y ago
    Isn't gradient descent basically PID over parameters? And tricks like momentum basically a low-pass filter integrated in the PID loop? It's quite weird how not that many concepts from analog electronics domain have gotten carried over to ML.

Each time I talk about any problem with our signal theory expert he says: "we could use a Kalman filter" (in jest). That's like saying "hey we could use a matrix".

aDfbrtVt 4y ago

I mean yes and no? Kalman filters fuses information about known system behaviour along with observations in an optimal predictor (for certain types of noise). It uses a matrix in the same way that almost every operation on a computer uses a matrix. Edit: I missed the quip about the Kalman comment being in jest, my reply probably wasn’t necessary but I’ll leave it up.

blululu 4y ago

Richard Hamming has a story where he was shocked/appalled that some folks wanted to include him on a paper after he suggested using Linear Regression in an obvious application of linear regression. The upshot is, if it were obvious it would have been done already. So many times people have done everything they can to avoid using a matrix decomposition or a proper filter.

cushychicken 4y ago
A lot of expertise is really: 1. Knowing when to use the right tool, and 2. Being in the right place at the right time to offer the insight. What’s obvious to the expert is frequently intractable to the outsider. In many cases, I wouldn’t call it so much “avoidance” as “not knowing the solution exists at all”.

thearn4 4y ago

In my area, every controls engineer says "we can just use a PID". 99% of the time it seems they're not wrong, despite control engineering being a pretty large area of research.

elevation 4y ago
I'm nominally familiar with PID loops but am not a control engineer. What other tools would you commonly bring to bear when a PID is not applicable?
sitkack 4y ago
And a simple PID controller built with op-amps looks an awful lot like a simple neural network. http://www.ecircuitcenter.com/Circuits/op_pid/op_pid.htm http://www.ecircuitcenter.com/Circuits/op_pid/op_pid.htm https://www.nutsvolts.com/magazine/article/the_perceptron_circuit https://www.nutsvolts.com/magazine/article/the_perceptron_ci...
- mordae 4y ago
  Isn't gradient descent basically PID over parameters? And tricks like momentum basically a low-pass filter integrated in the PID loop? It's quite weird how not that many concepts from analog electronics domain have gotten carried over to ML.