3 ms·

If I understand correctly, the reason a CNN is used here is because we want to find the splits that a human would visually agree "looks" the best? So rather th

by zawerf 9y ago

If I understand correctly, the reason a CNN is used here is because we want to find the splits that a human would visually agree "looks" the best? So rather than a regression it's more like the "line simplification" problem in graphics: (https://bost.ocks.org/mike/simplify/ https://bost.ocks.org/mike/simplify/ https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93...) Just thought this solution seems a little overkill. Surely you can pick some error metric over the splits to optimize instead?

ianbooker 9y ago
I think it is a great example for deployment and may be even a good example for tackling a problem that is not easy to understand. Either I missed it, or the nature of the problem is never thoroughly analyzed. I am not an expert when it comes to mechanical watches, but the main question hovering over this topic is: Will a watch deviate from the perfect time in a linear fashion? Or can there be other models, even more so: Is it possible that different watches will deviate in completely different modes? If so, the problem gets instantly three magnitudes harder...
- mr_toad 9y ago
  I’m guessing that it’s desirable to report linear deviations to the users (seconds per day), even if the deviations are non-linear.
moconnor 9y ago
Overkill is a point of view here. Training and deploying neural networks is becoming easier than ever. In my group at Arm there's a solid expectation that we'll see neural networks integrated into every part of a running application, and whether they execute on special NN processors or the general-purpose CPU will largely depend on where the data is needed. I cut a very long comment short and wrote the rest of this up here: http://yieldthought.com/post/170830096265/when-are-neural-networks-overkill http://yieldthought.com/post/170830096265/when-are-neural-ne...
- zawerf 9y ago
  I said it was overkill because I thought I had a simple analytical solution as follows. Note: I don't know anything about segmented regression, this is just your standard CS dynamic programming to calculate splits DP[i][j] = min over k of (DP[i][k] + (cost of splitting at k) + (linear regression error of points from kth to jth)) This should run in O(n^3) which will be fine for the author's requirement of ~100 points. But this isn't a complete solution since it's not obvious how to choose the cost of splitting (which is needed otherwise it will just split everything into 1 or 2 point segments). I think thinking about this more and explicitly trying to design this cost function is still better than labeling a bunch of data until the machine learning algorithm can reverse engineer the cost function from your head. Then you can be confident of what your code is doing and why and know that it won't randomly output potato.
  - loverofthings 9y ago
    If you have labeled data (meaning that you know where proper splits need to be made) it's quite easy to build a DP based classifier that minimizes the error over training set. For example, if you were building a model that spits out 0 for no split, and 1 for split, you can easily make a simple cost sensitive linear model that takes into account previous decisions (something like HMM). Viterbi algorithm would be the DP step. For some, to me unknown, reason CNN performs well if not better than DP based HMM.
casey 9y ago
I believe this is what the author tried first in the post. He even links to this test UI where you can compare the "plain math" approach to the neural network: https://attardi.org/pytorch-and-coreml-test-ui/ https://attardi.org/pytorch-and-coreml-test-ui/
mark_adams_iii 9y ago
This, I believe, is a discussion about analytical vs cognitive problem solving. And specialized vs generic approaches. Maybe the author of the app had other uses for CNNs in mind for other features in the future. So if you account for that, why have individual analytical solutions when you can solve a whole bunch of problems with one cognitive approach?

by zawerf 9y ago

ianbooker 9y ago

I think it is a great example for deployment and may be even a good example for tackling a problem that is not easy to understand. Either I missed it, or the nature of the problem is never thoroughly analyzed. I am not an expert when it comes to mechanical watches, but the main question hovering over this topic is: Will a watch deviate from the perfect time in a linear fashion? Or can there be other models, even more so: Is it possible that different watches will deviate in completely different modes? If so, the problem gets instantly three magnitudes harder...

mr_toad 9y ago
I’m guessing that it’s desirable to report linear deviations to the users (seconds per day), even if the deviations are non-linear.

moconnor 9y ago

Overkill is a point of view here. Training and deploying neural networks is becoming easier than ever. In my group at Arm there's a solid expectation that we'll see neural networks integrated into every part of a running application, and whether they execute on special NN processors or the general-purpose CPU will largely depend on where the data is needed. I cut a very long comment short and wrote the rest of this up here: http://yieldthought.com/post/170830096265/when-are-neural-networks-overkill http://yieldthought.com/post/170830096265/when-are-neural-ne...

zawerf 9y ago
I said it was overkill because I thought I had a simple analytical solution as follows. Note: I don't know anything about segmented regression, this is just your standard CS dynamic programming to calculate splits DP[i][j] = min over k of (DP[i][k] + (cost of splitting at k) + (linear regression error of points from kth to jth)) This should run in O(n^3) which will be fine for the author's requirement of ~100 points. But this isn't a complete solution since it's not obvious how to choose the cost of splitting (which is needed otherwise it will just split everything into 1 or 2 point segments). I think thinking about this more and explicitly trying to design this cost function is still better than labeling a bunch of data until the machine learning algorithm can reverse engineer the cost function from your head. Then you can be confident of what your code is doing and why and know that it won't randomly output potato.
- loverofthings 9y ago
  If you have labeled data (meaning that you know where proper splits need to be made) it's quite easy to build a DP based classifier that minimizes the error over training set. For example, if you were building a model that spits out 0 for no split, and 1 for split, you can easily make a simple cost sensitive linear model that takes into account previous decisions (something like HMM). Viterbi algorithm would be the DP step. For some, to me unknown, reason CNN performs well if not better than DP based HMM.

casey 9y ago

I believe this is what the author tried first in the post. He even links to this test UI where you can compare the "plain math" approach to the neural network: https://attardi.org/pytorch-and-coreml-test-ui/ https://attardi.org/pytorch-and-coreml-test-ui/

mark_adams_iii 9y ago

This, I believe, is a discussion about analytical vs cognitive problem solving. And specialized vs generic approaches. Maybe the author of the app had other uses for CNNs in mind for other features in the future. So if you account for that, why have individual analytical solutions when you can solve a whole bunch of problems with one cognitive approach?