3 ms·

Here's a problem: I'm pretty sure you'd get just as good (probably superior) results with a simpler, faster out-of-the-box algorithm like logistic regression, r

Here's a problem: I'm pretty sure you'd get just as good (probably superior) results with a simpler, faster out-of-the-box algorithm like logistic regression, random forests or boosted trees (depending on the amount of data). This just isn't a problem that seems well suited for neural networks. Gyroscope/accelerometer data might seem like the kind of extremely-noisy, high-dimensional data well-suited for neural nets, but 1) you don't have a lot of data, and 2) intuitively the data isn't actually that noisy; my guess is the data separates out quite nicely (not necessarily linearly).

blennon 11y ago
I agree and disagree. I would definitely start with a simpler algorithm like logistic regression or even trees and use temporal features like power at different frequencies by taking an FFT. I would even add time lagged features. After that I'd graduate to a single hidden layer MLP. If I had tons of data (a lot of different people using it), I'd experiment with LSTM neural networks because I think the temporal information is crucial for determining a movement. I think a killer app would be aimed at weight lifters. If you go to the gym, the serious weight lifters record their reps and weight for each exercise. The app would utilize the iWatch or some other wearable and detect the exercise and count the reps. Then it would prompt the user for how much weight they used.
- argilium 11y ago
  You're right on the ball. I'm indeed using temporal features to classify, along with various other statistical ones. The repetition counting in my next aim. Once apple opens up the gyro on the Watch, I think there could be a good chance to get something like this out the door.
- argonaut 11y ago
  I'm still pretty sure you'd get superior results with other standard techniques for time series data (HMMs, conditional random fields, etc.). You really need a crapload of data to train an RNN well.
argilium 11y ago
You're right that I could get just as good results with other potentially faster algorithms but those algorithms don't afford me the ability (especially decision trees) to personalise. The system is designed to be personalised. So for example if it's not doing well in recognising your squats for instance, you can immediately 'show' it what they look like. A neural network can immediately integrate that, and spit out great accuracy levels. Whilst i'd have to reconstruct a decision tree each time.
- argonaut 11y ago
  You definitely can use online learning with logistic regression. There are also ways to get it to work with random forests / boosted trees.
  - argilium 11y ago
    I never even considered logistic regression for this problem. Now that you've pointed it out, I'm going to look into. Neural networks aren't really core part of the system. As long as I can get good + quick results, I'm really open to swapping out the learning algorithm.

blennon 11y ago

I agree and disagree. I would definitely start with a simpler algorithm like logistic regression or even trees and use temporal features like power at different frequencies by taking an FFT. I would even add time lagged features. After that I'd graduate to a single hidden layer MLP. If I had tons of data (a lot of different people using it), I'd experiment with LSTM neural networks because I think the temporal information is crucial for determining a movement. I think a killer app would be aimed at weight lifters. If you go to the gym, the serious weight lifters record their reps and weight for each exercise. The app would utilize the iWatch or some other wearable and detect the exercise and count the reps. Then it would prompt the user for how much weight they used.

argilium 11y ago
You're right on the ball. I'm indeed using temporal features to classify, along with various other statistical ones. The repetition counting in my next aim. Once apple opens up the gyro on the Watch, I think there could be a good chance to get something like this out the door.
argonaut 11y ago
I'm still pretty sure you'd get superior results with other standard techniques for time series data (HMMs, conditional random fields, etc.). You really need a crapload of data to train an RNN well.

argilium 11y ago

You're right that I could get just as good results with other potentially faster algorithms but those algorithms don't afford me the ability (especially decision trees) to personalise. The system is designed to be personalised. So for example if it's not doing well in recognising your squats for instance, you can immediately 'show' it what they look like. A neural network can immediately integrate that, and spit out great accuracy levels. Whilst i'd have to reconstruct a decision tree each time.

argonaut 11y ago
You definitely can use online learning with logistic regression. There are also ways to get it to work with random forests / boosted trees.
- argilium 11y ago
  I never even considered logistic regression for this problem. Now that you've pointed it out, I'm going to look into. Neural networks aren't really core part of the system. As long as I can get good + quick results, I'm really open to swapping out the learning algorithm.