3 ms·

I started taking the Coursera ML class. Reading this article, something jumped at me: https://datadotblog.files.wordpress.com/2018/12/Screen-Shot-2018-12-22-at

I started taking the Coursera ML class. Reading this article, something jumped at me: https://datadotblog.files.wordpress.com/2018/12/Screen-Shot-2018-12-22-at-18.23.34.png https://datadotblog.files.wordpress.com/2018/12/Screen-Shot-... It mentions how it's 'impossible' to separate the data points in cartesian coodinates. Isn't logistic regression exactly the use case for this? Thus making the transformation irrelevant? Anyone with ML experience have an opinion on this?

ccdsacc 8y ago
Logistic regression's decisioun boundary is linear, so it wouldn't work well with the first example as is.
nerdponx 8y ago
No, linear regression does not imply separation. Yes, this is why we use regression, soft-margin SVM, etc. instead of hard-margin SVM. Because perfect linear separation is unrealistic.
- kelvin0 8y ago
  Please note I wrote 'Logistic Regression' and not 'Linear Regression' (as you seem to think). Logistic Regression based classification (with quadratic theta parameters) would seem to certainly be able to handle the cartesian case (without having to resort to convert into polar coordinates).
  - nerdponx 8y ago
    I meant to write "logistic", but it's worth noting that logistic regression is a linear model from which you derive a linear decision boundary. And yes, it can handle it, by finding a "optimal" boundary according to a criterion other than "is it separated or not?". But that's not the point. The data remains inseparable. And yes, while logistic regression can technically handle this case (by returning a solution and not blowing up), it will perform poorly unless you transform the data, because the decision boundary is still linear.
    - kelvin0 8y ago
      Really appreciate your feedback, I'll certainly look into your claims in the next few days. What's your background? Have you 'been' in ML long? Feel free to give me as much details as you feel comfortable with. Thanks!

ccdsacc 8y ago

Logistic regression's decisioun boundary is linear, so it wouldn't work well with the first example as is.

nerdponx 8y ago

No, linear regression does not imply separation. Yes, this is why we use regression, soft-margin SVM, etc. instead of hard-margin SVM. Because perfect linear separation is unrealistic.

kelvin0 8y ago
Please note I wrote 'Logistic Regression' and not 'Linear Regression' (as you seem to think). Logistic Regression based classification (with quadratic theta parameters) would seem to certainly be able to handle the cartesian case (without having to resort to convert into polar coordinates).
- nerdponx 8y ago
  I meant to write "logistic", but it's worth noting that logistic regression is a linear model from which you derive a linear decision boundary. And yes, it can handle it, by finding a "optimal" boundary according to a criterion other than "is it separated or not?". But that's not the point. The data remains inseparable. And yes, while logistic regression can technically handle this case (by returning a solution and not blowing up), it will perform poorly unless you transform the data, because the decision boundary is still linear.
  - kelvin0 8y ago
    Really appreciate your feedback, I'll certainly look into your claims in the next few days. What's your background? Have you 'been' in ML long? Feel free to give me as much details as you feel comfortable with. Thanks!