5 ms·

The more I get into machine learning and deep learning it seems like there is an incredible amount of configuration to get some decent results. Cleaning and sto

The more I get into machine learning and deep learning it seems like there is an incredible amount of configuration to get some decent results. Cleaning and storing the data takes a long time. And then you need to figure out exactly what you want to predict. If you predict some feature with any sort of error in your process the entire results will be flawed. There are a few very nice applications of the AI techniques, however most data sets don't fit well with machine learning. What you see is that in tutorials use the Iris data set so much because it breaks into categories very easily. In the real world, most things are in a maybe state rather than yes/no.

ffwd 10y ago
> In the real world, most things are in a maybe state rather than yes/no. Not to get too far afield, but I disagree with this on a certain philosophical level. All states are yes/no. All states of all things should result in a yes/no and be differentiable, with enough data. This doesn't speak to the practicality of that but as far as I can tell the theoretical potential is huge, almost infinite even.
- JonnieCache 10y ago
  Isn't this just shifting the ambiguity into your choice of state definitions, rather than the states themselves?
  - ffwd 10y ago
    But there should be no ambiguity, with enough data. Maybe that means there will always be ambiguity, but maybe it doesn't, especially not with man-made things and complex natural objects, and also if you can contextualize the data over time and 'geographically', there is more 'signal' there to differentiate
    - jonathankoren 10y ago
      > But there should be no ambiguity, with enough data. What? I'm sorry but this runs counter to everything in my experience, both professionally, and just casual very day experience. More data, helps to a point, but then there's diminishing returns, and it certainly doesn't eliminate the ambiguity. On the contrary, you discover diversity, and you still have a misclassification and perhaps even a harder data cleaning problem, because now you're seeing cases that aren't actually clear cut. Even if you're only talking about adding more features, well again, that works up to a point, but then you hit sparsity issues.
      - ffwd 10y ago
        Yeah that's why I called it philosophical, because the idea is a little more involved, shall we say. I'm not a god of this, so speculation ahead bewarned. In cases that aren't clear cut, you would also need contextual data like bigger actual physical area, or over time dimension, really any data point that can help narrow down what the thing is. It wouldn't just be pure deep learning stuff, it would be some kind of memory and data store of already classified objects and contexts. In the ultimate end, ALL of it would be sparse, but classify perfectly just that one thing it is built to classify. And if that doesn't work, several sparse things combined would result in one unique thing. On the sparsity matrix wikipedia page there is an example of balls with a string through them, this would correspond to the data being the balls and the systems we build (or alternatively unsupervised learning methods for finding new strings), whatever they may be, would be the strings (assuming all the strings are actual informational and correct to natural world). But you need the balls to begin with etc. Since all of this information should be in the natural world by its own, and also accessible to us
        jonathankoren 10y ago
        This is literally a philosophical problem. It's called ontology. And no amount of data solves this problem, because ultimately it's a labeling problem, and the border between things is ill defined, and additional data doesn't help resolve labeling ambiguity, if anything it finds out just how ill defined the world actually is. Think about it. Let's say you had a problem which was find the black squares. So you collect some data and you find that you have a whole bunch of squares that are on the blackness scale of 0.0, and bunch that are 0.1, and then there's one at 0.5. Is 0.5 black? Maybe not. What about 0.7? Maybe. What about 0.999? Probably, but is it? It's not 1.0. And if we say 0.9 and higher are black, why not 0.89? Even discounting measurement error, there's nothing that supports a threshold at 0.9 beyond, "Well, I think it should be this."
- goatlover 10y ago
  So we should just use Prolog to accomplish all our programming tasks?
  - ffwd 10y ago
    Nah. There is a difference in my mind between creating a 'map' of the natural world, and deciding which actions to perform in that world, and this is all theoretical/philosophical. If one were to do a pure AI type programming language it would have to come from the AI itself, and it would need some reason to create that language and who knows where/what all those rules would be and why (think of human programmers who make new languages).
- rampage101 10y ago
  What I mean is the Iris data set splits into 3 categories easily. Right now, they are saying AI self driving cars can get their predictions right 95%+ of the time. However, the cases where they cannot classify the object is the problem. Those are the "maybe" cases I was referring to where they algorithm simply cannot classify the object no matter how much it has seen.
bogomipz 10y ago
>" however most data sets don't fit well with machine learning" Could you elaborate on why this is?
- rampage101 10y ago
  Your average data-set does not fit into 3 nice categories like the Iris data-set does. For example, with the Iris data set if you know the sepal length and the petal length, you can say with near certainty which type of flower it is. Even trying to classify other objects in nature is much harder than this dataset. Now let's take sentiment analysis which tries to determine if some words are positive or negative. If someone said: 'that new machine learning algorithm is so sick'. The algorithm has no way of knowing that sick may be slang to mean good, because the system looks up 'sick' and finds that is a negative word. Sentiment analysis has no way of defining sarcasm or other natural language terms.
  - bogomipz 10y ago
    Thanks for the explanation and why the Iris data set is so ubiquitous in ML tutorials.
  - morgante 10y ago
    > Sentiment analysis has no way of defining sarcasm or other natural language terms. Of course it can. If humans are capable of detecting a given inflection, computers absolutely can as well (given enough data). Any sentiment analysis algorithm which classifies "that new machine learning algorithm is so sick" as negative is not worth an ounce of consideration. Compared to other problems, that is absolutely trivial to classify, especially since you're typically training off data sets which already include such vernacular.

ffwd 10y ago

> In the real world, most things are in a maybe state rather than yes/no. Not to get too far afield, but I disagree with this on a certain philosophical level. All states are yes/no. All states of all things should result in a yes/no and be differentiable, with enough data. This doesn't speak to the practicality of that but as far as I can tell the theoretical potential is huge, almost infinite even.

JonnieCache 10y ago
Isn't this just shifting the ambiguity into your choice of state definitions, rather than the states themselves?
- ffwd 10y ago
  But there should be no ambiguity, with enough data. Maybe that means there will always be ambiguity, but maybe it doesn't, especially not with man-made things and complex natural objects, and also if you can contextualize the data over time and 'geographically', there is more 'signal' there to differentiate
  - jonathankoren 10y ago
    > But there should be no ambiguity, with enough data. What? I'm sorry but this runs counter to everything in my experience, both professionally, and just casual very day experience. More data, helps to a point, but then there's diminishing returns, and it certainly doesn't eliminate the ambiguity. On the contrary, you discover diversity, and you still have a misclassification and perhaps even a harder data cleaning problem, because now you're seeing cases that aren't actually clear cut. Even if you're only talking about adding more features, well again, that works up to a point, but then you hit sparsity issues.
    - ffwd 10y ago
      Yeah that's why I called it philosophical, because the idea is a little more involved, shall we say. I'm not a god of this, so speculation ahead bewarned. In cases that aren't clear cut, you would also need contextual data like bigger actual physical area, or over time dimension, really any data point that can help narrow down what the thing is. It wouldn't just be pure deep learning stuff, it would be some kind of memory and data store of already classified objects and contexts. In the ultimate end, ALL of it would be sparse, but classify perfectly just that one thing it is built to classify. And if that doesn't work, several sparse things combined would result in one unique thing. On the sparsity matrix wikipedia page there is an example of balls with a string through them, this would correspond to the data being the balls and the systems we build (or alternatively unsupervised learning methods for finding new strings), whatever they may be, would be the strings (assuming all the strings are actual informational and correct to natural world). But you need the balls to begin with etc. Since all of this information should be in the natural world by its own, and also accessible to us
      - jonathankoren 10y ago
        This is literally a philosophical problem. It's called ontology. And no amount of data solves this problem, because ultimately it's a labeling problem, and the border between things is ill defined, and additional data doesn't help resolve labeling ambiguity, if anything it finds out just how ill defined the world actually is. Think about it. Let's say you had a problem which was find the black squares. So you collect some data and you find that you have a whole bunch of squares that are on the blackness scale of 0.0, and bunch that are 0.1, and then there's one at 0.5. Is 0.5 black? Maybe not. What about 0.7? Maybe. What about 0.999? Probably, but is it? It's not 1.0. And if we say 0.9 and higher are black, why not 0.89? Even discounting measurement error, there's nothing that supports a threshold at 0.9 beyond, "Well, I think it should be this."
goatlover 10y ago
So we should just use Prolog to accomplish all our programming tasks?
- ffwd 10y ago
  Nah. There is a difference in my mind between creating a 'map' of the natural world, and deciding which actions to perform in that world, and this is all theoretical/philosophical. If one were to do a pure AI type programming language it would have to come from the AI itself, and it would need some reason to create that language and who knows where/what all those rules would be and why (think of human programmers who make new languages).
rampage101 10y ago
What I mean is the Iris data set splits into 3 categories easily. Right now, they are saying AI self driving cars can get their predictions right 95%+ of the time. However, the cases where they cannot classify the object is the problem. Those are the "maybe" cases I was referring to where they algorithm simply cannot classify the object no matter how much it has seen.

bogomipz 10y ago

>" however most data sets don't fit well with machine learning" Could you elaborate on why this is?

rampage101 10y ago
Your average data-set does not fit into 3 nice categories like the Iris data-set does. For example, with the Iris data set if you know the sepal length and the petal length, you can say with near certainty which type of flower it is. Even trying to classify other objects in nature is much harder than this dataset. Now let's take sentiment analysis which tries to determine if some words are positive or negative. If someone said: 'that new machine learning algorithm is so sick'. The algorithm has no way of knowing that sick may be slang to mean good, because the system looks up 'sick' and finds that is a negative word. Sentiment analysis has no way of defining sarcasm or other natural language terms.
- bogomipz 10y ago
  Thanks for the explanation and why the Iris data set is so ubiquitous in ML tutorials.
- morgante 10y ago
  > Sentiment analysis has no way of defining sarcasm or other natural language terms. Of course it can. If humans are capable of detecting a given inflection, computers absolutely can as well (given enough data). Any sentiment analysis algorithm which classifies "that new machine learning algorithm is so sick" as negative is not worth an ounce of consideration. Compared to other problems, that is absolutely trivial to classify, especially since you're typically training off data sets which already include such vernacular.