4 ms·

Isn't this just shifting the ambiguity into your choice of state definitions, rather than the states themselves?

ffwd 10y ago
But there should be no ambiguity, with enough data. Maybe that means there will always be ambiguity, but maybe it doesn't, especially not with man-made things and complex natural objects, and also if you can contextualize the data over time and 'geographically', there is more 'signal' there to differentiate
- jonathankoren 10y ago
  > But there should be no ambiguity, with enough data. What? I'm sorry but this runs counter to everything in my experience, both professionally, and just casual very day experience. More data, helps to a point, but then there's diminishing returns, and it certainly doesn't eliminate the ambiguity. On the contrary, you discover diversity, and you still have a misclassification and perhaps even a harder data cleaning problem, because now you're seeing cases that aren't actually clear cut. Even if you're only talking about adding more features, well again, that works up to a point, but then you hit sparsity issues.
  - ffwd 10y ago
    Yeah that's why I called it philosophical, because the idea is a little more involved, shall we say. I'm not a god of this, so speculation ahead bewarned. In cases that aren't clear cut, you would also need contextual data like bigger actual physical area, or over time dimension, really any data point that can help narrow down what the thing is. It wouldn't just be pure deep learning stuff, it would be some kind of memory and data store of already classified objects and contexts. In the ultimate end, ALL of it would be sparse, but classify perfectly just that one thing it is built to classify. And if that doesn't work, several sparse things combined would result in one unique thing. On the sparsity matrix wikipedia page there is an example of balls with a string through them, this would correspond to the data being the balls and the systems we build (or alternatively unsupervised learning methods for finding new strings), whatever they may be, would be the strings (assuming all the strings are actual informational and correct to natural world). But you need the balls to begin with etc. Since all of this information should be in the natural world by its own, and also accessible to us
    - jonathankoren 10y ago
      This is literally a philosophical problem. It's called ontology. And no amount of data solves this problem, because ultimately it's a labeling problem, and the border between things is ill defined, and additional data doesn't help resolve labeling ambiguity, if anything it finds out just how ill defined the world actually is. Think about it. Let's say you had a problem which was find the black squares. So you collect some data and you find that you have a whole bunch of squares that are on the blackness scale of 0.0, and bunch that are 0.1, and then there's one at 0.5. Is 0.5 black? Maybe not. What about 0.7? Maybe. What about 0.999? Probably, but is it? It's not 1.0. And if we say 0.9 and higher are black, why not 0.89? Even discounting measurement error, there's nothing that supports a threshold at 0.9 beyond, "Well, I think it should be this."
      - ffwd 10y ago
        > if anything it finds out just how ill defined the world actually is. Yeah I hear this but it seems only half-true to me. While for most intents and purposes the world is ill-defined, in another sense the world itself is "100% signal" and no noise. If we "zoom out" and take a grand view, imagining that we have a supercomputer and a huge database, and the algorithms are solved, I think every 'thing' in the universe has some unique features, and if you start to have them all in a database you may be able to uniquely identify any thing, at least those important to us. Everything one has excludes something else, but it also includes that specific thing. Every thing adds context to one thing and removes context from another. If you can draw a map of it, it seems to me like deep learning can, hypothetically, automatically differentiate it. Deep learning isn't just about one vector or one hierarchy of features, it's about how the world is ALL vectors like this, even if right now, the CS around it is pretty limited. It seems to me intuitively true at least. At the bare minimum, seeing as us humans are absurd about categorizing everything into objects, and it actually works very well functionally (we can manipulate, create and predict in the world)
        jonathankoren 10y ago
        I think you would find metaphysics (specifically ontology) and cognitive science interesting. I think you'll find your ideas are actually very, very old. ;)

ffwd 10y ago

But there should be no ambiguity, with enough data. Maybe that means there will always be ambiguity, but maybe it doesn't, especially not with man-made things and complex natural objects, and also if you can contextualize the data over time and 'geographically', there is more 'signal' there to differentiate

jonathankoren 10y ago
> But there should be no ambiguity, with enough data. What? I'm sorry but this runs counter to everything in my experience, both professionally, and just casual very day experience. More data, helps to a point, but then there's diminishing returns, and it certainly doesn't eliminate the ambiguity. On the contrary, you discover diversity, and you still have a misclassification and perhaps even a harder data cleaning problem, because now you're seeing cases that aren't actually clear cut. Even if you're only talking about adding more features, well again, that works up to a point, but then you hit sparsity issues.
- ffwd 10y ago
  Yeah that's why I called it philosophical, because the idea is a little more involved, shall we say. I'm not a god of this, so speculation ahead bewarned. In cases that aren't clear cut, you would also need contextual data like bigger actual physical area, or over time dimension, really any data point that can help narrow down what the thing is. It wouldn't just be pure deep learning stuff, it would be some kind of memory and data store of already classified objects and contexts. In the ultimate end, ALL of it would be sparse, but classify perfectly just that one thing it is built to classify. And if that doesn't work, several sparse things combined would result in one unique thing. On the sparsity matrix wikipedia page there is an example of balls with a string through them, this would correspond to the data being the balls and the systems we build (or alternatively unsupervised learning methods for finding new strings), whatever they may be, would be the strings (assuming all the strings are actual informational and correct to natural world). But you need the balls to begin with etc. Since all of this information should be in the natural world by its own, and also accessible to us
  - jonathankoren 10y ago
    This is literally a philosophical problem. It's called ontology. And no amount of data solves this problem, because ultimately it's a labeling problem, and the border between things is ill defined, and additional data doesn't help resolve labeling ambiguity, if anything it finds out just how ill defined the world actually is. Think about it. Let's say you had a problem which was find the black squares. So you collect some data and you find that you have a whole bunch of squares that are on the blackness scale of 0.0, and bunch that are 0.1, and then there's one at 0.5. Is 0.5 black? Maybe not. What about 0.7? Maybe. What about 0.999? Probably, but is it? It's not 1.0. And if we say 0.9 and higher are black, why not 0.89? Even discounting measurement error, there's nothing that supports a threshold at 0.9 beyond, "Well, I think it should be this."
    - ffwd 10y ago
      > if anything it finds out just how ill defined the world actually is. Yeah I hear this but it seems only half-true to me. While for most intents and purposes the world is ill-defined, in another sense the world itself is "100% signal" and no noise. If we "zoom out" and take a grand view, imagining that we have a supercomputer and a huge database, and the algorithms are solved, I think every 'thing' in the universe has some unique features, and if you start to have them all in a database you may be able to uniquely identify any thing, at least those important to us. Everything one has excludes something else, but it also includes that specific thing. Every thing adds context to one thing and removes context from another. If you can draw a map of it, it seems to me like deep learning can, hypothetically, automatically differentiate it. Deep learning isn't just about one vector or one hierarchy of features, it's about how the world is ALL vectors like this, even if right now, the CS around it is pretty limited. It seems to me intuitively true at least. At the bare minimum, seeing as us humans are absurd about categorizing everything into objects, and it actually works very well functionally (we can manipulate, create and predict in the world)
      - jonathankoren 10y ago
        I think you would find metaphysics (specifically ontology) and cognitive science interesting. I think you'll find your ideas are actually very, very old. ;)