3 ms·

> In my experience, at least, the value in this kind of activity is relatively low. Sometimes you find something, usually you just find a low-rank version of no

> In my experience, at least, the value in this kind of activity is relatively low. Sometimes you find something, usually you just find a low-rank version of nothing in particular. Unless some significant developments appear in this space (and they very well might), the best low-dimensional approximations will be the means, variances, and correlations computed by business analysts, which are tremendously valuable quantities to keep in mind at all times. Definitely. Humans find and reject "low rank" correlations all the time. For example, North America and South America are both part of the "new world", but no sociologist would use that to make general predictions about the people on the continent. People both like steak and cookies, and would conceivably fall into a "low rank" "humans like this category", but they couldn't be more dissimilar in many respects. Conversely, some doctors have an encyclopedic knowledge of various ailments, and either memorize or infer from experience a particular ailment from a huge collection of possible ailments. If the number of possible ailments is large, their "understanding" is not low rank, it's the opposite. Low rank does not necessarily mean "understanding", it's just a local minimum in some function of a random variable.

ssivark 8y ago
I appreciate the thought-provoking examples :-) > Humans find and reject "low rank" correlations all the time ... steak and cookies ... Sure, but the appropriateness of a low rank approximation depends on what you want to predict. Eg: for predicting basketball success, {height, wingspan} might be a very useful "low rank" description, while for predicting obesity the relevant low rank description might be the body mass index (BMI) or something like it. Wingspan = chest width + 2 * arm length and BMI = weight/height^2 could be composite features "discovered" by such a model+inference toolkit. > Conversely, some doctors have an encyclopedic knowledge of various ailments, and either memorize or infer from experience a particular ailment from a huge collection of possible ailments. If the number of possible ailments is large, their "understanding" is not low rank, it's the opposite. That's exactly why I would not call that understanding, especially if they have to memorize a book full of ailments and symptoms. "Understanding" would entail a (causal) model of underlying physiological problems and the symptoms they give rise to, with a recipe for inferring in reverse. > Low rank does not necessarily mean "understanding", it's just a local minimum in some function of a random variable. I don't understand that statement at all. PS: I don't have any particular affinity to neural networks over other ML models, and don't mean to escalate the hype :-)
- nerdponx 8y ago
  composite features "discovered" by such a model+inference toolkit. This kind of "feature discovery" is sort of what random forests, gradient boosting, and neural networks already do. You can even do a neutered version of it with full quadratic interactions of all the inputs in a (regularized) linear regression model. For a linear version of this task as an unsupervised problem, you will want to look at PCA. The problem right now is not that we don't have enough unsupervised data analysis techniques, it's that they are hard to extract useful information out of. Interpreting t-SNE, for example, is notoriously difficult. This is why I said advances in the space are necessary. As it stands, you can run all sorts of low dimensional mappings and embeddings on a given data set, and spend an afternoon finding a whole lot of nothing that your business analysts didn't already know.
nerdponx 8y ago
Interesting aside about doctors though. Many people in alternative medicine would argue that there are obvious associations between various ailments, causes, and/or treatments that are under-recognized and under-studied. Maybe if a machine produced those same associations instead of an alternative medicine practitioner, they might be more interesting to medical researchers.
- taeric 8y ago
  You have to be careful. If you only feed data that has the associations, both humans and machines will find them. This is the entire premise behind introducing double blind studies. Turns out you have to actively try and falsify claims if you want solid results. Not just try and confirm them. :)

ssivark 8y ago

I appreciate the thought-provoking examples :-) > Humans find and reject "low rank" correlations all the time ... steak and cookies ... Sure, but the appropriateness of a low rank approximation depends on what you want to predict. Eg: for predicting basketball success, {height, wingspan} might be a very useful "low rank" description, while for predicting obesity the relevant low rank description might be the body mass index (BMI) or something like it. Wingspan = chest width + 2 * arm length and BMI = weight/height^2 could be composite features "discovered" by such a model+inference toolkit. > Conversely, some doctors have an encyclopedic knowledge of various ailments, and either memorize or infer from experience a particular ailment from a huge collection of possible ailments. If the number of possible ailments is large, their "understanding" is not low rank, it's the opposite. That's exactly why I would not call that understanding, especially if they have to memorize a book full of ailments and symptoms. "Understanding" would entail a (causal) model of underlying physiological problems and the symptoms they give rise to, with a recipe for inferring in reverse. > Low rank does not necessarily mean "understanding", it's just a local minimum in some function of a random variable. I don't understand that statement at all. PS: I don't have any particular affinity to neural networks over other ML models, and don't mean to escalate the hype :-)

nerdponx 8y ago
composite features "discovered" by such a model+inference toolkit. This kind of "feature discovery" is sort of what random forests, gradient boosting, and neural networks already do. You can even do a neutered version of it with full quadratic interactions of all the inputs in a (regularized) linear regression model. For a linear version of this task as an unsupervised problem, you will want to look at PCA. The problem right now is not that we don't have enough unsupervised data analysis techniques, it's that they are hard to extract useful information out of. Interpreting t-SNE, for example, is notoriously difficult. This is why I said advances in the space are necessary. As it stands, you can run all sorts of low dimensional mappings and embeddings on a given data set, and spend an afternoon finding a whole lot of nothing that your business analysts didn't already know.

nerdponx 8y ago

Interesting aside about doctors though. Many people in alternative medicine would argue that there are obvious associations between various ailments, causes, and/or treatments that are under-recognized and under-studied. Maybe if a machine produced those same associations instead of an alternative medicine practitioner, they might be more interesting to medical researchers.

taeric 8y ago
You have to be careful. If you only feed data that has the associations, both humans and machines will find them. This is the entire premise behind introducing double blind studies. Turns out you have to actively try and falsify claims if you want solid results. Not just try and confirm them. :)