4 ms·

"Unsupervised learning"? Well, setting aside the jargon, one approach is just a statistical hypothesis test, and a lot is known about that.

by graycat 11y ago

nl 11y ago
Unsupervised feature extraction (eg in visual tasks with backprog) goes well beyond what would conventionally be called "a statistical hypothesis test". For example, look at the image of the features learnt by the AlexNet CNN[1]. In a more conventional "statistical hypothesis test" these would be manually defined by hand. IMHO a system that automatically extracts those features itself deserves to be called more than "just a statistical hypothesis test" [1] "Example filters learned by Krizhevsky et al" http://cs231n.github.io/convolutional-networks/ http://cs231n.github.io/convolutional-networks/
- graycat 11y ago
  Clearly my remark was > ... one approach is just a statistical hypothesis test ... So we have no disagreement. E.g., is this JPG an image of a human face? So, for some variables, extract data on those variables from known (supervised?) human faces and do an hypothesis test with null hypothesis that the JPG is of a human face. Or, are these human faces like those human faces (unsupervised)? Then get data on the variables and do a two sample test with null hypothesis that the two collections of faces are the same, i.e., from the same distribution.
  - nl 11y ago
    Well yeah. But that is like saying that IBM Watson is just a statistical hypothesis test: "Is this answer or this answer correct?" That is absolutely true, and completely useless. The hard part isn't telling the difference once you've done the feature extraction, it's doing the feature extraction automatically itself. (Actually that undersells it too, because you have so many combinations of features to evaluate that making that part work is non-trivial too. But at least that part is just calculus)
    - graycat 11y ago
      We are being too brief. Here is a relatively non-standard approach to hypothesis testing: https://news.ycombinator.com/item?id=10484602 https://news.ycombinator.com/item?id=10484602 So, that was looking for bias. The training was to say what people were the women. So, could use that to say if these people were like those people. So, could get some people recognition. Or, go to a valley with a lot of rocks and for each of 1000 rocks measure 10 quantities. Then here is one more rock; did it come from that valley? So, doing valley rock recognition. Here are 1000 faces from North Korea and 500 from South Korea. Measure 10 quantities on each and ask if the faces are different (this would be a multi-dimensional two sample test, distribution-free). Here's a more general picture: Take some data. Ask if the data has property X. Make a null hypothesis that the data does have property X, and with that assumption take a function of the data and calculate some probability that would be high if the data has property X. Then see what the probability is for the data do have. It that probability is low, then reject the null hypothesis and conclude that the data does not have property X. So, have now recognized that the data is not X. It may from some further knowledge that conclude that the data has property Y. This might help a self driving car decide if the stuff in the road is just some harmless paper and can drive over it or if it is dangerous and must avoid it, if the ice about to step on is solid, if that knob is to be pulled on or twisted, ..., a lot of questions in AI. I'm being brief. Questions?
      - cgearhart 11y ago
        > Take some data. Ask if the data has property X. Unsupervised learning would ask, "does this bag of data have any properties?" or "what are the most significant properties in this bag of data?" or "are there any partitions of this bag of data that seem to be more similar than others?" If you know what you're looking for in advance then there are more direct ways to search for it (like hypothesis testing). Unsupervised learning is what you get when you relax the requirement to know in advance what it is you want to test, and relax the requirement that you can tell once you find it.
        graycat 11y ago
        Ah, we're getting a C in communications. > "does this bag of data have any properties?" WOW! That's one heck of a hard question! There can be a LOT of properties! No properties might mean random. So, one case is old random number testing! People settled on the Fourier test -- essentially hypothesis testing. You seem to be saying that hypothesis testing requires knowing something too specific about what is being looked for. Well, in some of the elementary stuff. But there's the old distribution free two sample tests: Joe teaches calculus to 30 students, and Mary teaches the same subject to 25 students. Is their different performances significant? Specific? We are not saying much about what is different, but we do get estimates of Type I error, that is, saying that there is a big difference when there is not. So, how? Put all 30 + 25 student scores (right, something specific, unidimensional, but see below) in a pot, stir briskly, pull out 30 and 25, that the difference in their averages, do this 1000 times, get an empirical distribution of the difference under the hypothesis that there is no difference, then see where the actual difference is in the empirical distribution and estimate false alarm rate. For measuring several quantities not just one, can look for a significant difference even if have no idea what the heck the cause is. So, are not being specific about a cause. Generally in making AI decisions, what to have some sense of being correct. A first cut approach is hypothesis testing. A second cut would be, say, looking for a diagnosis. So, have some data and 20 candidate diagnoses. So, do hypothesis tests on the appropriate data for each of the 20, maybe different parts of the data for each of the 20, and pick the diagnosis based on the Type I estimates. Hypothesis testing has a cute and powerful idea: Make an hypothesis of no effect and use that to calculate some probabilities. There's got to be a good role for that in AI.
cgearhart 11y ago
Unsupervised learning is broad set of techniques that seek to infer structure from data, which is pretty fundamentally different from hypothesis testing. Where a hypothesis test asks "is <assertion> true about the structure in this data?", unsupervised techniques ask "is there any structure in this data?"
- graycat 11y ago
  We're not communicating well. I said a little more in https://news.ycombinator.com/item?id=10543615 https://news.ycombinator.com/item?id=10543615 just above.

"Unsupervised learning"? Well, setting aside the jargon, one approach is just a statistical hypothesis test, and a lot is known about that.

by graycat 11y ago

"Unsupervised learning"? Well, setting aside the jargon, one approach is just a statistical hypothesis test, and a lot is known about that.

nl 11y ago

Unsupervised feature extraction (eg in visual tasks with backprog) goes well beyond what would conventionally be called "a statistical hypothesis test". For example, look at the image of the features learnt by the AlexNet CNN[1]. In a more conventional "statistical hypothesis test" these would be manually defined by hand. IMHO a system that automatically extracts those features itself deserves to be called more than "just a statistical hypothesis test" [1] "Example filters learned by Krizhevsky et al" http://cs231n.github.io/convolutional-networks/ http://cs231n.github.io/convolutional-networks/

graycat 11y ago
Clearly my remark was > ... one approach is just a statistical hypothesis test ... So we have no disagreement. E.g., is this JPG an image of a human face? So, for some variables, extract data on those variables from known (supervised?) human faces and do an hypothesis test with null hypothesis that the JPG is of a human face. Or, are these human faces like those human faces (unsupervised)? Then get data on the variables and do a two sample test with null hypothesis that the two collections of faces are the same, i.e., from the same distribution.
- nl 11y ago
  Well yeah. But that is like saying that IBM Watson is just a statistical hypothesis test: "Is this answer or this answer correct?" That is absolutely true, and completely useless. The hard part isn't telling the difference once you've done the feature extraction, it's doing the feature extraction automatically itself. (Actually that undersells it too, because you have so many combinations of features to evaluate that making that part work is non-trivial too. But at least that part is just calculus)
  - graycat 11y ago
    We are being too brief. Here is a relatively non-standard approach to hypothesis testing: https://news.ycombinator.com/item?id=10484602 https://news.ycombinator.com/item?id=10484602 So, that was looking for bias. The training was to say what people were the women. So, could use that to say if these people were like those people. So, could get some people recognition. Or, go to a valley with a lot of rocks and for each of 1000 rocks measure 10 quantities. Then here is one more rock; did it come from that valley? So, doing valley rock recognition. Here are 1000 faces from North Korea and 500 from South Korea. Measure 10 quantities on each and ask if the faces are different (this would be a multi-dimensional two sample test, distribution-free). Here's a more general picture: Take some data. Ask if the data has property X. Make a null hypothesis that the data does have property X, and with that assumption take a function of the data and calculate some probability that would be high if the data has property X. Then see what the probability is for the data do have. It that probability is low, then reject the null hypothesis and conclude that the data does not have property X. So, have now recognized that the data is not X. It may from some further knowledge that conclude that the data has property Y. This might help a self driving car decide if the stuff in the road is just some harmless paper and can drive over it or if it is dangerous and must avoid it, if the ice about to step on is solid, if that knob is to be pulled on or twisted, ..., a lot of questions in AI. I'm being brief. Questions?
    - cgearhart 11y ago
      > Take some data. Ask if the data has property X. Unsupervised learning would ask, "does this bag of data have any properties?" or "what are the most significant properties in this bag of data?" or "are there any partitions of this bag of data that seem to be more similar than others?" If you know what you're looking for in advance then there are more direct ways to search for it (like hypothesis testing). Unsupervised learning is what you get when you relax the requirement to know in advance what it is you want to test, and relax the requirement that you can tell once you find it.
      - graycat 11y ago
        Ah, we're getting a C in communications. > "does this bag of data have any properties?" WOW! That's one heck of a hard question! There can be a LOT of properties! No properties might mean random. So, one case is old random number testing! People settled on the Fourier test -- essentially hypothesis testing. You seem to be saying that hypothesis testing requires knowing something too specific about what is being looked for. Well, in some of the elementary stuff. But there's the old distribution free two sample tests: Joe teaches calculus to 30 students, and Mary teaches the same subject to 25 students. Is their different performances significant? Specific? We are not saying much about what is different, but we do get estimates of Type I error, that is, saying that there is a big difference when there is not. So, how? Put all 30 + 25 student scores (right, something specific, unidimensional, but see below) in a pot, stir briskly, pull out 30 and 25, that the difference in their averages, do this 1000 times, get an empirical distribution of the difference under the hypothesis that there is no difference, then see where the actual difference is in the empirical distribution and estimate false alarm rate. For measuring several quantities not just one, can look for a significant difference even if have no idea what the heck the cause is. So, are not being specific about a cause. Generally in making AI decisions, what to have some sense of being correct. A first cut approach is hypothesis testing. A second cut would be, say, looking for a diagnosis. So, have some data and 20 candidate diagnoses. So, do hypothesis tests on the appropriate data for each of the 20, maybe different parts of the data for each of the 20, and pick the diagnosis based on the Type I estimates. Hypothesis testing has a cute and powerful idea: Make an hypothesis of no effect and use that to calculate some probabilities. There's got to be a good role for that in AI.

cgearhart 11y ago

Unsupervised learning is broad set of techniques that seek to infer structure from data, which is pretty fundamentally different from hypothesis testing. Where a hypothesis test asks "is <assertion> true about the structure in this data?", unsupervised techniques ask "is there any structure in this data?"

graycat 11y ago
We're not communicating well. I said a little more in https://news.ycombinator.com/item?id=10543615 https://news.ycombinator.com/item?id=10543615 just above.