3 ms·

> That's one heck of a hard question! Exactly. That's the problem. It's not just a question of deciding whether the data is random or has some particular prope

> That's one heck of a hard question! Exactly. That's the problem. It's not just a question of deciding whether the data is random or has some particular property, but trying to answer the question of whether there is _any_ signal buried in the data -- and then deciding whether that signal is meaningful. For the most part, unsupervised learning is more like data pre-processing than an alternative to hypothesis testing. General frequentist statistical techniques (including hypothesis testing, goodness of fit, etc.) are an integral part of evaluating the results of unsupervised learning.

graycat 11y ago
From a paper I published, go to a server farm and for three months collect data on each of 10 variables 100 times a second. Now, in real time, given values on each of those 10 variables, ask if the farm is sick or healthy? Get known, adjustable false alarm rate and a relatively high detection rate. So, it's a multi-dimensional, distribution-free hypothesis test. Don't know what patterns or properties are looking for and don't get a diagnosis, but still, if get a detection at a low false alarm rate, then have good evidence the farm is sick. So, this hypothesis test is not the same as you have described. So, is this unsupervised learning?