3 ms·

What surprises you about the unsupervised learning?

• disgruntledphd2 6y ago
  Normally supervised learning is much, much more effective.
  • wenc 6y ago
    I wonder if for random content discovery, unsupervised learning methods are less likely to be overconstrained by independent variables, and hence are freer to make better open-ended recommendations. Supervised algorithms almost assume too much about a user -- they assume the correlation structures that are true in the past are also likely to hold in the future. This assumption holds in deterministic environments, but is false or unnecessarily constraining in stochastic domains, where it is widely known simple models and heuristics tend to perform much better. I feel "new content discovery" is more of a stochastic problem rather than a deterministic one (which is the environment most conventional recommendation engines dwell in, hence most rec engines only make conservative recommendations). For all we know TikTok's algorithm could simply be a combination of rules of thumb + simple clusters + randomization that happen to work well. I've seen so many instances (in real life) where simple models vastly outperform complex models in stochastic situations. (p.s. in my opinion, YouTube's recommendations tend to be rather on the deterministic side. My recommendations seem to be mostly based on what I've watched and liked in the past and so the recs tend to be a bit boring. It clearly works great for YouTube from a monetization perspective, but it doesn't unearth a lot of interesting new content for me -- I have to search for those.)