3 ms·
Everybody brings their own terminology to these issues, given their background. This is all just statistics. While I think the intuition of your terminology m
by tristanz 12y ago
Everybody brings their own terminology to these issues, given their background. This is all just statistics. While I think the intuition of your terminology makes some sense, it's probably the wrong frame. It's important for people to use same terminology, with a clear mathematical definition. Unfortunately there's still a big gap between statistics and machine learning communities.
Everything in "learning" follows a good parameterization of p(y, x), the joint distribution of the data, whether unobserved or not. If you have that, you can get everything else.
The core idea of representational learning is important, even though it's obvious in hindsight. When doing statistics, many people assume the parameters governing the conditional distribution p(y | x) are distinct from those governing p(x). So if you're just interested in predicting y, you don't need to model p(x) to get the posterior distribution p(y | x). Representational learning suggests that the parameters of these distributions are coupled. It's saying that if you understand the structure of the world -- you have a good representation of p(x) -- you can make better predictions p(y | x) with less data. This makes a lot of sense because the distinction between x and y is totally arbitrary.