3 ms·
In general search can be machine learning. Google certainly uses machine learning as part of it's ranking. I guess you can make the argument that even tf-idf a
by ma2rten 8y ago
In general search can be machine learning. Google certainly uses machine learning as part of it's ranking.
I guess you can make the argument that even tf-idf as described in the article is a form of unsupervised machine learning because you obtain ("learn") the idf from the data.
- jahewson 8y agoTF-IDF is a feature extracted from the data, much like a simple count of words, but it is not learned. It is simply computed. An example of learned features are word embeddings where it is necessary to train on data to obtain them. If you want to apply machine learning to search then you need clickstream data, embeddings, or learned feature weights.
- yorwba 8y agoWord embeddings are also "simply computed." If you use GloVe, then the vectors are obtained by factoring a matrix of co-occurence counts. The difference between machine learning and "simple" feature extraction is mostly just in the choice of metaphors used to describe the computation, not in any fundamental properties.
- ma2rten 8y agoRight. Naive Bayes is considered to be a machine learning algorithm, but also consists of just "simple counting".
- drongoking 8y agoYour distinction between TFIDF as simply computed vs embeddings as learned is odd and artificial. Both are computations from data, but TFIDF has an understandable closed form and word embeddings do not. As for machine learning, it has to do with improvement and doesn't even necessarily need data.
- jahewson 8y agoI think you’re right. It’s the improvement in the learning process that’s the important bit. TF-IDF lacks that.