2 ms·

I'm definitely not kidding. I'll point out that this is very much tractable (for instance Spotify runs a matrix factorization job every few days on a users x so

I'm definitely not kidding. I'll point out that this is very much tractable (for instance Spotify runs a matrix factorization job every few days on a users x songs matrix, maybe 50 million by 30 million, which is several orders of magnitude more). But anyway if you want to do that on your laptop, you don't have to use the entire space to build a good function, just a statistically representative sample of your dictionary will suffice.

beagle3 12y ago
Sparse matrix SVD (or, more likely, truncated/approximate SVD) like Spotify and Netflix contestants used to do is easy. But anything dense is not quite as easy (StefanKarpinski gave a $50K estimate for a machine with enough memory to do that -- I don't consider that "easy") Also, do you actually have any kind of experience with PCAing/SVDing an edit distance matrix? It's not easy to embed in a eucleadean geometry. If you have that experience, can you point to more reference material? If you don't, it would be prudent to say "I believe this might work though I have never tried anything similar".