3 ms·
I've worked in recommender systems for a while, and it's great to see them publicized. SASRec was released in 2018 just after transformer paper, and uses the s
by skayvr 11mo ago
I've worked in recommender systems for a while, and it's great to see them publicized.
SASRec was released in 2018 just after transformer paper, and uses the same attention mechanism but different losses than LLMs. Any plans to upgrade to other item/user prediction models?
- costco 11mo agoI'm not an expert by any means but as far as sequential recommendations go, aren't SASRec and its derivatives pretty much the name of the game? I probably should have looked into HSTUs more. Also this / sparse transformers in general: https://arxiv.org/pdf/2212.04120 https://arxiv.org/pdf/2212.04120
- bigskydog 11mo agoRecommend OneRec which is an improvement of HSTU and it recently became open source
- skayvr 11mo agoThere's a few alternatives, but SASRec is a good baseline for next-item recommendation. I'd look at BERT4Rec too. HSTU is definitely a strong step forward, but stays in the domain of ID models. HSTU also seems to rely heavily on some extra item information that SASRec does not (timestamps). Other models include Google's TIGER model which uses a VAE to encode more information about items. Similar to how modern text-to-voice operates.
- costco 11mo agoThank you for the recommendations. I didn't try BERT4Rec because I assumed it would perform the same or worse as what I already had after having read https://dl.acm.org/doi/pdf/10.1145/3699521 https://dl.acm.org/doi/pdf/10.1145/3699521. The TIGER paper seems interesting - I definitely want to explore semantic IDs in general and also because I think it could allow including more long-tail items.