4 ms·

It is common to switch to a conservative mode of thought upon entering one's domain of competence; The question is, though, if the expertise and intuition deve

It is common to switch to a conservative mode of thought upon entering one's domain of competence; The question is, though, if the expertise and intuition developed during, say, running XGboost classifiers at scale in AdTech really of much relevance when thinking about large transformer models trained with a self-supervised objective and RLHF? If you try to study this in depth, these models can do something the usual "datascience"-tier ones commonly cannot: https://arxiv.org/abs/2205.10343 https://arxiv.org/abs/2205.10343 https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ https://moultano.wordpress.com/2020/10/18/why-deep-learning-...

benreesman 4y ago
If you’re curious about how we used ensembled weak learners to form a strong enough learner to kick the shit out of everyone except DoubleClick in AdTech, this is a decent start: https://en.m.wikipedia.org/wiki/Ensemble_learning https://en.m.wikipedia.org/wiki/Ensemble_learning. By the standards of AdTech, latent space stuff was pretty unproven when I left the game a few years ago. Low-rank approximations were in some sense implicit in that, but were not at that time an explicit goal. If I were doing an AdTech system from scratch I’d most likely reach for the kind of recommender systems that you’re alluding to sooner. Sounds like you know your stuff :)