3 ms·

Glad you like the post. I strongly agree with all of your points, especially custom loss functions can be a great tool. If the problem you are trying to solve

by nri 4y ago

Glad you like the post. I strongly agree with all of your points, especially custom loss functions can be a great tool. If the problem you are trying to solve has some grounding in e.g. physics you can even go a step further and let the model itself mirror the physical equations. It's like you say, of course these big models are really cool, but I feel like most of the popular machine learning online courses are too narrowly focused on them and many people discard useful techniques if they are not popular in kaggle competitions.

andersource 4y ago
I fully agree with your perspective, and I think there's a lot of cargo-culting in that area that explains your observations. Sure, if you're a FAANG collecting massive amounts of data comes almost for free, and it makes sense to find a way to properly utilize that data. But for many startups collecting that amount of data doesn't make sense (either because they don't have a lot of users yet or because their domain isn't high-freq digital activity or both), and in those cases it can be more worthwhile figuring out what to do with the little data they have than shoe-horning their way into Big Data (TM). There's much more money to be had convincing startups to use big data infrastructure and train huge nets on GPU clusters etc. than convincing them to iterate on small ad-hoc models developed in-house. Not saying that big data or models are necessarily wrong for small startups, just that they don't have to be the default.
Pamar 4y ago
One question (please take in account that I am no more than a dabbler in this field) - you conclude your piece with ... the nature of the problem slowly changes over time and prediction quality deteriorates. Isn't this a problem anyway, even with models based on much larger datasets?
- nri 4y ago
  You are right, data/concept drift affects both approaches. What I meant to say was that retraining your model might not fix that if you baked strong assumptions into it. I edited the blog post to make it clearer.
  - crabbygrabby 4y ago
    With small datasets you need to know how the concept drifts well enough to model it or model the failure modes. That's half the game in my opinion.

by nri 4y ago

andersource 4y ago

I fully agree with your perspective, and I think there's a lot of cargo-culting in that area that explains your observations. Sure, if you're a FAANG collecting massive amounts of data comes almost for free, and it makes sense to find a way to properly utilize that data. But for many startups collecting that amount of data doesn't make sense (either because they don't have a lot of users yet or because their domain isn't high-freq digital activity or both), and in those cases it can be more worthwhile figuring out what to do with the little data they have than shoe-horning their way into Big Data (TM). There's much more money to be had convincing startups to use big data infrastructure and train huge nets on GPU clusters etc. than convincing them to iterate on small ad-hoc models developed in-house. Not saying that big data or models are necessarily wrong for small startups, just that they don't have to be the default.

Pamar 4y ago

One question (please take in account that I am no more than a dabbler in this field) - you conclude your piece with ... the nature of the problem slowly changes over time and prediction quality deteriorates. Isn't this a problem anyway, even with models based on much larger datasets?

nri 4y ago
You are right, data/concept drift affects both approaches. What I meant to say was that retraining your model might not fix that if you baked strong assumptions into it. I edited the blog post to make it clearer.
- crabbygrabby 4y ago
  With small datasets you need to know how the concept drifts well enough to model it or model the failure modes. That's half the game in my opinion.