4 ms·
I just spent yesterday applying Kaparthy's autoresearch on an ML problem. I teach ML for a living and was amazed with what the tokens gave back to me after man
by __mharrison__ 6mo ago
I just spent yesterday applying Kaparthy's autoresearch on an ML problem.
I teach ML for a living and was amazed with what the tokens gave back to me after many rounds of experiments. If Kaggle was still a thing, AI would generally beat it.
The challenge I've seen is that most data science/ml modeling work is quite weak. Folks don't even know the basic tools well. Not sure if giving AI to them will really open up many doors to them.
As always experts love minions of juniors doing their deeds. Non-experts get to wade through slop.
- twelfthnight 6mo agoI agree AI could probably do a decent job on Kaggle problems. Of course, almost no DS job is building models with well-defined objectives and perfect data. The DS and MLE folks I work with mostly spend their time reframing ill-posed product requests into ML systems that can be maintained and improved with feedback loops. A _huge_ part of a DS is saying "No" to bad ideas posed by non-experts. The issue with LLMs is all they ever say is "Yes" and "Wow, that's such a great idea!"
- deleted 6mo ago[deleted]
- jimbokun 6mo agoThe data scientist is like in house lawyers in that respect.
- mscbuck 6mo agoYeah, once you move onto legitimate business evaluation metrics (where Precision@k or Recall@k don't actually fit your business model without modification), GPTs just seem to suffer without context, and hey, knowing the context is part of what gives a data scientist his value.
- LaserPineapple 6mo agoIs Kaggle no longer a thing?
- whattheheckheck 6mo agoKarparthys autoresearch is just automated overfitting no?