3 ms·
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
by stavros 2y ago
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
- exe34 2y agometa or smol?
- stavros 2y agoSmol
- jgalt212 2y agoI would hope Simon would not fall victim to such shenanigans, and has his own test dataset.
- stavros 2y agoI'm not saying either did this, just that that's what most fine tunes tend to do.
- simonw 2y agoMy test dataset is mostly dumb prompts about pelicans. You'll note that I didn't quote their benchmarks in my own post at all, because I didn't want to boost them without feeling confident in what they were stating. I posted about this because my own very limited initial experiments passed a loose vibe check! I'm impressed any time a 1.7GB (or 130MB) model file appears to be able to do anything useful at all.
- jerpint 2y agoDo you have proof for this? Why accuse one team and not the other?
- stavros 2y agoWhich team did I accuse? I said "usually".
- deleted 2y ago[deleted]