4 ms·
Giving a domain a hill to climb: benchmarking as data activation
- 2key 3mo agointeresting take not sure I 100% agree and yet, Interesting take
- galsapir 3mo agocurious where the disagreement lands: the claim i'm least sure of myself is that measurement alone already counts as activation (nothing in the weights changes, so it's a looser sense of the word than usual) the part i'd defend harder is the eval -> reward one: once a benchmark becomes the thing you train against, its flaws stop being measurement error and start being incentives. if you're pushing back somewhere in there, i'd genuinely like to hear it
- Terretta 3mo agoI wanted to like this and appreciated the genAI tokens were teased into a less AI-ish vernacular, but surfaces and honest caveats drifted back too far to overcome the "I'm human" signaling attempted through lowercase sentences and em dashes replaced with colons. I agree with the problem described within, and in the dead comment here.
- galsapir 3mo agothanks for reading it properly and engaging with the argument! writing is hard, expressing ideas cleanly is harder! working on it.
- zarzavat 3mo agoWhat's with the lack of capitalization?
- nnevatie 3mo agoIt is a new brand of a “cool” FAD.
- deleted 3mo ago[deleted]
- kjeiwoo 3mo agoThe flaw becomes an incentive...Goood point