4 ms·
I mean all optimization algorithms do is game a benchmark. That’s the whole point. The hard part is making the benchmark meaningful in the first place.
by dartos 2y ago
I mean all optimization algorithms do is game a benchmark. That’s the whole point.
The hard part is making the benchmark meaningful in the first place.
- TeMPOraL 2y agoYeah, and if anything, RL has a rep of being too good at this job, because of all the cases where it gamed a benchmark by picking up on some environmental factor the supervisors hadn't thought of (numerical instabilities, rounding, bugs, etc.).
- porridgeraisin 2y agoMy favourite is this one: https://news.ycombinator.com/item?id=43113941 https://news.ycombinator.com/item?id=43113941
- magicalhippo 2y agoThe ML version of Professor Farnsworth[1]: It came to me in a dream, and I forgot it in another dream. [1]: https://www.imdb.com/title/tt0584424/quotes/?item=qt0439248 https://www.imdb.com/title/tt0584424/quotes/?item=qt0439248
- einpoklum 2y agoNo, that is patently false. Many optimization algorithms which computer scientists, mathematicians or software developers devise do not involve benchmakrs at all, and apply to all possible inputs/instances of their respective computational problems.
- CuriouslyC 2y agoPlot twist: the loss function for training is basically a benchmark
- dartos 2y agoIs it a plot twist if it’s the whole plot?
- brookst 2y agoExample?
- hgomersall 2y agoThose times when people write code with loads of theoretical micro optimisations that they never actually test against a benchmark because otherwise they wouldn't do it.
- brookst 2y agoPerhaps “example” means different things in different cultures.
- hgomersall 2y agoI was being somewhat facetious.