3 ms·
They gesture at not using benchmarks for some reason...
by simianwords 19d ago
They gesture at not using benchmarks for some reason...
- meric_ 19d agohttps://typesafe.ai/blog/antibenchmaxxing https://typesafe.ai/blog/antibenchmaxxing But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do