27 ms·
Can someone please explain how these models aren’t just fine tuned for benchmarks? I’m not plugged in to this space much but it seems like such an obvious probl
by namuol 2mo ago
Can someone please explain how these models aren’t just fine tuned for benchmarks? I’m not plugged in to this space much but it seems like such an obvious problem…
- sroerick 2mo agoThey definitely are - but also people are using them pretty extensively for work. So ultimately you can't really fake "is it good". But there's no real measurements of that when a model is released, so we are stuck with benchmarks.
- namuol 2mo agoI will continue to ignore the benchmarks.