4 ms·
This actually speaks volumes about the progress AI hs made in recent years. Its capabilities have become so intangible that people begin to attack standardised
by sigmoid10 2y ago
This actually speaks volumes about the progress AI hs made in recent years. Its capabilities have become so intangible that people begin to attack standardised testing - just like they do for humans. Because we all know that a human who does good on a test might still suck at the real job and vice-versa. If we get to the point where you we need to do personal interviews with new models to see if they could be used for a certain job, a lot of people will get the denial rug pulled out from under them pretty hard.
- viraptor 2y ago> where you we need to do personal interviews with new models to see if they could be used for a certain job This is already happening. See the arenas like https://chat.lmsys.org/ https://chat.lmsys.org/ where you get two blinded answers and choose the better one, or choose specific models to compare.
- sigmoid10 2y agoI already know about this, but it is more of a playground than a serious testing environment. I'm talking full blown multi-round interviews with varying human teams to look at all aspects of the job. I have not seen anyone outside the field put these kinds of resources into evaluating models for their business purpose. But I feel we are not far from that.
- htrp 2y agoUnlike humans, you can make an llm do the job you want it to do as part of the interview process.
- sigmoid10 2y agoOnly for very narrow and strictly defined use cases.