3 ms·

I don't trust lmsys leaderboard anymore. I tried playing with the new model and on my very first prompt it told me a map that I asked it to analyze was sexuall

I don't trust lmsys leaderboard anymore. I tried playing with the new model and on my very first prompt it told me a map that I asked it to analyze was sexually explicit. First prompt! Do they have a QA team? Image was similar to this: https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex-2..v1569486091.png https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex...

rany_ 2y ago
Jeez are they literally just using 90s style pattern matching for their bad word filter? I guess it just errored out because the image contains the word "Penetration"
- ivanvanderbyl 2y ago
  I was working on a project where GitHub copilot refused to work in a currencies.go file because it contained “Vietnamese Dong”. Removing that line made it work.
jakderrida 2y ago
Sweet! I'll be hiding that under my mattress.
throwup238 2y ago
> Image was similar to this: https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex-2..v1569486091.png https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex... Anyone with even a passing knowledge of cartography will tell you that is one dirty, dirty map.
- zapkyeskrill 2y ago
  With a capital Pee to boot! FBI knocking on your door yet?
sigmoid10 2y ago
I wouldn't be surprised if they have another model classifying explicit images in their toolchain and don't even send anything to the actual LLM if it fails the test. In the same way OpenAI's models are thought to do segmentation and OCR as part of the toolchain, making them far better at certain vision tasks than other models. The only ways you can trust a model on lmsys (or any other test), is if it's open weights and you can run the model yourself. And even then you have to use private tests that were never put on the internet or submitted to any AI API if you want results to be reliable and untainted.
worstspotgain 2y ago
I'd argue the quality of the filtering and of the model itself should be scored separately. Since the filtering is a meta-level problem, it's inherently orthogonal. More generally, the cat is so out of the bag at this point that I'm starting to equate calls for more filtering and stricter regulation as just attacks on AI in general. Anything that will stick, anything but the kitchen sink.
- GuB-42 2y ago
  Here, I put it as a failure of the model. It is a LLM, it is supposed to understand language, or at least respond as if it does, here it didn't. Maybe that technically, filtering is a separate component, but from the reviewer perspective it doesn't matter, it is a misinterpretation, an incorrect answer. A filtering problem would if a model refused to answer about how porn is considered in some country because it is a sexual topic. That would be a correct interpretation (porn is a sexual topic indeed), but a filtering failure, because talking about that from a legal, cultural, religious,... perspective is usually not considered offensive. Anyways, ChatGPT doesn't have any problem with these questions (in text form). "broadband household penetration by state" is properly interpreted, though, if asked to, it would suggest using the world "coverage" instead to make it less awkward, which is a perfectly reasonable answer. Asking about how pornography is considered in some country also results in a perfectly appropriate answer about legal aspects, culture, etc...
  - worstspotgain 2y ago
    Well, sure, sometimes it can be hard to draw the line. The filter can be implemented differently (e.g. RLHF or not), leading to different ways in which it surfaces. The way I see it, these differences can be stark precisely because it's conceptually orthogonal. If it was closer to the knowledge, the models would filter the same because their knowledge corpus is roughly the same. Some other factors probably at play are the companies' different tolerance levels for Type I and Type II errors, and different aversion levels to jailbreaking. At some point the meta-knowledge aspects that define AI behavior will become a more integral part of the preparation, like Education students taking required classes in Pedagogy. I think that's when things can really start to go wrong. I'd much rather have a chatbot that just wants to please me than one that thinks it knows what's good for me.
littlestymaar 2y ago
obviously maps can be sexually explicit, there's /r/mapporn after all. /s
kbelder 2y ago
Same thing with me. I popped over and asked it one thing, and it said that it wouldn't answer me because it "won't create content that encourages self-loathing". Which isn't what I asked it to do, anyway. Maybe the model is high-quality if you can get a proper answer, but I would much rather have a lower-quality result with less filtering, guardrails, and condescension.
mamp 2y ago
Was it the "map of Tasmania"? Then I could understand why... https://info.umkc.edu/womenc/2016/05/20/showin-off-her-map-of-tasmania/#:~:text=The%20term%20“Map%20of%20Tasmania,really%20is%20up%20to%20you https://info.umkc.edu/womenc/2016/05/20/showin-off-her-map-o....
orbital-decay 2y ago
>I don't trust lmsys leaderboard anymore. Yes, and it's not even related to the filter. LMSYS Leaderboard is consistently all over the place lately. It's clearly some uncaught systemic bias or maybe outright manipulation, either way LMSYS results are not useful or sometimes even hard to believe/nonsensical for anyone who actually compared the models in the corresponding tasks for practical matters. It wasn't like that a few months ago.
- worstspotgain 2y ago
  I'm all for honest assessments, but this is one of the topics where I feel I have to push for constructive criticism. The LLM Arena is maybe the only comparison tool that's collectively seen as above suspicion, which is probably why it's under attack (if it actually is.) Benchmarks and tests are too easy to fit the model to, and there is a proliferation of them so they can be cherry picked. So unless we can point to something better, we can be constructive and just improve LMSYS by using it.
  - orbital-decay 2y ago
    What if we cannot point to something better? Freeform human ranking is too easy to manipulate as each model has recognizable outputs, and publicity boost might be enough to make manipulations appealing. To be more constructive, here's a few things that make me instantly suspicious of this leaderboard in its current state. I'm talking about the API experience of course, not web frontends with unspecified amount of instructions behind the scenes. - Anything ranking above Anthropic models in non-English languages, maybe except Gemini. - 4o mini ranking above Sonnet. Seriously? It handles every task I give it worse than Sonnet, which is consistent with its purpose. - Sonnet and 4o ranking way above Opus in multiturn conversations. Every model is plagued by self-reinforced repetition in multiturn, but 4o and Sonnet are especially bad at this, the latter is nearly unusable in multiturn due to this. They're clearly not optimized for multiturn chats. While Anthropic models have terrible long context degradation and lost-in-the-middle issue, Opus is pretty reluctant to repeat itself, and can actually be used in multiturn, despite often being less performant than both. I also have plenty of questions regarding open-weights models as well.
attentive 2y ago
gpt-4o-mini on top of 3.5 sonnet and way above Opus is a suspect for sure.
snthpy 2y ago
Does disabling the content safety filtering not work?

rany_ 2y ago

Jeez are they literally just using 90s style pattern matching for their bad word filter? I guess it just errored out because the image contains the word "Penetration"

ivanvanderbyl 2y ago
I was working on a project where GitHub copilot refused to work in a currencies.go file because it contained “Vietnamese Dong”. Removing that line made it work.

jakderrida 2y ago

Sweet! I'll be hiding that under my mattress.

throwup238 2y ago

> Image was similar to this: https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex-2..v1569486091.png https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex... Anyone with even a passing knowledge of cartography will tell you that is one dirty, dirty map.

zapkyeskrill 2y ago
With a capital Pee to boot! FBI knocking on your door yet?

sigmoid10 2y ago

I wouldn't be surprised if they have another model classifying explicit images in their toolchain and don't even send anything to the actual LLM if it fails the test. In the same way OpenAI's models are thought to do segmentation and OCR as part of the toolchain, making them far better at certain vision tasks than other models. The only ways you can trust a model on lmsys (or any other test), is if it's open weights and you can run the model yourself. And even then you have to use private tests that were never put on the internet or submitted to any AI API if you want results to be reliable and untainted.

worstspotgain 2y ago

I'd argue the quality of the filtering and of the model itself should be scored separately. Since the filtering is a meta-level problem, it's inherently orthogonal. More generally, the cat is so out of the bag at this point that I'm starting to equate calls for more filtering and stricter regulation as just attacks on AI in general. Anything that will stick, anything but the kitchen sink.

GuB-42 2y ago
Here, I put it as a failure of the model. It is a LLM, it is supposed to understand language, or at least respond as if it does, here it didn't. Maybe that technically, filtering is a separate component, but from the reviewer perspective it doesn't matter, it is a misinterpretation, an incorrect answer. A filtering problem would if a model refused to answer about how porn is considered in some country because it is a sexual topic. That would be a correct interpretation (porn is a sexual topic indeed), but a filtering failure, because talking about that from a legal, cultural, religious,... perspective is usually not considered offensive. Anyways, ChatGPT doesn't have any problem with these questions (in text form). "broadband household penetration by state" is properly interpreted, though, if asked to, it would suggest using the world "coverage" instead to make it less awkward, which is a perfectly reasonable answer. Asking about how pornography is considered in some country also results in a perfectly appropriate answer about legal aspects, culture, etc...
- worstspotgain 2y ago
  Well, sure, sometimes it can be hard to draw the line. The filter can be implemented differently (e.g. RLHF or not), leading to different ways in which it surfaces. The way I see it, these differences can be stark precisely because it's conceptually orthogonal. If it was closer to the knowledge, the models would filter the same because their knowledge corpus is roughly the same. Some other factors probably at play are the companies' different tolerance levels for Type I and Type II errors, and different aversion levels to jailbreaking. At some point the meta-knowledge aspects that define AI behavior will become a more integral part of the preparation, like Education students taking required classes in Pedagogy. I think that's when things can really start to go wrong. I'd much rather have a chatbot that just wants to please me than one that thinks it knows what's good for me.

littlestymaar 2y ago

obviously maps can be sexually explicit, there's /r/mapporn after all. /s

kbelder 2y ago

Same thing with me. I popped over and asked it one thing, and it said that it wouldn't answer me because it "won't create content that encourages self-loathing". Which isn't what I asked it to do, anyway. Maybe the model is high-quality if you can get a proper answer, but I would much rather have a lower-quality result with less filtering, guardrails, and condescension.

mamp 2y ago

Was it the "map of Tasmania"? Then I could understand why... https://info.umkc.edu/womenc/2016/05/20/showin-off-her-map-of-tasmania/#:~:text=The%20term%20“Map%20of%20Tasmania,really%20is%20up%20to%20you https://info.umkc.edu/womenc/2016/05/20/showin-off-her-map-o....

orbital-decay 2y ago

>I don't trust lmsys leaderboard anymore. Yes, and it's not even related to the filter. LMSYS Leaderboard is consistently all over the place lately. It's clearly some uncaught systemic bias or maybe outright manipulation, either way LMSYS results are not useful or sometimes even hard to believe/nonsensical for anyone who actually compared the models in the corresponding tasks for practical matters. It wasn't like that a few months ago.

worstspotgain 2y ago
I'm all for honest assessments, but this is one of the topics where I feel I have to push for constructive criticism. The LLM Arena is maybe the only comparison tool that's collectively seen as above suspicion, which is probably why it's under attack (if it actually is.) Benchmarks and tests are too easy to fit the model to, and there is a proliferation of them so they can be cherry picked. So unless we can point to something better, we can be constructive and just improve LMSYS by using it.
- orbital-decay 2y ago
  What if we cannot point to something better? Freeform human ranking is too easy to manipulate as each model has recognizable outputs, and publicity boost might be enough to make manipulations appealing. To be more constructive, here's a few things that make me instantly suspicious of this leaderboard in its current state. I'm talking about the API experience of course, not web frontends with unspecified amount of instructions behind the scenes. - Anything ranking above Anthropic models in non-English languages, maybe except Gemini. - 4o mini ranking above Sonnet. Seriously? It handles every task I give it worse than Sonnet, which is consistent with its purpose. - Sonnet and 4o ranking way above Opus in multiturn conversations. Every model is plagued by self-reinforced repetition in multiturn, but 4o and Sonnet are especially bad at this, the latter is nearly unusable in multiturn due to this. They're clearly not optimized for multiturn chats. While Anthropic models have terrible long context degradation and lost-in-the-middle issue, Opus is pretty reluctant to repeat itself, and can actually be used in multiturn, despite often being less performant than both. I also have plenty of questions regarding open-weights models as well.

attentive 2y ago

gpt-4o-mini on top of 3.5 sonnet and way above Opus is a suspect for sure.

snthpy 2y ago

Does disabling the content safety filtering not work?