3 ms·

AI Model Weight Providers Should Not Police Uses, No Matter How Awful They Are

1 points by andy99 3y ago

robswc 3y ago
Yea. Honestly there's nothing worse than getting a "Sorry, as an AI language model, I cannot do..." This is especially awful when you're asking it to do something like "estimate the number of X in Y" or some task that it could clearly help with. I know exactly why they do this, its the safe option... but it is frustrating to deal with.
- orbz 3y ago
  There’s a difference between “can’t” and “won’t”. I don’t think there’s anything wrong with it being up front that an attempt to answer a question is inherently impossible for it to answer. Information like estimating X in Y may or may not be possible from the underlying training data depending on X and Y.
  - robswc 3y ago
    That's fair. Usually though when I ask these questions, it can provide an answer because it gives the information it needs in the response, it just doesn't want to take ownership of the statement "I guess there were X number of Y." I can usually get a rough estimate (that definitely makes sense or checks out) but it takes a lot of back and forth. Most memorable example was getting estimates of cars sold and what make/model. It would say "there were X of Y sold in 2020, but I cannot provide an estimate" only after some back and forth, so it certainly could have told me earlier.
    - orbz 3y ago
      That’s the thing though, it can be convinced to make a statement with conviction but that might not be based on fact. I just got ChatGPT4 to say that there might have been a few hundred mint green Kia Cherokees sold in 1997.
      - robswc 3y ago
        I guess it depends on how accurate you need the info to be. I double check its answers and it provides decent estimates every time but only after getting around the "as an AI model I cannot..." wall.
- raxxorraxor 3y ago
  We also saw that with image synthesis. As you can imagine, if you scrap the internet for content, any model will tend to generate "controversial" images itself. An example is stable diffusion. Of course there is interest in making it commercially viable or just SFW in general. The 2.0 model version was lobotomized to not create anything "bad". As a result it was pretty useless. There was a successor version 2.1 that did it to a lesser degree, but the old 1.5 model was still more flexible. Many user prefer it even if the training data for 2.0+ had higher resolution images. There are new models were we have to see. But I think the overall tendency to lobotomize AI is pretty negative for its overall development. There should always be an option for the educated user. Sadly, unlike with LLM, it is harder to circumvent these restriction by changing your input.
orbz 3y ago
I think the author is missing a really key piece here: they’re not picking these lists of things to prevent from a licensing perspective for moral or ethical reasons; they’re limiting their liability. Companies are sociopathic by nature. They will do everything they can to not limit how much money they can make and do everything they can do to limit how much it will cost them.
- andy99 3y ago
  I wrote it, I agree that's probably a big reason. It's liable to be a reason why ISPs block certain traffic too like in the EFF article. I don't have enough information to know for sure which is why I said we don't know their motivations. The point still stands though, it's bad for everyone if big model providers start making their own pet lists of prohibited uses. The rest of open source manages fine without doing this.
  - orbz 3y ago
    I appreciate the thinking behind the piece then! The fact they mentioned ITAR leads me to believe this was spearheaded by corporate counsel to limit liability. Also the main difference between this and other open source is they’re still hosting the language models so have a much larger liability in usage.

1 points by andy99 3y ago

Yea. Honestly there's nothing worse than getting a "Sorry, as an AI language model, I cannot do..." This is especially awful when you're asking it to do something like "estimate the number of X in Y" or some task that it could clearly help with. I know exactly why they do this, its the safe option... but it is frustrating to deal with.

orbz 3y ago
There’s a difference between “can’t” and “won’t”. I don’t think there’s anything wrong with it being up front that an attempt to answer a question is inherently impossible for it to answer. Information like estimating X in Y may or may not be possible from the underlying training data depending on X and Y.
- robswc 3y ago
  That's fair. Usually though when I ask these questions, it can provide an answer because it gives the information it needs in the response, it just doesn't want to take ownership of the statement "I guess there were X number of Y." I can usually get a rough estimate (that definitely makes sense or checks out) but it takes a lot of back and forth. Most memorable example was getting estimates of cars sold and what make/model. It would say "there were X of Y sold in 2020, but I cannot provide an estimate" only after some back and forth, so it certainly could have told me earlier.
  - orbz 3y ago
    That’s the thing though, it can be convinced to make a statement with conviction but that might not be based on fact. I just got ChatGPT4 to say that there might have been a few hundred mint green Kia Cherokees sold in 1997.
    - robswc 3y ago
      I guess it depends on how accurate you need the info to be. I double check its answers and it provides decent estimates every time but only after getting around the "as an AI model I cannot..." wall.
raxxorraxor 3y ago
We also saw that with image synthesis. As you can imagine, if you scrap the internet for content, any model will tend to generate "controversial" images itself. An example is stable diffusion. Of course there is interest in making it commercially viable or just SFW in general. The 2.0 model version was lobotomized to not create anything "bad". As a result it was pretty useless. There was a successor version 2.1 that did it to a lesser degree, but the old 1.5 model was still more flexible. Many user prefer it even if the training data for 2.0+ had higher resolution images. There are new models were we have to see. But I think the overall tendency to lobotomize AI is pretty negative for its overall development. There should always be an option for the educated user. Sadly, unlike with LLM, it is harder to circumvent these restriction by changing your input.

orbz 3y ago

I think the author is missing a really key piece here: they’re not picking these lists of things to prevent from a licensing perspective for moral or ethical reasons; they’re limiting their liability. Companies are sociopathic by nature. They will do everything they can to not limit how much money they can make and do everything they can do to limit how much it will cost them.

andy99 3y ago
I wrote it, I agree that's probably a big reason. It's liable to be a reason why ISPs block certain traffic too like in the EFF article. I don't have enough information to know for sure which is why I said we don't know their motivations. The point still stands though, it's bad for everyone if big model providers start making their own pet lists of prohibited uses. The rest of open source manages fine without doing this.
- orbz 3y ago
  I appreciate the thinking behind the piece then! The fact they mentioned ITAR leads me to believe this was spearheaded by corporate counsel to limit liability. Also the main difference between this and other open source is they’re still hosting the language models so have a much larger liability in usage.