4 ms·

It’s unlikely to be sabotage though. What may have happened is that the teams working on RLHF may have only trained it on a dataset containing the strictest int

It’s unlikely to be sabotage though. What may have happened is that the teams working on RLHF may have only trained it on a dataset containing the strictest interpretation of liberal values without also training it on other data that grounds it in reality. As a result, the model tried to inject the liberal interpretation of values where it doesn’t make sense, such as Black Nazis, and so on.

Arnt 3y ago
It's difficult though. A lot of societies up through the ages depicted themselves unrealistically, and our depiction of past societies can be rather questionable too. Cast a glance at https://www.moviestillsdb.com/movies/ben-hur-i52618 https://www.moviestillsdb.com/movies/ben-hur-i52618 if you want. Those pictures of nazis were clearly wrong, but describing something as wrong is easier than saying what's right. Should an AI match a society's prejudiced descriptions of itself (making the nazis blond), our later images, often also prejudiced, or the current best (yet fuzzy) historical notions of what reality was like?
- candiodari 3y ago
  > describing something as wrong is easier than saying what's right This 1000x. This problem you also see with education of children. You have 2 problems: 1) there's a VERY large number of possible depictions. Let's say 10 million. Of those, a VERY small number are correct/acceptable. Let's say 10. Then it's easy to see. A correct example has 1/10 = "10%" of all information so to speak. If 100% is what you need to be guaranteed a correct classification of all possible depictions. Yet a negative example has 1/10,000,000= "0.000001%" of the information you need to classify all examples correctly. If you attempt to create correct behavior by giving wrong examples ... it'll be a long day. A positive example, telling an AI (or a child/student) what to DO has ~10000 times more information than telling them what NOT to do, in this example. In practice, you'll find the difference is even more extreme. 2) BUT there is a problem, that's also always brought up with wikipedia. Giving positive examples requires that there are examples that everyone agrees with. And we all know there are rather serious problems with that. A positive example HAS to be in the intersection of what all groups find acceptable. And ... well easy examples are always wars: Who is right? Who started the war? Is it justified? Applied to Russia, Ukraine, Israel, India, Sudan, ... but in practice a lot of issues, some not at all that controversial (which word is correct "New York" english? Bupkis or bupkez?)
- Arnt 3y ago
  Replying to myself to say it better. What should Gemini have done, what would be right? Should it match the historical record, which the Nazis twisted by choosing to photograph mostly tall blond soldiers that looked a bit Danish, or should it match fact, which then would look different from the historical record, and is difficult to establish accurately anyway? Similar questions apply for other societies, both current and ancient. Current e.g. Russia, where the census asks people to select ethnicity, but any choice except "Russian" might as well be labelled "please discriminate against me". Whatever an AI does is going to be wrong, and calling anything right is extremely difficult. Ancient examples include India and Egypt, which both had long-lived dynasties that considered themselves somehow foreign to that land and were very selective about how their represented their realms. It's really easy to find a horrible example and call it horrible, but it's low-effort thinking. Describing something that's right is so much harder and so much more valuable.
  - mananaysiempre 3y ago
    > Current e.g. Russia, where the census asks people to select ethnicity, but any choice except "Russian" might as well be labelled "please discriminate against me". I’m going to be—well, not the, but among the—last to defend the current Russian political system, but in which sense is this true? There’s a hell of a lot of discrimination in Russia, but the targeting for it is usually along the lines of not having a Russian passport or a registered address of residence in a particular region, or having the “wrong” surname or appearance. I’m not aware of anybody ever caring what you wrote in the census papers—and the data from the latest census is so bogus basically everybody who does population statistics has disregarded it—but maybe it happens in smaller towns? That would be interesting. The only place I’m aware of that actually associates your declared (not implied) ethnicity with your name is the ostensibly statistical slip you fill in when changing your registered address, but at least fifteen years ago quoting the relevant parts of the constitution resulted in surprise followed by begrudging acceptance. (Art. 26: Everyone can determine and specify their ethnicity. Noone shall be forced to determine or specify their ethnicity.—a direct attempt to preclude the “item 5” shenanigans common in Soviet times.) Again, none of this is to say that there isn’t a lot of discrimination happening. (As one example, the disproportionately large military draft in some areas could arguably amount to ethnic cleansing, were it perpetrated with intent instead of arising through general administrative laziness seeking a softer target. Either way, some groups and possibly even languages are not going to survive.) I’m just very surprised to hear about the census, specifically, being important in that respect.

Arnt 3y ago

It's difficult though. A lot of societies up through the ages depicted themselves unrealistically, and our depiction of past societies can be rather questionable too. Cast a glance at https://www.moviestillsdb.com/movies/ben-hur-i52618 https://www.moviestillsdb.com/movies/ben-hur-i52618 if you want. Those pictures of nazis were clearly wrong, but describing something as wrong is easier than saying what's right. Should an AI match a society's prejudiced descriptions of itself (making the nazis blond), our later images, often also prejudiced, or the current best (yet fuzzy) historical notions of what reality was like?

candiodari 3y ago
> describing something as wrong is easier than saying what's right This 1000x. This problem you also see with education of children. You have 2 problems: 1) there's a VERY large number of possible depictions. Let's say 10 million. Of those, a VERY small number are correct/acceptable. Let's say 10. Then it's easy to see. A correct example has 1/10 = "10%" of all information so to speak. If 100% is what you need to be guaranteed a correct classification of all possible depictions. Yet a negative example has 1/10,000,000= "0.000001%" of the information you need to classify all examples correctly. If you attempt to create correct behavior by giving wrong examples ... it'll be a long day. A positive example, telling an AI (or a child/student) what to DO has ~10000 times more information than telling them what NOT to do, in this example. In practice, you'll find the difference is even more extreme. 2) BUT there is a problem, that's also always brought up with wikipedia. Giving positive examples requires that there are examples that everyone agrees with. And we all know there are rather serious problems with that. A positive example HAS to be in the intersection of what all groups find acceptable. And ... well easy examples are always wars: Who is right? Who started the war? Is it justified? Applied to Russia, Ukraine, Israel, India, Sudan, ... but in practice a lot of issues, some not at all that controversial (which word is correct "New York" english? Bupkis or bupkez?)
Arnt 3y ago
Replying to myself to say it better. What should Gemini have done, what would be right? Should it match the historical record, which the Nazis twisted by choosing to photograph mostly tall blond soldiers that looked a bit Danish, or should it match fact, which then would look different from the historical record, and is difficult to establish accurately anyway? Similar questions apply for other societies, both current and ancient. Current e.g. Russia, where the census asks people to select ethnicity, but any choice except "Russian" might as well be labelled "please discriminate against me". Whatever an AI does is going to be wrong, and calling anything right is extremely difficult. Ancient examples include India and Egypt, which both had long-lived dynasties that considered themselves somehow foreign to that land and were very selective about how their represented their realms. It's really easy to find a horrible example and call it horrible, but it's low-effort thinking. Describing something that's right is so much harder and so much more valuable.
- mananaysiempre 3y ago
  > Current e.g. Russia, where the census asks people to select ethnicity, but any choice except "Russian" might as well be labelled "please discriminate against me". I’m going to be—well, not the, but among the—last to defend the current Russian political system, but in which sense is this true? There’s a hell of a lot of discrimination in Russia, but the targeting for it is usually along the lines of not having a Russian passport or a registered address of residence in a particular region, or having the “wrong” surname or appearance. I’m not aware of anybody ever caring what you wrote in the census papers—and the data from the latest census is so bogus basically everybody who does population statistics has disregarded it—but maybe it happens in smaller towns? That would be interesting. The only place I’m aware of that actually associates your declared (not implied) ethnicity with your name is the ostensibly statistical slip you fill in when changing your registered address, but at least fifteen years ago quoting the relevant parts of the constitution resulted in surprise followed by begrudging acceptance. (Art. 26: Everyone can determine and specify their ethnicity. Noone shall be forced to determine or specify their ethnicity.—a direct attempt to preclude the “item 5” shenanigans common in Soviet times.) Again, none of this is to say that there isn’t a lot of discrimination happening. (As one example, the disproportionately large military draft in some areas could arguably amount to ethnic cleansing, were it perpetrated with intent instead of arising through general administrative laziness seeking a softer target. Either way, some groups and possibly even languages are not going to survive.) I’m just very surprised to hear about the census, specifically, being important in that respect.