4 ms·

> For example, Google’s Gemini had secret meta prompts that biased it towards certain types of answers and also caused it to produce hallucinated images that we

> For example, Google’s Gemini had secret meta prompts that biased it towards certain types of answers and also caused it to produce hallucinated images that were funny but also dystopian (https://arstechnica.com/information-technology/2024/02/googl https://arstechnica.com/information-technology/2024/02/googl...). Such a bizarre take to call this "dystopian". The model happened to create some out-there pictures. I mean, it's no more outlandish then giant dragons and snakes and such being created yet the thought of a person of color being something historically inaccurate is this massive outcry against revisionism? Who cares? Besides, the article identifies the probable goal which was to eliminate very known biases in existing models (i.e. when generating "angry person" you mainly got black people). Clearly this one wasnt tuned well for that goal, but the objective is not only noble but absolutely should be required for anyone producing LLM models.

blackeyeblitzar 2y ago
If I may explain: the dystopian part to me is the lack of transparency around training code, training data sources, tuning, meta prompting, and so forth. In Google’s case, they’re a large corporation that controls how much of society accesses information. If they’re secretly curating what that information is, rather than presenting it as neutrally as they can, it does feel dystopian to me. I’d like transparency as a consumer of information, so I know to the extent possible, what the sources of information were or how I am being manipulated by choices the humans building these systems made. I appreciate the issue you’re drawing attention to in the example you shared about images of an angry person. I think I agree that focused tuning for situations like that might be noble and I would be okay with a model correcting for that specific example you shared. But I also struggle with how to clearly draw that line where such tuning may go too far, which is why I favor less manual biasing. But I disagree that such tuning should be required, if you meant required by the law. Like with speech or art in general, I think anyone should be able to produce software systems that generate controversial or offensive speech or art. Individual consumers can choose what they want to interact with, and reject LLMs that don’t meet their personal standards.
lynx23 2y ago
Right, "who cares" about the truth in our dystopian world? 1984 is apparently too long ago for people to remember the ministry of truth...
- Daddy_cat 2y ago
  [dead]

blackeyeblitzar 2y ago

If I may explain: the dystopian part to me is the lack of transparency around training code, training data sources, tuning, meta prompting, and so forth. In Google’s case, they’re a large corporation that controls how much of society accesses information. If they’re secretly curating what that information is, rather than presenting it as neutrally as they can, it does feel dystopian to me. I’d like transparency as a consumer of information, so I know to the extent possible, what the sources of information were or how I am being manipulated by choices the humans building these systems made. I appreciate the issue you’re drawing attention to in the example you shared about images of an angry person. I think I agree that focused tuning for situations like that might be noble and I would be okay with a model correcting for that specific example you shared. But I also struggle with how to clearly draw that line where such tuning may go too far, which is why I favor less manual biasing. But I disagree that such tuning should be required, if you meant required by the law. Like with speech or art in general, I think anyone should be able to produce software systems that generate controversial or offensive speech or art. Individual consumers can choose what they want to interact with, and reject LLMs that don’t meet their personal standards.

lynx23 2y ago

Right, "who cares" about the truth in our dystopian world? 1984 is apparently too long ago for people to remember the ministry of truth...

Daddy_cat 2y ago
[dead]