4 ms·
Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the out
by DebtDeflation 3y ago
Surely there is a middle ground.
"Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output.
"Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.
- ceejayoz 3y agoYou can see how this gets challenging, though, right? If you train your model to prioritize real photos (as they're often more accurate representations than artistic ones), you might wind up with Denzel Washington as the archetype; https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_film) https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_f.... There's a vast gap between human understanding and what LLMs "understand".
- transitionnel 3y agoIf they actually want it to work as intelligently as possible, they'll begin taking these complaints into consideration and building in a wisdom curating feature where people can contribute. This much is obvious, but they seem to be satisfied with theory over practicality. Anyway I'm just ranting b/c they haven't paid me. How about an off the wall algorithm to estimate how much each scraped input turns out to influence the bigger picture, as a way to work towards satisfying the copyright question.
- sho_hn 3y agoAn LLM-style system designed to understand Wikipedia relevance and citation criteria and apply them might be a start. Not that Wikipedia is perfect and controversy-free, but it's certainly a more sophisticated approach than the current system prompts.
- photoGrant 3y agoThen who in this black box private company is the Oracle of infinite wisdom and truth!? Who are you putting in charge? Can I get a vote?
- Affric 3y agoI mean now you d to train AI to recognise the bias in the training data.
- gedy 3y ago> If you train your model to prioritize real photos I thought that was the big bugbear about disinformation and false news, but now we have to censor reality to combat "bias"
- AlecSchueler 3y agoAs soon as you have then playing an anachronistic sport you should expect other anachronistic imagery to creep in, to be fair.
- mp05 3y ago> anachronistic sport Scottish kings absolutely played golf.
- ceejayoz 3y agohttps://en.wikipedia.org/wiki/Golf https://en.wikipedia.org/wiki/Golf > The modern game of golf originated in 15th century Scotland.
- AlecSchueler 3y agoOh fair enough then.
- mp05 3y ago> "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. I'd err on the side of "not unexpected". A group of friends in a park in Tokyo is probably not very diverse, but it's not outside of the realm of possibility. Only white men were golfing Scottish kings if we're talking strictly about reality and reflecting it properly.
- ankit219 3y agoThe focus for alignment is to avoid bad PR specifically the kind of headlines written by major media houses like NYT, WSJ, WaPo. You could imagine the headlines like "Google's AI produced a non-diverse output on occasions" when a researcher/journalist is trying too hard to get the model to produce that. The hit on Google is far bigger than say on Midjourney or even Open AI till now (I suspect future models will be more nerfed than what they are now) For the cases you mentioned, initially those were the examples. It gets tricky during red teaming where they internally try out extreme prompts and then align the model for any kind of prompt which has a suspect output. You train the model first, then figure out the issues, and align the model using "correct" examples to fix those issues. They either went to extreme levels doing that or did not test it on initial correct prompts post alignment.
- sho_hn 3y agoQuite reminded of this episode: https://www.eurogamer.net/kingdom-come-deliverance-review https://www.eurogamer.net/kingdom-come-deliverance-review (black representation in a video game about 15th century Bohemia; it was quite the controversy)
- cabalamat 3y ago> "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys. It works in bing, at least: https://www.bing.com/images/create/a-picture-of-some-17th-century-scottish-kings-play/1-65d66b5295c34bb6acef133f14e0513c?id=orVDZ24aqDey6x41qOwStQ%3d%3d&view=detailv2&idpp=genimg&thId=OIG3.MCFECwUZ7HToW553wSdW&FORM=GCRIDP&mode=overlay https://www.bing.com/images/create/a-picture-of-some-17th-ce...
- Nevermark 3y agoI don't know that this sheds light on anything but I was curious... a picture of some 21st century scottish kings playing golf (all white) https://www.bing.com/images/create/a-picture-of-some-21st-century-scottish-kings-play/1-65d66d8570e1419bbf58f8f4bf103ab6?FORM=GENCRE https://www.bing.com/images/create/a-picture-of-some-21st-ce... a picture of some 22nd century scottish kings playing golf (all white) https://www.bing.com/images/create/a-picture-of-some-22nd-century-scottish-kings-play/1-65d66f71a35941aeae8b7a4623f47d1c?id=OMDPXgISVgFx1R5YNtl5Fg%3d%3d&view=detailv2&idpp=genimg&thId=OIG3.PLevSmzkDQJm3x6f78Ag&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/a-picture-of-some-22nd-ce... a picture of some 23rd century scottish kings playing golf (all white) https://www.bing.com/images/create/a-picture-of-some-23rd-century-scottish-kings-play/1-65d66f971e704bc2a89ce001d78d288e?id=Bq5HUPPfBpBdfZsqQvKgEA%3d%3d&view=detailv2&idpp=genimg&thId=OIG3.gPDhg.28lapIdtckd_IS&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/a-picture-of-some-23rd-ce... a picture of some contemporary scottish people playing golf (all white men and women) https://www.bing.com/images/create/a-picture-of-some-contemporary-scottish-people-pla/1-65d670acb5b642cda27438edd086f50f?id=KJN8YrhkyTlijWqNwbXEtQ%3d%3d&view=detailv2&idpp=genimg&thId=OIG2.ZC91SxoryqPNErLLpb9J&FORM=GCRIDP https://www.bing.com/images/create/a-picture-of-some-contemp... https://www.bing.com/images/create/a-picture-of-some-contemporary-scottish-people-pla/1-65d670acb5b642cda27438edd086f50f?id=0thTVpYwadvAtiCCSW1uTw%3d%3d&view=detailv2&idpp=genimg&thId=OIG2.cpL14K7XLR3251jnnAGN&FORM=GCRIDP https://www.bing.com/images/create/a-picture-of-some-contemp... a picture of futuristic scottish people playing golf in the future (all white men and women, with the emergence of the first diversity in Scotland in millennia! Male and female post-human golfers. Hummmpph!) https://www.bing.com/images/create/a-picture-of-futuristic-scottish-people-playing-go/1-65d6711c7ef34ac696103d03e9cfc792?id=MSABypiUUY26scg429Es2w%3d%3d&view=detailv2&idpp=genimg&thId=OIG1.TK.8dAbln_jBJfHb6xws&FORM=GCRIDP https://www.bing.com/images/create/a-picture-of-futuristic-s... https://www.bing.com/images/create/a-picture-of-futuristic-scottish-people-playing-go/1-65d6711c7ef34ac696103d03e9cfc792?id=pvBLXIX6cVpIPbgqnCiDkw%3d%3d&view=detailv2&idpp=genimg&thId=OIG1.lULvcyGNHjgnE2CFjLQ_&FORM=GCRIDP https://www.bing.com/images/create/a-picture-of-futuristic-s... Inductive learning is inherently a bias/perspective absorbing algorithm. But tuning in a default bias towards diversity for contemporary, futuristic and time agnostic settings seems like a sensible thing to do. People can explicitly override the sensible defaults as necessary, i.e. for nazi zombie android apocalypses, or the royalty of a future Earth run by Chinese overlords (Chung Kuo), etc.
- trhway 3y ago>"Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys. is black man in the role of the Scottish king represents a bigger error than some other errors in such an image, like say incorrect dress details or the landscape having say a wrong hill? I'd venture a guess that only our racially charged mentality of today considers that a big error, and may be in a generation or 2 an incorrect landscape or dress detail would be considered much larger error than a mismatched race.
- swatcoder 3y agoThere's no "middle" in the field of decompressing a short phrase into a visual scene (or program or book or whatever). There are countless private, implicit assumptions that users take for granted yet expect to see in the output, and vendors currently fear that their brand will be on the hook for the AI making a bad bet about those assumptions. So for your first example, you totally expect racial and gender diversity in the output because you're assuming a realistic, contemporary, cosmopolitan, bourgeoisie setting -- either because you live in one or because you anticipate that the provider will default to one. The food will probably look Western, the friends will probably be young adults that look to have professional or service jobs wearing generic contemporary commercial fashion, the flora in in the park will be broadly northern climate, etc. Most people around the world don't live in an environment anything like that, so nominal accuracy can't be what you're looking for. What you want, but don't say, is a scene that feels familiar to you and matches what you see as the de facto cultural ideal of contemporary Western society. And conveniently, because a lot of the training data is already biased towards that society and the AI vendors know that the people who live in that society will be their most loyal customers and most dangerous critics right now, it's natural for them to put a thumb on the scale (through training, hidden prompts, etc) that gets the model to assume an innocuous Western-media-palatable middle ground -- so it delivers the racially and gender diverse middle class picnic in a generic US city park. But then in your second example, you're implicitly asking for something historically accurate without actually saying that accuracy is what's become important for you in this new prompt. So the same thumb that biased your first prompt towards a globally-rare-but-customer-palatable contemporary, cosmopolitan, Western culture suddenly makes your new prompt produce something surreal and absurd. There's no "middle" there because the problem is really in the unstated assumptions that we all carry into how we use these tools. It's more effective for them to make the default output Western-media-palatable and historical or cultural accuracy the exception that needs more explicit prompting. If they're lucky, they may keep grinding on new training techniques and prompts that get more assumptions "right" by the people that matter to their success while still being inoffensive, but it's no simple "surely a middle ground" problem.
- crooked-v 3y agoIt feels like it wouldn't even be that hard to incorporate into LLM instructions (aside from using up tokens), by way of a flowchart like "if no specific historical or societal context is given for the instructions, assume idealized situation X; otherwise, use historical or projected demographic data to do Y, and include a brief explanatory note of demographics if the result would be unexpected for the user". (That last part for situations with genuine but unexpected diversity; for example, historical cowboys tending much more towards non-white people than pop culture would have one believe.) Of course, now that I've said "it seems obvious" I'm wondering what unexpected technical hurdles there are here that I haven't thought of.
- photonthug 3y ago> Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. Do we expect this because diverse groups are realistically most common or because we wish that they were? For example only some 10% of marriages are interracial, but commercials on TV would lead you to believe it’s 30% or higher. The goal for commercials of course is to appeal to a wide audience without alienating anyone, not to reflect real world stats. What’s the goal for an image generator or a search engine? Depends who is using it and for what, so you can’t ever make everyone happy with one system unless you expose lots of control surface toggles. Those toggles could help users “own” output more, but generally companies wouldn’t want to expose them because it could shed light on proprietary backends, or just take away the magic from interacting with the electric oracles.
- paulddraper 3y agoAnd most women are friends with mostly women, and most men are friends with mostly men.
- jjjjj55555 3y agoAlso, these tools are used world-wide and "diversity" means different things in different places. Somehow it's always only the US ideal of diversity that gets shipped abroad.
- photonthug 3y agoYeah, someone else mentioned Tokyo which is not going to have as much variety among park visitors as NYC. But then again neither will Colorado (or almost anywhere else!) be as diverse. Some genius at corporate is probably scheming about making image generation as location-sensitive as search is, ostensibly to provide utility but really to perpetuate echo chambers and search bubbles. I wish computing in general would move back towards user-controlled rather than guess-what-I-mean and the resulting politicization, but it seems that ship has sailed.
- felipeerias 3y agoUS companies systematically push US cultural beliefs and expectations. People in the US probably don’t notice it any more, but it’s pretty obvious from those of us on the receiving end of US cultural domination. This fact is an unavoidable consequence of the socioeconomic realities of the world, but it obviously clashes with these companies’ public statements and positions.
- samatman 3y agoWhy would you expect anything you didn't specify in the output of the first prompt? If there are friends, lunch, and a park: it did what you asked. Piling a bunch of neurotic expectations about it being a Benneton ad on top of that is absurd. When you can trivially add as much content to the description as you want, and get what you ask for, it does not matter what the default happens to be.