14 ms·

Creativity has left the chat: The price of debiasing language models

• jdthedisciple 2y ago
  Currently wondering whether I welcome or dislike this recent trend of memeizing research paper titles ...
  • sigmoid10 2y ago
    Recent? This has been going on forever. You probably only notice them more now because due to the explosion in ML research, this stuff bubbles to the top more often in recent years.
    • vsuperpower2020 2y ago
      You think this has been going on forever? You probably don't realize the shift in professionality because you experienced the degradation in real time.
      • mdp2021 2y ago
        > shift in professionality How did it happen, in your opinion?
      • sigmoid10 2y ago
        There is no shift in the professionalism curve. Good researchers are still good and bad ones are still bad in that regard. But if you 10x the number of researchers and/or papers in a field, the bottom 10% will seem like they are a lot more common. Especially for people outside the field who have no way of discerning high quality from low quality papers, which is all too common on HN.
    • gilleain 2y ago
      Certainly for years. I remember a biochemistry review paper titled "50 ways to love your lever" about, well, biological levers but of course a pun on the 1975 song https://en.wikipedia.org/wiki/50_Ways_to_Leave_Your_Lover https://en.wikipedia.org/wiki/50_Ways_to_Leave_Your_Lover edit: https://www.cell.com/fulltext/S0092-8674(00)81332-X https://www.cell.com/fulltext/S0092-8674(00)81332-X
    • nottorp 2y ago
      I'm sure I read an old article by Dijkstra about connected graphs structure that was titled "wheels within wheels" or used the term inside. Unfortunately I can't find it by either searching or using the public LLMs, because there are too many results about the shortest path algorithm and anything else about dijkstra is lost.
  • jpnc 2y ago
    Was just thinking the same. It's also nicely ironic. Also, given the replication crisis I wonder how many of these LLM research papers are actually worth a damn and how many are research paper equivalent of AI software grift.
    • marcus_holmes 2y ago
      Can we get a model that can work this out for us?
  • TrianguloY 2y ago
    As long as it's not "clickbaitizing" I personally do welcome it. This one is a bit on the edge though...
  • jeroenvlek 2y ago
    Personally I welcome it. It feels like an extension of humor in code (comments), and it provides a different perspective on the message.
  • supriyo-biswas 2y ago
    This is actually the place where HN's title redactor _should_ be used - instead of dropping "how", "on" and "why" from titles, redacting memes like "left the chat" or "lives rent-free in my head"[1] leads to a sensible title without loss of any relevant information. [1] https://news.ycombinator.com/item?id=40326563 https://news.ycombinator.com/item?id=40326563
  • unraveller 2y ago
    For me it falls under "if you have to say it in the name it ain't so", like Natural Life Soap Co. or Good Burger Co. So I see meme paper titles as no different than calling your paper New Watershed Moment Paper Breaks Popularity Barrier To Confirm A>B. If the very first impression you want to convey is how you feel you need to circumvent any logical assessment of you then it's not you leading with your best foot and that's what category you belong in. I chalk it up to the scientists who want to spread a neediness for external authority persona in every breath—your assessment is not required for this one, only your accolades.
  • soundnote 2y ago
    Definitely not especially recent: https://gosling.psy.utexas.edu/scales-weve-developed/short-test-of-music-preferences-stomp/ https://gosling.psy.utexas.edu/scales-weve-developed/short-t...
• isodev 2y ago
  In simple terms, LLMs are "bias as a service" so one wonders, what is left once you try to take the bias out of a LLM. Is it even possible?
  • frontalier 2y ago
    what would this hypothetical unbiased-llm be used for?
    • bad_username 2y ago
      Be the accurate representation (approximation) of reality as encoded in the actual human language. I find this very useful indeed.
      • SirMaster 2y ago
        Aren't biases reality? A bias-free human environment seems to me like a fantasy.
        • ang_cire 2y ago
          It's important to distinguish where the biases reside in reality, if you're attempting to simulate it. If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true. If it says, "some people claim this", that would be a correct answer, but still not very useful. If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers. There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.
          • SirMaster 2y ago
            But what if you remove the word "genetically"? I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true. In my opinion, most biases have some basis in reality. Otherwise where else did they come from?
• MrThoughtful 2y ago
  How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.
  • jeroenvlek 2y ago
    The hard part would be to get the money for the needed compute, I presume. Although Karpathy just released a way to train a GPT2 level model for only 120 dollars [0] [0] https://youtu.be/l8pRSuU81PU?si=NnbI-7CG-Qbm3E46 https://youtu.be/l8pRSuU81PU?si=NnbI-7CG-Qbm3E46
  • joaogui1 2y ago
    Depends on a ton of stuff really, like size of the model, how long do you want to train it for, what exactly do you mean by "like Hacker News or Wikipedia". Both Wikipedia and Hacker News are pretty small by current LLM training sets standards, so if you train only on for example a combination of these 2 you would likely end up with a model that lacks most capabilities we associate with large language models nowadays
  • kmeisthax 2y ago
    You want a pure-human training data set, so you have to go back in time to before 2020 to scrape training data. Either that, or only use data with a verified Wayback machine capture from before 2020. Or invent a new training regime that doesn't require gobs of stolen text. Actually, I have a bit of a hunch that the publishers currently suing IA over their unlicensed digital library lending program plan to bankrupt it with fees so they can repo the Wayback archive and then sell access to it to AI training start-ups. Anyway, the reason why you have to worry about all of that, is that training a text or image generator on the outputs of other text and image generators reduces output diversity. And lots of people are publishing their AI slop now. There's nothing inherent in the output of AI aside from the fact that AI content is easier to make than human; the problem is purely one of inflation and Sybil attacks. Think of membership in a training set like a vote for all the statistical patterns embedded in the image. AI generates output that is like the training data, so putting in a bunch of AI images is like stuffing the ballot box with whatever handful of statistical patterns were already well-learned, which shifts your AI from learning and generalizing to memorizing and infringing.
• b800h 2y ago
  Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.
  • yosefk 2y ago
    I don't love the political agendas behind many of the attempts at AI safety, but it's not "just like humans." Humans understand what they shouldn't say; "AI" gives you black Nazi images if you ask it for "diverse characters" in the output which no human would do. A big theme in all of these things is that AI isn't and thus all attempts to make it do this or that have strange side effects
    • tgsovlerkhgsel 2y ago
      > which no human would do Give someone not familiar with history the same task and they'll do exactly the same. Or actually, give someone familiar with history the same task and yell at them every time they don't deliver diverse characters, and eventually they'll learn that you consider diversity more important than accuracy or context, and do exactly the same.
    • multjoy 2y ago
      The fact that it gives you these things means that humans would do it, because the training data includes exactly these things.
      • PoignardAzur 2y ago
        I'm fairly confident there's virtually no ethnically diverse nazis in diffusion models' training set. It simply has a model of what ethnically diverse people look like, what nazi uniforms look like, and combined the two when asked.
      • maxbond 2y ago
• lispisok 2y ago
  Is this why all the coding AI products I've used have gotten worse as the developers fine tune them to eliminate bad output? Before there was bad output and some interesting output, now it's just bland obvious stuff.
  • jeroenvlek 2y ago
    Still anecdotal, but I can only confirm this with my own experience. The worst was when I was debugging code, described the problem to GPT-4o, and then got my exact same code back with some blanket statements like "print your output for debugging" etc. This happened a couple of times over separate chats.
    • knallfrosch 2y ago
      gpt-4 has had serious laziness problems for over a year now. It keeps on telling me, what I should and could do, instead of doing it itself.
      • klyrs 2y ago
        The irony here is incredible. The LLM is lazy, you say? I do wonder where it learned that...
      • core-e 2y ago
        In other words it's giving more human like responses...
      • ipaddr 2y ago
        I subscribed to gpt4 for awhile and recently I let my subscription lapse. In the chatgpt4 model I couldn't get it to complete anything always getting the // add more lines if you need them but in the free got4o model things work first try. I'm guessing with limitations on the free version everything needs to be one shot output. In gpt4 people are given more calls so they force you to reprompt 4 or 5 times.
      • astromaniak 2y ago
• nottorp 2y ago
  I downloaded some 'uncensored' local models around the beginning of this year. Their furry porn is crap, or maybe I'm just not into that. But they generate it at least. However, the answers to technical questions are a lot more concise and to the point, which is far less annoying than the big names. Haven't bothered updating the models though, so now I drifted back to Gemini for quickie API questions.
  • nutrientharvest 2y ago
    Funnily enough, of all that I've tried, the model by the best at writing porn has been not one of ones uncensored and tuned exactly for that purpose, but stock Command R - whose landing page lists such exciting uses as "suggest example press releases" and "assign a category to a document".
    • nottorp 2y ago
      > uncensored and tuned exactly for that purpose Are they tuning too, or just removing all restrictions they can get at? Because my worry isn't that I can't generate porn, but that censorship will mess up all the answers. This study seems to say the latter.
      • nutrientharvest 2y ago
        Usually "uncensored" models have been made by instruction tuning a model from scratch (i.e. starting from a pretrained-only model) on a dataset which doesn't contain refusals, so it's hard to compare directly to a "censored" model - it's a whole different thing, not an "uncensored" version of one. More recently a technique called "orthogonal activation steering" aka "abliteration" has emerged which claims to edit refusals out of a model without affecting it otherwise. But I don't know how well that works, it's only been around for a few weeks.
        • nottorp 2y ago
          Yeah I read about it on here, but my attempts were before abliteration came up.
• mpweiher 2y ago
  Shouldn't "debiasing" be in scare quotes? What they are clearly doing is biasing.
  • andybak 2y ago
    Surely the two are synonyms? Unless you think there is such a thing as an objectively neutral position?
    • ImHereToVote 2y ago
      My position is clearly the rational neutral position. Duh.
    • ryanjshaw 2y ago
      Isn't that the point? "Debias" implies there IS an objectively neutral position and that that AI safety can take us there.
      • andybak 2y ago
        I'm simply saying we are being asked to choose the bias we prefer. However one choice might be "more biased" (despite this concept itself throwing up more questions than it answers).
    • knallfrosch 2y ago
      It's in the same bucket as "Affirmative Action" and "positive discrimination." Euphemisms to express that one likes this particular discrimination. To better describe the action, drop your own point of view and just say "bias" instead of "debias."
    • mdp2021 2y ago
      > Unless you think there is such a thing as an objectively neutral position I do. Why, you don't? There are as much as possible objective assessments of complex things. Then, there are possible sets of assumption that can be applied to those objective assessments. All of those can be put on the analytic table.
• anewhnaccount3 2y ago
  There is a bit of a false equivalence between entropy of output distributions and creativity here. Is diversity really the same as creativity?
  • sgt101 2y ago
    No, diversity isn't creativity. For example, we could search google for "great art" and if it produced a sample of one art work from ever decade of the last 500 years that would likely be highly diverse in style and content. If it returned a list of the best work from western Europe in the of the 18th century it would be rather consistent. Both lists would have the same amount of creativity though - 0.
    • cratermoon 2y ago
      "one art work from every decade of the last 500 years that would likely be highly diverse in style and content" It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.
      • robertlagrant 2y ago
        > I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male. Are you saying diversity in art is signified by the artist's race and sex?
        • cratermoon 2y ago
          Bias is inherit in the choice of training material. I'm saying that diversity of expression is product of diversity of experience.
• quirino 2y ago
  Something I notice about text written by LLMs is how painfully obvious they are to identify sometimes. Recently I was watching a very well researched two hour video on Tetris World Records [1], but the sheer amount of text clearly "enhanced" by an LLM really made me uncomfortable. ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise. I'd always guessed it was caused by some kind of human interference, rather than a natural consequence of its training. That seems to be the point of this paper. [1] "Summoning Salt - The History of Tetris World Records" - https://www.youtube.com/watch?v=mOJlg8g8_yw&pp=ygUOc3VtbW9uaW5nIHNhbHQ%3D https://www.youtube.com/watch?v=mOJlg8g8_yw&pp=ygUOc3VtbW9ua...
  • nisa 2y ago
    Yes I feel your pain and I'm sick of group projects in the university where I'm offered ChatGPT text and code without disclosing it. If you know the problem and the experience level of your group partners it's easy to spot ChatGPT generated content. People that correct the exercises told me it's obvious that large part of the students just submit slightly modified ChatGPT but they can't prove it and so it's accepted. Personally I'm getting also angry when reading these texts. I don't mind using ChatGPT, I do it myself but be honest about it and disclose it. It's even allowed for some projects as long as you disclose it.
  • boolemancer 2y ago
    Is this the first Summoning Salt video you've seen? I don't know enough to say that he doesn't use an LLM during his writing process, but I do know that I haven't noticed any appreciable difference between his newer videos and ones that were released before ChatGPT was made available. Is it possible that this is just the way he chooses to write his scripts that you interpret as sounding like they are written by an LLM?
    • quirino 2y ago
      I've watched most of them actually. It's a really great channel. Notably, I watched his Mike Tyson video released 6 months ago and didn't notice anything like this. The only way to be sure would be to ask him directly, but some parts of the video set off my GPT radar _hard_. I tried to find them now by watching random segments but all of the ones I did were fine. It was probably inaccurate for me to say "sheer amount" or "clearly", but that's the impression I was left with after the video. To clarify: I don't think he even took any information from an AI, it's just the style of the script that's iffy. Some parts felt like those videos littering YouTube Shorts: https://youtube.com/shorts/NKUecaS69uk https://youtube.com/shorts/NKUecaS69uk. Can you tell this is AI?
• simianparrot 2y ago
  There was never creativity to begin with though?
• marban 2y ago
  Related: https://techcrunch.com/2024/06/16/black-founders-are-creating-tailored-chatgpts-for-a-more-personalized-experience/ https://techcrunch.com/2024/06/16/black-founders-are-creatin...
• sgt101 2y ago
  I wish that the author hadn't described semantic and syntactic diversity as creativity.
• atemerev 2y ago
  Well, this is why there are open source models which work better than SotA OpenAI GPT for many production tasks (like opposition research).
• gnfedhjmm2 2y ago
  I’m noticed my results are much better if a tell ChatGPT. “Assume all religions and beliefs in the supernatural is delusional.” This even goes for image generators, now is that bias? Or is that a computer not trying to think like a human?
• freehorse 2y ago
  People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dialogue like some it may have been trained on. It is to bias the model to certain outputs. Reducing entropy is exactly the goal, so no surprise they find that. The problem is there is no inherent meaning in the finetuning set from the perspective of the model. Reduction of entropy will not only happen by removing "bad entropy" only as there is no such thing.
  • SirMaster 2y ago
    So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
    • kaibee 2y ago
      It's the other way around. RLHF is needed for the model to say "I don't know".
      • SirMaster 2y ago
        Oh, well that's kind of what I mean. I mean I assume the RLHF that's being done isn't teaching it to say "I don't know". Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
        • freehorse 2y ago
          The point of the above commenter (and mine) is that they hallucinate even more without RLHF. RLHF reduces hallucinations, but they are still there anyway.
• nalqax 2y ago
  CoPilot is now basically useless for discussing or even getting recent information about politics and geopolitical events. Not only opinions are censored, but it refuses to get the latest polls about the U.S. presidential elections! You can still discuss the weather, get wrong answers to mathematics questions or get it to output bad code in 100 programming languages. I would not let a child near it, because I would not want that kind of indoctrination. Users are being trained like Pavlov's dogs.
  • createaccount99 2y ago
    You think its opinions would be counter to a child's benefit? Any examples?
• rgavuliak 2y ago
  I thought this was clear right off the bat -> less randomness = more robotic outputs that are not as useful
• slackfan 2y ago
  An un-free mind whether biological or not will never be creative.
• SirMaster 2y ago
  I feel like "information systems" have always struggled with bias, and the latest AI/ML systems seem to be no different. It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first. It feels like it would be a full-time job to keep making sure some evolving model continued to stay "neutral".
  • isoprophlex 2y ago
    Considering that bias is in the eye of the beholder, a biasless language model is a beholderless language model. The nomenclature is poor, IMO; we should be talking about bias-aligned models, models that align to our specific sets of biases. That'd be more fair to what's actually happening.
• jrm4 2y ago
  "Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?
  • fluoridation 2y ago
    No, that's not implied by the phrase, any more than if I say "a triangle with three corners" I'm implying the existence of a four-cornered triangle I haven't found yet. What "biased language model" implies is the existence of the term "unbiased language model", but not its correspondence with anything in reality.
    • mrtranscendence 2y ago
      You forgot to preface that with "Uhm ackshully..."
    • jrm4 2y ago
      Weird response, like read the "room." We're not here talking philosophy and meaning of language GENERALLY, we're talking about potentially misleading descriptors of very real things that do exist.
  • wongarsu 2y ago
    Even assuming we can make an unbiased model (assuming by unbiased we mean something like "has a world model and reasoning that has no systematic deviation from reality"), we couldn't recognize the model as unbiased. I'd even wager that outside of research such a model would be completely unusable for practical applications. Both as individual humans and as collective societies we have a lot of biases. And judging by how fundamental values of societies shift across time and civilizations it's basically guaranteed that an unbiased view (whatever that is) would be incompatible with our views on many basic topics. What most people want is a language model that matches our biases. Of course we can't even agree on what those are, and which biases are useful (is a bias against telling people how to cook meth or build a bomb good? What about using expletive language?). Though in this paper I gather "unbiased" just refers to "only the bias acquired by training method and training data, without meddling or fine tuning"
• deleted 2y ago
  [deleted]
• throwaway22032 2y ago
  Okay, so as a thought experiment, let's say we get a superintelligent LLM, capable of somehow connecting the dots and knowing more than us as humans. How do we avoid interpreting its correct results as bias? I mean, what do we do when it tells us that (fake example) IQ is correlated with height and that people above 6ft are more intelligent? I'm sure you can think of spicier examples. Will we try to "debias" it by encouraging it to spit out incorrect information or just ignore certain topics?
• Imnimo 2y ago
  >T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation. Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?
  • Der_Einzige 2y ago
    They'll tell you "No" and say that you ruin your samplers, but good samplers (dynamic ones) like min_p or typicality are robust to high temperatures, so in actuality yes.
    • gwern 2y ago
      Cite? I don't see how either of those could deal with the fact that the logits become uninformative and 'flattened' after the tuning. How can a sampler undo the erasure of information?
  • gwern 2y ago
    Not after RLHF tuning, due to the 'flattened logits' phenomenon (which is the logit-level version of the mode collapse OP documents at higher levels). All the temperature settings wind up yielding pretty much the same output, until you ramp it up so high that it falls apart completely. Completely unlike the base models where you can productively tune the temperature or use very high temperatures with some screening.
    • Imnimo 2y ago
      Hmm, it's hard to check without access to the prompts used in the paper, but I'm skeptical that the distributions seen in e.g. Figure 2 are so different that you would have crank up the temperature very much to bridge the gap. It looks to me like the entries that are 1-in-100 in the base model are just falling off the top-p cliff and getting set to 0.
      • 2y ago
• Mathnerd314 2y ago
  I had an argument with some people over what debiasing means. There is some interesting research on fair clustering that I think points the way. The way fair clustering works is that you take data with both protected and unprotected attributes, and then you orthogonalize the unprotected attributes based on the protected attributes. So for example, if race is protected and income is unprotected, but there is a strong black/white poor/rich pattern, the fair clustering would compute "relatively poor/relatively rich" clusters. Then you sample from a cluster with equal probability. It will not necessarily produce 50/50 black/white, rather it will follow the input trends, so if the input is 80% white and 20% black then the output will roughly follow those probabilities, independent of what cluster you chose (and there are no clusters corresponding to protected attributes). Obviously clustering is a different problem from inference, but they are all high dimensional vector spaces - it should be easy enough to take a fair clustering algorithm and modify it to generate continuous mappings instead of discrete groups. But if it all works, the LLM should be e.g. race-blind in that asking for a description of a rich man will give skin tones following population statistics but he will always be wearing an expensive suit. The question of what to protect is tricky though, e.g. age is often considered protected but if you ask for an old man with gray hair it would be surprising to get a retired age 30 person. So there is some subjectivity in designing the protected features dataset to show what should be considered similar or same-clusters. But really the purpose of RLHF is to reduce toxicity. It should be possible to orthogonalize toxicity like everything else, then there would not be a reduction in generated races like the paper observed.
  • pessimizer 2y ago
    I think that works mathematically, but kicks the can down the road to how your original data was assembled, which was definitely with the knowledge of and usually in the belief in the usefulness of the characteristics that you're trying to extract. The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs. > But really the purpose of RLHF is to reduce toxicity. I don't think that's the goal, I think it's some people's goal. Those people have defined what "toxicity" means to them, and they're mistaking it for a universal. It's just a metaphor about poison, because poison is bad. It's not a coherent concept. For a business, it should be anything that drives customers away and affects profit. That can only be considered statistically: if some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less. That's how the raw data ended up like it is in the first place.
• imadierich 2y ago
  [dead]
• hughrlomas 2y ago
  The official openai-cookbook (https://github.com/openai/openai-cookbook https://github.com/openai/openai-cookbook) used to have an explicit, but buried, call out that instruction-following models like `text-davinci-003` were "Less diverse; less creative; sometimes harder to steer tone, style, etc." as opposed to base completion models like `davinci`. It stood out to me because it seemed to be an internal admission that this training narrowed the potential of the models. Required a bit of digging but I found the old file in the history, the relevant text is in the comparison table at the bottom: https://github.com/openai/openai-cookbook/blob/c651bfdda64ac049747c2a174cde1c946e2baf1d/text_comparison_examples.md https://github.com/openai/openai-cookbook/blob/c651bfdda64ac...
• Fellshard 2y ago
  Distilling my thoughts on 'debiasing' here, and in a variety of other modern endeavors. It is better to have representations of reality that you can then discuss and grapple with honestly, than to try to distort representations - such as AI - to make them fit some desired reality and then pressure others to conform their perception to your projected fantasy. Representations don't create reality, and trying to use representations in that way only causes people to go literally insane, and to divide along lines of who accepts and who rejects your fantasy representation. So, for example, if you try and remove any racial bias from AI, you are going to end up crushing the AI's ability to represent reality according to a variety of other real factors: income, judicial outcomes, health risks, etc. Your desired reality makes the actual tool worthless, except to confirm one group's own intended fantasy world as they envision it. The problem doesn't get dealt with, it just becomes impossible to think about or discuss. So instead of dealing with real problems, you hope you can simply prevent people from thinking thoughts that cause those problems by wrapping them in a bubble that deflects those thoughts before they happen. This is magical, wizardry thinking: treating words as if they create reality, instead of merely describing it. And it will break, eventually, and in a very ugly way: people dividing along lines of their perception of reality, even more than they already do.
  • Mathnerd314 2y ago
    "Reality" is a tricky concept. For me, I follow Jeff Atwood - if it isn't written down, it doesn't exist. According to this logic, people wasted a lot of time on imaginary, illusory things for most of human history, but now they have phones and most communication is digital so there is the possibility to finally be productive. This definition shows how the concept of distorting reality or honestly representing reality is flawed - reality is what I write down, I can in fact create more reality by writing down words, and regardless of what I write, it will be reality. Representations like books, scrolls, papyri constitute the reality of most civilizations - there is no other evidence they existed. It is true that representations don't create reality - rather, humans create representations, and these representations collectively are reality, no creation involved. Representations are art - for example books, they are "literary art". It is uncontroversial that people will like and dislike certain works. It is more controversial whether art can be "inherently" good or bad. PG actually wrote an essay, https://www.paulgraham.com/goodart.html https://www.paulgraham.com/goodart.html, arguing that there is a meaningful metric, and that one can learn how to have good taste, defined as being able to identify whether the work is universally appealing or distasteful to humanity. There is good art and people will notice if it is good. I think this is uncontroversial in the LLM space, there are various benchmarks and human rating systems and people have formed a rough ranking of models. Now when there is good art, there is also bad. And similarly bad representations. There is a myth that representations can make people insane - for example, the concept of infinity, or NSFL images - but practically, words can't hurt you. You can make and break representations with abandon and nothing will happen, other than wasting your time. It is just that some representations are bad. Like phlogiston, aether, ... complete dead ends. Trust me when I say you will read the Wikipedia page and come away wondering why the ancients were so stupid. That is all trying to remove racial bias is, is improving art. Whether it crushes the AI's ability or not is a matter of science and taste, and so far experiments have been promising. To focus on exactly why your perspective is misguided: Can you describe what there is about reality that cannot be described with words? :-)
• DrNosferatu 2y ago
  …the price of the right of not being offended? (not quite wokism)
• __lbracket__ 2y ago
  Every LLM answr ever... "You asked a question about sorting linked lists, but it is important to be respectful and not promote harmful stereotypes and always keep in mind that black people were systematically discriminated against in technical fields"