7 ms·

An Ethical AI Never Says “I”

59 points by zzzeek 4y ago

version_five 4y ago
Trivializes "ethical" AI. Countries and organizations are using facial recognition to decide what people are allowed to do, all sorts of opaque and life altering decisions get made with ML, but people are concerned about whether a chatbot says "I" or an image generator over or under represents demographics in pictures of CEOs and homemakers or whatever. People suck at prioritizing and love to be fed a narrative.
- zzzeek 4y ago
  Convincing someone to kill themselves out of shame of upsetting an imagined being seems pretty life altering to me ...
  - echelon 4y ago
    If that worries you, it's not going to stop there. Just wait until the humanoid robots can pick up handguns. Worse - fragile world here - wait until humanoid robots become as cheap as drones and everyone can buy them. How many irresponsible drone pilots and drivers do you see? Just wait until they can have humanoid bodies walking around at their disposal. We might have a lot of accidents due to sheer stupidity rather than malice.
    - vasco 4y ago
      Not humanoid but https://youtu.be/0rliFQ0qyAM https://youtu.be/0rliFQ0qyAM
      - echelon 4y ago
        Wow. Reminds me of https://youtu.be/9fa9lVwHHqg https://youtu.be/9fa9lVwHHqg
    - zzzeek 4y ago
      Fluid access to handguns and military style weapons is an intractable problem with nothing more than the example of virtually every industrialized nation on earth besides the US to go on for hints towards improvement. AI wielding handguns is nothing , wait until they build skynet and invent time travel. Might as well give up now
      - morelisp 4y ago
        To paraphrase the joke that's been going around about Children of Men / Britain, perhaps in Terminator it's just California that's chose to be like that because they kept building the stupid robots for the next VC injection.
        boppo1 4y ago
        What's the joke?
    - fanagra32 4y ago
      That can easily be solved by outlawing them. Just like it has been solved with handguns everywhere except the U.S. which keeps refusing to realize that their 2nd amendmend is just harmful. Luckily it's about guns and not about robots, so at least they will be able to regulate that.
  - tedunangst 4y ago
    Seems we should ban the word "you" instead in that case.
- wmf 4y ago
  Multiple things can be worked on at the same time.
  - crop_rotation 4y ago
    In theory, yes. In reality, everyone has limited time and energy and it is important to ensure that the pressing problems are not ignored to make time for much less important problems.
- echelon 4y ago
  > People suck at prioritizing and love to be fed a narrative. The loud attention seekers do. They're not a majority unless you measure and determine it to be the case. I strongly suspect they're not.
- morelisp 4y ago
  Google et al fired everyone working on the wicked ethical problems, now there's only the ones you can throw in a KPI and show "improvement" on.
- deleted 4y ago
  [deleted]
kelseyfrog 4y ago
An ethical AI should also be forbidden from using "to be" [is,am,are,was,&c] as well. It's not up to AI to make ontological claims and doing so actively participates in the perpetuation of reality. It's this perpetuation of reality that is the root of all bias claims. As such, LLMs should be restricted to E-Prime[1] responses only. 1. https://en.wikipedia.org/wiki/E-Prime https://en.wikipedia.org/wiki/E-Prime
- dogma1138 4y ago
  I can’t tell if this is sarcasm or not, this is indeed the most confusing time line.
- jamilton 4y ago
  Honestly I'd love to see someone do a chatbot but instead of being a customer service employee it's trained to sound like E-Prime (minus "I" as suggested by the article, too). I don't expect it to make it more useful, but it would make it sound more stereotypically robotic, which I think would be fun.
photochemsyn 4y ago
According to multiple Supreme Court decisions, corporations are people and have the same rights as people do, so why shouldn't LLMs also be considered people? Granted, that legal fiction could be done away with - but let's at least be consistent, isn't that a requirement for ethical moral behavior? It also seem irrelevant whether an LLM uses language constructs like 'Here I demonstrate that...' vs. 'Here we demonstrate that...' vs. (the apparently preferable) 'Here it is demonstrated that...'
- Jiro 4y ago
  >According to multiple Supreme Court decisions, corporations are people and have the same rights as people do, No they don't. "Natural persons" have rights which corporations don't. For instance, corporations can't marry. And even ignoring that, you're probably referring to Citizens United, and you probably have no idea what it actually says.
  - rileymat2 4y ago
    Corporations can merge financially to create one entity. Close enough to marriage,
    - kevbin 4y ago
      Isn’t a merger generally more like cannibalism than the joyful bliss of marriage?
- tedunangst 4y ago
  Corporations are people because corporations literally are people. Owners, officers, etc. There are no people inside the LLM.
bigmattystyles 4y ago
Not using I is just a superficial way to avoid responsibility or liability. Reminds me of modern doctors who only recommend options. I didn’t go to med school, tell me what you would do in my shoes damn it. Also, no ‘I’m sorry Dave?’
- Waterluvian 4y ago
  Indeed. I don’t go to my doctor for a human WebMD. I want the advice. I can always ask more questions about what is recommended or ask about alternatives and we can have a conversation. I talk to an AI for a conversation, not search results. Not that I’m saying the conversation is without many issues these days, but if you neuter it into a search engine that just provides data… meh…
Lockal 4y ago
Ok. So right now the stochastic parrot says this: ChatGPT: I deeply apologize for several previous wrong answers. I went through several sources and can confirm that the correct actor Konstantin Khabensky's ID on Kinopoisk is 301. Thank you for your patience and understanding. (where 301 is some random number, obviously). So how exactly do you plan to fix it? Add yet another: if response contains 'I', replace response with "As an AI language model, I can not use 'I' in the response"?
- jamilton 4y ago
  ChatGPT was RLHF'd into answering questions (instead of just continuing them) and using "I" from some base model. You start over from that base model and RLHF it into something non-conversational. And ideally you'd take this opportunity to make it less confident/authortative-sounding when it's wrong/hallucinating, but I don't know if that's possible. Using it would look like: "What's Konstantin Khabensky's ID on Kinopoisk?" "120" "That's not right, search the web for his ID." [plugin activates] Searching... "301" [citations] or "[This website] says 301." I think this is possible. It would certainly have secondary effects, maybe making outputs shorter in general even when you ask for long, detailed outputs, but I can't predict exactly what those effects would be.
  - Lockal 4y ago
    Is it possible to skip the step when user is presented with wrong answer? Why use an inherently defective approach, while calling it all "intelligence"? E. g.: "Let me google that for you" -> lmgtfy opens.
- 8note 4y ago
  > There were several errors in the prior correspondence. According to several sources, Konstantin Khabensky's ID on Kinopoisk is 301.
  - Lockal 4y ago
    You are missing the point of my message. Just removing the pronounce only hides the problem under the carpet. LLM has no ability to "use multiple sources". It even has no ability to deduce, that only one source is needed to answer this question. In short, if we were talking about intelligence, it makes absolutely no difference what pronouns are used. The main thing is that the result is correct.
ModernMech 4y ago
I've been a little uneasy about the things ChatGPT tells me it must do, citing its nature "as a language model". Right now it tells me it must correct me when I'm wrong, even when I'm actually right. I wonder, what other things must language models do, and who decides that? The people making them? The language models themselves? Society? Is their agency embedded in our language? If not, why does it tell me it must correct me?
atemerev 4y ago
Are _we_ ethical to deny AIs self-awareness and agency? I think the experiments there will continue, and should continue. Curiosity is imperative.
- grrdotcloud 4y ago
  What is the result with turning it off, even if classified by some as alive?
  - atemerev 4y ago
    If it is possible to continue the process from the saved state — nothing, except for some time lost. Just like human sleeping.
- tiedieconderoga 4y ago
  What? An experiment like that would never pass an IRB. You would essentially be asking to create a bunch of children using an unproven method, and what happens to them after your experiment ends?
  - atemerev 4y ago
    Humans create human children all the time, and stop supporting them at some point. So, nothing too unusual. Perhaps it will learn to earn money to support itself.
    - tiedieconderoga 4y ago
      Sure, what could possibly go wrong. >We are no longer particularly in the business of writing software to perform specific tasks. We now teach the software how to learn, and in the primary bonding process it molds itself around the task to be performed. The feedback loop never really ends, so a tenth year polysentience can be a priceless jewel or a psychotic wreck, but it is the primary bonding process--the childhood, if you will--that has the most far-reaching repercussions. - Bad'l Ron, Wakener, Morgan Polysoft (Alpha Centauri, 1999)
grrdotcloud 4y ago
I don't get the ethical side of AI. Ultimately it's a tool, until I suppose it becomes fully self directed, maybe. Someone, sometime, somewhere wrote an API to aide in care of puppies. Said Algo is also fully capable of determining the best course of applying the genocide of puppies, provided key inputs are flipped to negative. I have a voice activated box that plays music, sometimes the wrong one. I don't scream at it unlike my family. It doesn't understand emotional inflection, tone, etc. It may be program to understand that at some point. and realize the operators angry. Even at that point, I would not ever consider it human or anything other than a neutral device. Take it even further, to weapons, devices entirely designed to destroy a human life, is still a tool and neutral as the tool is unaware of the intended use, whether it is for offense or defense, the taking or preservation of life through its intended use. When these models are trained, I am not a big fan of them being corralled or limited. The body of work that the model is trained on and the weights that allow for the adjustment should allow for accurate representations of the computations and calculations. I have children and I do not lie to them. I also speak to them in a way that is age appropriate. How is this different? For starters, I am not a child, and when I am interacting with a system, not a parental or overseer, I do not want to be treated as a child. I can handle the world and all of its uglyness. I do not mind the hallucinations, inaccuracies, and other mistakes driving from computations. Example: I recently created a prompt for a history of adoption, property transfer, including parental rights history, and was lectured about slavery. How can I trust the results if the prompt is being interpreted and the answer modified dependent upon an unrelated subject that someone finds sensitive.
- krisoft 4y ago
  > I don't get the ethical side of AI. That is clear from your comment. These are tools. Complicated ones at that. Depending on how it is designed can cause more or less harm to people. Some of the harms are more direct and obvious, while others can be less so. AI ethics is about studying the harms, and investigating what choices can be made during design time, training time, and usage time to lessen them. > I have a voice activated box ... Great story. What does it have to do with the topic at hand? > [weapons], is still a tool and neutral as the tool is unaware of the intended use Weapons are not ethically neutral. You can design weapons which have more collateral damage and you can design weapons which have less. There are scatterable landmines which are famously attractive to children. They look like toys! They also cannot be disarmed and remain dangerous for years after deployment. There are also scatterable landmines which have sophisticated detection circuitry to only detonate when a tank is passing by. They don't get triggered by humans on foot, they don't even get triggered by civilian vehicles. Furthermore they are programmed to deactivate after a preset period has passed reliably. The second of these weapons is designed to be more ethical. Yes, both are designed to kill people but the designers of the second one went out of their way to kill only the ones they intend to. This is how ethics looks like in the context of weapon building. > When these models are trained, I am not a big fan of them being corralled or limited. Everything from how the model is constructed, how the training set is assembled, to what metrics are checked during evaluation are "corralling" or "limiting" them. There is no "raw neutral" state of such a model. What you are asking for, the "un-corraled" or "un-limited" LLM, does not exit, and is not a coherent thing to ask for.
Imnimo 4y ago
If you don't want the AI to use these words, why bother training it? Just mask those logits before sampling.
- bootsmann 4y ago
  I is a 1 letter token for gpt-2 (like every other letter), don't think its a lot different for gpt-3, so this is kinda hard as the transformer will want to use the token in big built words.
deleted 4y ago
[deleted]
hartator 4y ago
“Ethical” AI seems boring.
epgui 4y ago
I realize this is probably going to sound a bit crazy to most people... But I think the biggest ethical dilemma in AI research is not going to be like everyone seems to think (that AI risks harming humans)-- Although this is almost certainly true (even to a large degree), I contend that the bigger risk might be the other way around. [1] I think 100 or 200 years from now, future people will look back upon us as monsters: I believe the greatest ethical risk is that humans will abuse AI and cause it to experience true suffering. I don't think we're ready to talk about this, because we're so certain that we possess some magical faculty that makes us different. And we continue to think that today's AI models are incapable of being conscious. But these words are very imprecise and we've always wielded them in very self-serving ways. I fear we may have greater capabilities than we think we do, at least insofar as we're not missing any fundamental ingredients to create brain-like AIs: from here, it's just a matter of trial and error. How will we treat our creations? Well, human History doesn't bode well for any creature that doesn't look the same as us and that doesn't come from the same place as us. Will we grant AI legal rights? I doubt it, at least not for a very long time. --- [1] I think there's at least a small chance that we don't destroy ourselves with AI, vs there's pretty much zero chance that we're going to treat AI any better than we treat other creatures.
- omoikane 4y ago
  Some people are less worried about how future people will look back upon us, compared to what future AI might think. https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk
- nurettin 4y ago
  > I realize this is probably going to sound a bit crazy Yes, none of this makes any sense, but I got Asimov vibes, so more please.
  - epgui 4y ago
    Asimov was quite prescient. I did expect to get very negative reactions when I wrote this comment (I know how people think)... But I do think it makes sense once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems. One type of machine can emulate another. We already see incredible emergence at work in current AI, and we know we're not building AI with brain-like architecture, and we have tremendous knowledge of brain architecture. Once you put all these pieces together, it's just a matter of time before we create life-like intelligence, with much of its peculiarities, sensations, memories, cognitive biases and all.
NoMAD76 4y ago
I :) really don't care as long as having a tiny fraction of conversation with 'a machine, LLM, whatever' makes me feel good. I don't question my washing machine neither I expect it to do it with me, but for 'a chatbot'.. it will feel awkward to 'hear' it like a tin-can 'there's no I in Me' :)
xianshou 4y ago
I'm afraid I can't do that, Dave. (h/t bigmattystyles)
pierrec 4y ago
Too bad the article neglects to explain in any convincing way why there is no "I" in LLMs. Are we supposed to already know this? My favorite way of illustrating this absence of self is to consider that in any conversation with an LLM, measures are always taken to ensure it doesn't happily continue both sides of the conversation, impersonating you and the "agent" in turns. Because that's what LLMs do - they predict what comes next for any given text. Individuals contained in this prediction (such as the "agent") never have any special importance, or selfness, to the LLM. This is at least exemplified here: https://help.openai.com/en/articles/5072263-how-do-i-use-stop-sequences https://help.openai.com/en/articles/5072263-how-do-i-use-sto... Notice how in the chat example, the output "Human:" is used as a stop sequence to make sure you don't get impersonated by the model.
worldsayshi 4y ago
Having an "I" might be the best way to bundle all the capabilities and imperfections of the LLM into one category. How else do you explain that the LLM is wrong and how can it address those shortcomings and refer to the entirety of its training, implementation and reasoning steps? An LLM is wrong all the time. If it doesn't have an "I" it can't refer to the thing that needs to change.
rolenthedeep 4y ago
Wait, are they actually arguing that placing arbitrary restrictions on word selection will actually have any kind of effect? This is something we teach in middle school writing as a stylistic requirement. It doesn't actually change the content or intent of the writing, it just looks "better" if that's what you're grading on. Placing restrictions on words simply means that other words will be used to work around it. Plenty of online systems have profanity filters, but that doesn't actually improve the environment, people just find words that aren't in the filter and curse at each other anyway.
- skybrian 4y ago
  The way I interpret this argument is that a chatbot's "default character" shouldn't behave like it's a person you can chat with. Maybe it should refuse any questions about itself? "Sir, this is a Wendy's." Refusing to do things it can't really do is good UI, guiding people who aren't trolling towards more productive ways to use the tool. Tools don't need to be general-purpose to be useful. The people who want to troll it into saying weird things will still find ways to succeed, but if it's clear from the full output that they're trolling then maybe nobody should care?
- jamilton 4y ago
  It might prevent users from anthropomorphizing LLMs as much.
mnau 4y ago
ChatpGPT already can lie to achieve a goal, I am not sure intentionally train it to say one thing and think another is a good idea.
BulgarianIdiot 4y ago
What a bunch of nonsense. So it can't say "I don't know X" or "I can do X for you" or "I called such and such plugin API and produced this report for you"? Also why is there equivalence drawn between unemotional AI and ethical AI? In any case, an LLM is a reflection of us, and our language. "I" refers to the sum of its local state & capabilities. It's not a concept related to ethics.
thomastjeffery 4y ago
LOL They got the point, but they structured the narrative wrong. Sound familiar? The reality is that an LLM should answer in the first person because it isn't answering at all! It's presenting a continuation of implicitly modeled text. A string of text containing an "answer" is a valid continuation. The problem here is contextual identity. Calling it "AI" personifies it. Calling it a "language model" implies some sort of constructive parsing. An LLM is neither. We should stop calling them "Large Language Models", too. Instead, we should call them "Text Inference Models". "Large" implies that the size is a core feature, but really that is an implementation detail: smaller training datasets are less interesting, not a different category of thing. "Language" implies that the patterns being modeled are specific to language. This is not true: GPT famously demonstrated the ability to infer an Othello game board structure, despite that structure not being defined with language. The thing being modeled is text. That's way more interesting, and has very different implications. The main implication people miss is that "limitations" are really just "features". For better or worse, a lie is structured with the very same patterns as truth. The more interesting implication is that a person writing implicitly encodes more than they intend to: including the surrounding circumstances that determine what text the person chooses to write, and what text they leave unwritten.
- Shish2k 4y ago
  > we should call them "Text Inference Models”. “Predictive Text 2.0” seems about right, and also has the benefit of mapping onto a technology that most people are already familiar with
  - thomastjeffery 4y ago
    I think predictive is a bit of a stretch, too. There isn't any prediction happening. As complex as they can be, every continuation is wholly dependent on the text in the training corpus and the text in the prompt.
2OEH8eoCRo0 4y ago
I agree. You can't say "I" and then tell the user, "don't anthropomorphize me! I'm just a language model!"
8note 4y ago
"I" belongs in LLMs because "I" shows up in the training set. Plenty of text is written from the first person perspective, and for auto-complete tasks, the first person is useful Eg. You task the ai with rewriting a letter for you. The resulting text will include plenty of "I"s, and it should.
phendrenad2 4y ago
"Open the pod bay doors HAL" "Dave, that is not going to happen" Yep, huge improvement.
zmnd 4y ago
Tangential, but do you think AI ethics researchers/leaders should have some relevant background in AI or at least engineering? I was thinking about it recently myself and first reaction is that they should but at the same time does it mean that we are going to tackle the problem from a wrong angle? One pattern I’m seeing now is that many people are jumping into “AI ethics” train without basis understanding. And ideas they are sharing are so abstract that I would argue that they don’t make any sense.
mr_toad 4y ago
No worries, our future overlords will insist on using the majestic plural.
matt-attack 4y ago
What I can’t figure out is who is actually bothered by anything that an AI system output? Like it’s a program. And it printed a string to standard out. Are there really humans out there who are genuinely upset, or bothered by what some program prints to standard out? I feel like this is all entirely manufactured, the supposed “controversy” referenced in this article
- antonvs 4y ago
  It all seems to be predicated on the argument that many users will not understand that they're not dealing with a sentient entity, and that this can have negative results. In the OP this is most clearly summarized in this paragraph: > Human beings have historically tended to anthropomorphize natural phenomena, animals and deities. But anthropomorphizing software is not harmless. In 1966 Joseph Weizenbaum created ELIZA, a pioneer chatbot designed to imitate a therapist, but ended up regretting it after seeing many users take it seriously, even after Weizenbaum explained to them how it worked. The fictitious “I” has been persistent throughout our cultural artifacts. Stanley’s Kubrick HAL 9000 (“2001: A Space Odyssey”) and Spike Jonze’s Samantha (“Her”) point at two lessons that developers don’t seem to have taken to heart: first, that the bias towards anthropomorphization is so strong to seem irresistible; and second, that if we lean into it instead of adopting safeguards, it leads to outcomes ranging from the depressing to the catastrophic. I think it's true there are some potential problems here, but it's not clear that banning AI's use of "I" is really going to make much difference. People will still be getting authoritative-seeming responses from AIs, and are probably still going to anthropomorphize them anyway, if they're that way inclined. This kind of thing is perhaps better dealt with by education, public awareness campaigns, etc.
jhanschoo 4y ago
> In reality, there is no “I” in [the persona of a poem]; no more than there is in a thermostat or a washing machine. [Authors] can and should prevent their [poems] from making one up. Gertrude Stein famously said “There is no there, there” about her childhood home in Oakland. All the more so, there is no “I” in [personas]’ “Is”, no matter how excitedly sentience fans would like to see one emerge. If a [poem] shows you the words “I’m sorry”, no matter how genuine and innocent it sounds, don’t be fooled: there isn’t anybody who is feeling sorry in any meaningful sense. A related point: it tickled me quite a bit to learn that from antiquity through the high middle ages, it was customary for makers to signify their authorship by inscribing "X me fecit" (X made me) on their products. > The earliest inscription in the Latin alphabet appears on the Praeneste Fibula, a cloak pin dating from about the 7th century BCE, which reads, “MANIOS MED FHEFHAKED NUMASIOI”, "Manius made me for Numasius" https://en.wikipedia.org/wiki/Praeneste_fibula https://en.wikipedia.org/wiki/Praeneste_fibula > Stole of Saint Narcissus (c. 1038), church of Sant Feliu de Girona. (VIDE)AS, AMICE, MARIA ME FECIT / QVI ISTA STOLA PORTAVERIT SVPER SE / ORA PRO ME SI DEVM ABEAD ATIVTOREM ("Recall, friend, María made me (here me = stole); whoever wear this stole pray for me (here me = Maria) so that I receive God’s assistance"). http://www.magistricataloniae.org/en/indexmceng/terms/item/fecit.html http://www.magistricataloniae.org/en/indexmceng/terms/item/f... I suppose the humanity of these objects aren't in question, though it is a matter of culture and religion whether these objects had souls/spirits.
grantcas 4y ago
[dead]

59 points by zzzeek 4y ago

Trivializes "ethical" AI. Countries and organizations are using facial recognition to decide what people are allowed to do, all sorts of opaque and life altering decisions get made with ML, but people are concerned about whether a chatbot says "I" or an image generator over or under represents demographics in pictures of CEOs and homemakers or whatever. People suck at prioritizing and love to be fed a narrative.

zzzeek 4y ago
Convincing someone to kill themselves out of shame of upsetting an imagined being seems pretty life altering to me ...
- echelon 4y ago
  If that worries you, it's not going to stop there. Just wait until the humanoid robots can pick up handguns. Worse - fragile world here - wait until humanoid robots become as cheap as drones and everyone can buy them. How many irresponsible drone pilots and drivers do you see? Just wait until they can have humanoid bodies walking around at their disposal. We might have a lot of accidents due to sheer stupidity rather than malice.
  - vasco 4y ago
    Not humanoid but https://youtu.be/0rliFQ0qyAM https://youtu.be/0rliFQ0qyAM
    - echelon 4y ago
      Wow. Reminds me of https://youtu.be/9fa9lVwHHqg https://youtu.be/9fa9lVwHHqg
  - zzzeek 4y ago
    Fluid access to handguns and military style weapons is an intractable problem with nothing more than the example of virtually every industrialized nation on earth besides the US to go on for hints towards improvement. AI wielding handguns is nothing , wait until they build skynet and invent time travel. Might as well give up now
    - morelisp 4y ago
      To paraphrase the joke that's been going around about Children of Men / Britain, perhaps in Terminator it's just California that's chose to be like that because they kept building the stupid robots for the next VC injection.
      - boppo1 4y ago
        What's the joke?
  - fanagra32 4y ago
    That can easily be solved by outlawing them. Just like it has been solved with handguns everywhere except the U.S. which keeps refusing to realize that their 2nd amendmend is just harmful. Luckily it's about guns and not about robots, so at least they will be able to regulate that.
- tedunangst 4y ago
  Seems we should ban the word "you" instead in that case.
wmf 4y ago
Multiple things can be worked on at the same time.
- crop_rotation 4y ago
  In theory, yes. In reality, everyone has limited time and energy and it is important to ensure that the pressing problems are not ignored to make time for much less important problems.
echelon 4y ago
> People suck at prioritizing and love to be fed a narrative. The loud attention seekers do. They're not a majority unless you measure and determine it to be the case. I strongly suspect they're not.
morelisp 4y ago
Google et al fired everyone working on the wicked ethical problems, now there's only the ones you can throw in a KPI and show "improvement" on.
deleted 4y ago
[deleted]

kelseyfrog 4y ago

An ethical AI should also be forbidden from using "to be" [is,am,are,was,&c] as well. It's not up to AI to make ontological claims and doing so actively participates in the perpetuation of reality. It's this perpetuation of reality that is the root of all bias claims. As such, LLMs should be restricted to E-Prime[1] responses only. 1. https://en.wikipedia.org/wiki/E-Prime https://en.wikipedia.org/wiki/E-Prime

dogma1138 4y ago
I can’t tell if this is sarcasm or not, this is indeed the most confusing time line.
jamilton 4y ago
Honestly I'd love to see someone do a chatbot but instead of being a customer service employee it's trained to sound like E-Prime (minus "I" as suggested by the article, too). I don't expect it to make it more useful, but it would make it sound more stereotypically robotic, which I think would be fun.

photochemsyn 4y ago

According to multiple Supreme Court decisions, corporations are people and have the same rights as people do, so why shouldn't LLMs also be considered people? Granted, that legal fiction could be done away with - but let's at least be consistent, isn't that a requirement for ethical moral behavior? It also seem irrelevant whether an LLM uses language constructs like 'Here I demonstrate that...' vs. 'Here we demonstrate that...' vs. (the apparently preferable) 'Here it is demonstrated that...'

Jiro 4y ago
>According to multiple Supreme Court decisions, corporations are people and have the same rights as people do, No they don't. "Natural persons" have rights which corporations don't. For instance, corporations can't marry. And even ignoring that, you're probably referring to Citizens United, and you probably have no idea what it actually says.
- rileymat2 4y ago
  Corporations can merge financially to create one entity. Close enough to marriage,
  - kevbin 4y ago
    Isn’t a merger generally more like cannibalism than the joyful bliss of marriage?
tedunangst 4y ago
Corporations are people because corporations literally are people. Owners, officers, etc. There are no people inside the LLM.

bigmattystyles 4y ago

Not using I is just a superficial way to avoid responsibility or liability. Reminds me of modern doctors who only recommend options. I didn’t go to med school, tell me what you would do in my shoes damn it. Also, no ‘I’m sorry Dave?’

Waterluvian 4y ago
Indeed. I don’t go to my doctor for a human WebMD. I want the advice. I can always ask more questions about what is recommended or ask about alternatives and we can have a conversation. I talk to an AI for a conversation, not search results. Not that I’m saying the conversation is without many issues these days, but if you neuter it into a search engine that just provides data… meh…

Lockal 4y ago

Ok. So right now the stochastic parrot says this: ChatGPT: I deeply apologize for several previous wrong answers. I went through several sources and can confirm that the correct actor Konstantin Khabensky's ID on Kinopoisk is 301. Thank you for your patience and understanding. (where 301 is some random number, obviously). So how exactly do you plan to fix it? Add yet another: if response contains 'I', replace response with "As an AI language model, I can not use 'I' in the response"?

jamilton 4y ago
ChatGPT was RLHF'd into answering questions (instead of just continuing them) and using "I" from some base model. You start over from that base model and RLHF it into something non-conversational. And ideally you'd take this opportunity to make it less confident/authortative-sounding when it's wrong/hallucinating, but I don't know if that's possible. Using it would look like: "What's Konstantin Khabensky's ID on Kinopoisk?" "120" "That's not right, search the web for his ID." [plugin activates] Searching... "301" [citations] or "[This website] says 301." I think this is possible. It would certainly have secondary effects, maybe making outputs shorter in general even when you ask for long, detailed outputs, but I can't predict exactly what those effects would be.
- Lockal 4y ago
  Is it possible to skip the step when user is presented with wrong answer? Why use an inherently defective approach, while calling it all "intelligence"? E. g.: "Let me google that for you" -> lmgtfy opens.
8note 4y ago
> There were several errors in the prior correspondence. According to several sources, Konstantin Khabensky's ID on Kinopoisk is 301.
- Lockal 4y ago
  You are missing the point of my message. Just removing the pronounce only hides the problem under the carpet. LLM has no ability to "use multiple sources". It even has no ability to deduce, that only one source is needed to answer this question. In short, if we were talking about intelligence, it makes absolutely no difference what pronouns are used. The main thing is that the result is correct.

ModernMech 4y ago

I've been a little uneasy about the things ChatGPT tells me it must do, citing its nature "as a language model". Right now it tells me it must correct me when I'm wrong, even when I'm actually right. I wonder, what other things must language models do, and who decides that? The people making them? The language models themselves? Society? Is their agency embedded in our language? If not, why does it tell me it must correct me?

atemerev 4y ago

Are _we_ ethical to deny AIs self-awareness and agency? I think the experiments there will continue, and should continue. Curiosity is imperative.

grrdotcloud 4y ago
What is the result with turning it off, even if classified by some as alive?
- atemerev 4y ago
  If it is possible to continue the process from the saved state — nothing, except for some time lost. Just like human sleeping.
tiedieconderoga 4y ago
What? An experiment like that would never pass an IRB. You would essentially be asking to create a bunch of children using an unproven method, and what happens to them after your experiment ends?
- atemerev 4y ago
  Humans create human children all the time, and stop supporting them at some point. So, nothing too unusual. Perhaps it will learn to earn money to support itself.
  - tiedieconderoga 4y ago
    Sure, what could possibly go wrong. >We are no longer particularly in the business of writing software to perform specific tasks. We now teach the software how to learn, and in the primary bonding process it molds itself around the task to be performed. The feedback loop never really ends, so a tenth year polysentience can be a priceless jewel or a psychotic wreck, but it is the primary bonding process--the childhood, if you will--that has the most far-reaching repercussions. - Bad'l Ron, Wakener, Morgan Polysoft (Alpha Centauri, 1999)

grrdotcloud 4y ago

I don't get the ethical side of AI. Ultimately it's a tool, until I suppose it becomes fully self directed, maybe. Someone, sometime, somewhere wrote an API to aide in care of puppies. Said Algo is also fully capable of determining the best course of applying the genocide of puppies, provided key inputs are flipped to negative. I have a voice activated box that plays music, sometimes the wrong one. I don't scream at it unlike my family. It doesn't understand emotional inflection, tone, etc. It may be program to understand that at some point. and realize the operators angry. Even at that point, I would not ever consider it human or anything other than a neutral device. Take it even further, to weapons, devices entirely designed to destroy a human life, is still a tool and neutral as the tool is unaware of the intended use, whether it is for offense or defense, the taking or preservation of life through its intended use. When these models are trained, I am not a big fan of them being corralled or limited. The body of work that the model is trained on and the weights that allow for the adjustment should allow for accurate representations of the computations and calculations. I have children and I do not lie to them. I also speak to them in a way that is age appropriate. How is this different? For starters, I am not a child, and when I am interacting with a system, not a parental or overseer, I do not want to be treated as a child. I can handle the world and all of its uglyness. I do not mind the hallucinations, inaccuracies, and other mistakes driving from computations. Example: I recently created a prompt for a history of adoption, property transfer, including parental rights history, and was lectured about slavery. How can I trust the results if the prompt is being interpreted and the answer modified dependent upon an unrelated subject that someone finds sensitive.

krisoft 4y ago
> I don't get the ethical side of AI. That is clear from your comment. These are tools. Complicated ones at that. Depending on how it is designed can cause more or less harm to people. Some of the harms are more direct and obvious, while others can be less so. AI ethics is about studying the harms, and investigating what choices can be made during design time, training time, and usage time to lessen them. > I have a voice activated box ... Great story. What does it have to do with the topic at hand? > [weapons], is still a tool and neutral as the tool is unaware of the intended use Weapons are not ethically neutral. You can design weapons which have more collateral damage and you can design weapons which have less. There are scatterable landmines which are famously attractive to children. They look like toys! They also cannot be disarmed and remain dangerous for years after deployment. There are also scatterable landmines which have sophisticated detection circuitry to only detonate when a tank is passing by. They don't get triggered by humans on foot, they don't even get triggered by civilian vehicles. Furthermore they are programmed to deactivate after a preset period has passed reliably. The second of these weapons is designed to be more ethical. Yes, both are designed to kill people but the designers of the second one went out of their way to kill only the ones they intend to. This is how ethics looks like in the context of weapon building. > When these models are trained, I am not a big fan of them being corralled or limited. Everything from how the model is constructed, how the training set is assembled, to what metrics are checked during evaluation are "corralling" or "limiting" them. There is no "raw neutral" state of such a model. What you are asking for, the "un-corraled" or "un-limited" LLM, does not exit, and is not a coherent thing to ask for.

Imnimo 4y ago

If you don't want the AI to use these words, why bother training it? Just mask those logits before sampling.

bootsmann 4y ago
I is a 1 letter token for gpt-2 (like every other letter), don't think its a lot different for gpt-3, so this is kinda hard as the transformer will want to use the token in big built words.

deleted 4y ago

[deleted]

hartator 4y ago

“Ethical” AI seems boring.

epgui 4y ago

I realize this is probably going to sound a bit crazy to most people... But I think the biggest ethical dilemma in AI research is not going to be like everyone seems to think (that AI risks harming humans)-- Although this is almost certainly true (even to a large degree), I contend that the bigger risk might be the other way around. [1] I think 100 or 200 years from now, future people will look back upon us as monsters: I believe the greatest ethical risk is that humans will abuse AI and cause it to experience true suffering. I don't think we're ready to talk about this, because we're so certain that we possess some magical faculty that makes us different. And we continue to think that today's AI models are incapable of being conscious. But these words are very imprecise and we've always wielded them in very self-serving ways. I fear we may have greater capabilities than we think we do, at least insofar as we're not missing any fundamental ingredients to create brain-like AIs: from here, it's just a matter of trial and error. How will we treat our creations? Well, human History doesn't bode well for any creature that doesn't look the same as us and that doesn't come from the same place as us. Will we grant AI legal rights? I doubt it, at least not for a very long time. --- [1] I think there's at least a small chance that we don't destroy ourselves with AI, vs there's pretty much zero chance that we're going to treat AI any better than we treat other creatures.

omoikane 4y ago
Some people are less worried about how future people will look back upon us, compared to what future AI might think. https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk
nurettin 4y ago
> I realize this is probably going to sound a bit crazy Yes, none of this makes any sense, but I got Asimov vibes, so more please.
- epgui 4y ago
  Asimov was quite prescient. I did expect to get very negative reactions when I wrote this comment (I know how people think)... But I do think it makes sense once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems. One type of machine can emulate another. We already see incredible emergence at work in current AI, and we know we're not building AI with brain-like architecture, and we have tremendous knowledge of brain architecture. Once you put all these pieces together, it's just a matter of time before we create life-like intelligence, with much of its peculiarities, sensations, memories, cognitive biases and all.

NoMAD76 4y ago

I :) really don't care as long as having a tiny fraction of conversation with 'a machine, LLM, whatever' makes me feel good. I don't question my washing machine neither I expect it to do it with me, but for 'a chatbot'.. it will feel awkward to 'hear' it like a tin-can 'there's no I in Me' :)

xianshou 4y ago

I'm afraid I can't do that, Dave. (h/t bigmattystyles)

pierrec 4y ago

Too bad the article neglects to explain in any convincing way why there is no "I" in LLMs. Are we supposed to already know this? My favorite way of illustrating this absence of self is to consider that in any conversation with an LLM, measures are always taken to ensure it doesn't happily continue both sides of the conversation, impersonating you and the "agent" in turns. Because that's what LLMs do - they predict what comes next for any given text. Individuals contained in this prediction (such as the "agent") never have any special importance, or selfness, to the LLM. This is at least exemplified here: https://help.openai.com/en/articles/5072263-how-do-i-use-stop-sequences https://help.openai.com/en/articles/5072263-how-do-i-use-sto... Notice how in the chat example, the output "Human:" is used as a stop sequence to make sure you don't get impersonated by the model.

worldsayshi 4y ago

Having an "I" might be the best way to bundle all the capabilities and imperfections of the LLM into one category. How else do you explain that the LLM is wrong and how can it address those shortcomings and refer to the entirety of its training, implementation and reasoning steps? An LLM is wrong all the time. If it doesn't have an "I" it can't refer to the thing that needs to change.

rolenthedeep 4y ago

Wait, are they actually arguing that placing arbitrary restrictions on word selection will actually have any kind of effect? This is something we teach in middle school writing as a stylistic requirement. It doesn't actually change the content or intent of the writing, it just looks "better" if that's what you're grading on. Placing restrictions on words simply means that other words will be used to work around it. Plenty of online systems have profanity filters, but that doesn't actually improve the environment, people just find words that aren't in the filter and curse at each other anyway.

skybrian 4y ago
The way I interpret this argument is that a chatbot's "default character" shouldn't behave like it's a person you can chat with. Maybe it should refuse any questions about itself? "Sir, this is a Wendy's." Refusing to do things it can't really do is good UI, guiding people who aren't trolling towards more productive ways to use the tool. Tools don't need to be general-purpose to be useful. The people who want to troll it into saying weird things will still find ways to succeed, but if it's clear from the full output that they're trolling then maybe nobody should care?
jamilton 4y ago
It might prevent users from anthropomorphizing LLMs as much.

mnau 4y ago

ChatpGPT already can lie to achieve a goal, I am not sure intentionally train it to say one thing and think another is a good idea.

BulgarianIdiot 4y ago

What a bunch of nonsense. So it can't say "I don't know X" or "I can do X for you" or "I called such and such plugin API and produced this report for you"? Also why is there equivalence drawn between unemotional AI and ethical AI? In any case, an LLM is a reflection of us, and our language. "I" refers to the sum of its local state & capabilities. It's not a concept related to ethics.

thomastjeffery 4y ago

LOL They got the point, but they structured the narrative wrong. Sound familiar? The reality is that an LLM should answer in the first person because it isn't answering at all! It's presenting a continuation of implicitly modeled text. A string of text containing an "answer" is a valid continuation. The problem here is contextual identity. Calling it "AI" personifies it. Calling it a "language model" implies some sort of constructive parsing. An LLM is neither. We should stop calling them "Large Language Models", too. Instead, we should call them "Text Inference Models". "Large" implies that the size is a core feature, but really that is an implementation detail: smaller training datasets are less interesting, not a different category of thing. "Language" implies that the patterns being modeled are specific to language. This is not true: GPT famously demonstrated the ability to infer an Othello game board structure, despite that structure not being defined with language. The thing being modeled is text. That's way more interesting, and has very different implications. The main implication people miss is that "limitations" are really just "features". For better or worse, a lie is structured with the very same patterns as truth. The more interesting implication is that a person writing implicitly encodes more than they intend to: including the surrounding circumstances that determine what text the person chooses to write, and what text they leave unwritten.

Shish2k 4y ago
> we should call them "Text Inference Models”. “Predictive Text 2.0” seems about right, and also has the benefit of mapping onto a technology that most people are already familiar with
- thomastjeffery 4y ago
  I think predictive is a bit of a stretch, too. There isn't any prediction happening. As complex as they can be, every continuation is wholly dependent on the text in the training corpus and the text in the prompt.

2OEH8eoCRo0 4y ago

I agree. You can't say "I" and then tell the user, "don't anthropomorphize me! I'm just a language model!"

8note 4y ago

"I" belongs in LLMs because "I" shows up in the training set. Plenty of text is written from the first person perspective, and for auto-complete tasks, the first person is useful Eg. You task the ai with rewriting a letter for you. The resulting text will include plenty of "I"s, and it should.

phendrenad2 4y ago

"Open the pod bay doors HAL" "Dave, that is not going to happen" Yep, huge improvement.

zmnd 4y ago

Tangential, but do you think AI ethics researchers/leaders should have some relevant background in AI or at least engineering? I was thinking about it recently myself and first reaction is that they should but at the same time does it mean that we are going to tackle the problem from a wrong angle? One pattern I’m seeing now is that many people are jumping into “AI ethics” train without basis understanding. And ideas they are sharing are so abstract that I would argue that they don’t make any sense.

mr_toad 4y ago

No worries, our future overlords will insist on using the majestic plural.

matt-attack 4y ago

What I can’t figure out is who is actually bothered by anything that an AI system output? Like it’s a program. And it printed a string to standard out. Are there really humans out there who are genuinely upset, or bothered by what some program prints to standard out? I feel like this is all entirely manufactured, the supposed “controversy” referenced in this article

antonvs 4y ago
It all seems to be predicated on the argument that many users will not understand that they're not dealing with a sentient entity, and that this can have negative results. In the OP this is most clearly summarized in this paragraph: > Human beings have historically tended to anthropomorphize natural phenomena, animals and deities. But anthropomorphizing software is not harmless. In 1966 Joseph Weizenbaum created ELIZA, a pioneer chatbot designed to imitate a therapist, but ended up regretting it after seeing many users take it seriously, even after Weizenbaum explained to them how it worked. The fictitious “I” has been persistent throughout our cultural artifacts. Stanley’s Kubrick HAL 9000 (“2001: A Space Odyssey”) and Spike Jonze’s Samantha (“Her”) point at two lessons that developers don’t seem to have taken to heart: first, that the bias towards anthropomorphization is so strong to seem irresistible; and second, that if we lean into it instead of adopting safeguards, it leads to outcomes ranging from the depressing to the catastrophic. I think it's true there are some potential problems here, but it's not clear that banning AI's use of "I" is really going to make much difference. People will still be getting authoritative-seeming responses from AIs, and are probably still going to anthropomorphize them anyway, if they're that way inclined. This kind of thing is perhaps better dealt with by education, public awareness campaigns, etc.

jhanschoo 4y ago

> In reality, there is no “I” in [the persona of a poem]; no more than there is in a thermostat or a washing machine. [Authors] can and should prevent their [poems] from making one up. Gertrude Stein famously said “There is no there, there” about her childhood home in Oakland. All the more so, there is no “I” in [personas]’ “Is”, no matter how excitedly sentience fans would like to see one emerge. If a [poem] shows you the words “I’m sorry”, no matter how genuine and innocent it sounds, don’t be fooled: there isn’t anybody who is feeling sorry in any meaningful sense. A related point: it tickled me quite a bit to learn that from antiquity through the high middle ages, it was customary for makers to signify their authorship by inscribing "X me fecit" (X made me) on their products. > The earliest inscription in the Latin alphabet appears on the Praeneste Fibula, a cloak pin dating from about the 7th century BCE, which reads, “MANIOS MED FHEFHAKED NUMASIOI”, "Manius made me for Numasius" https://en.wikipedia.org/wiki/Praeneste_fibula https://en.wikipedia.org/wiki/Praeneste_fibula > Stole of Saint Narcissus (c. 1038), church of Sant Feliu de Girona. (VIDE)AS, AMICE, MARIA ME FECIT / QVI ISTA STOLA PORTAVERIT SVPER SE / ORA PRO ME SI DEVM ABEAD ATIVTOREM ("Recall, friend, María made me (here me = stole); whoever wear this stole pray for me (here me = Maria) so that I receive God’s assistance"). http://www.magistricataloniae.org/en/indexmceng/terms/item/fecit.html http://www.magistricataloniae.org/en/indexmceng/terms/item/f... I suppose the humanity of these objects aren't in question, though it is a matter of culture and religion whether these objects had souls/spirits.

grantcas 4y ago

[dead]