4 ms·
Mirages: On Anthropomorphism in Dialogue Systems
- toomim 3y ago> In this paper, we discuss the linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise, arguing that it can reinforce stereotypes of gender roles and notions of acceptable language. Humans are dangerous. They reinforce stereotypes of gender roles and notions of acceptable language. /s
- antiquark 3y ago[flagged]
- throwaway675309 3y agoThis is exactly the reason why I strongly dislike when people use the verb "lying" to describe when ChatGPT presents information that is factually inaccurate. It implies intent and people already have a tendency to over-anthropomorphize these systems. Hallucinates is a far more accurate description.
- JieJie 3y agoIsn't "hallucinate" anthropomorphizing as well? Likewise for "confabulating". Naomi Klein wrote in The Guardian, "Why not algorithmic junk? Or glitches?" I think those are pretty descriptive and not anthropomorphic.
- LelouBil 3y agoI guess so, but hallucination doesn't imply intent.
- AdieuToLogic 3y agoAscribing "hallucination" to an algorithm does reinforce anthropomorphizing however.
- ithkuil 3y agoBut it also helps lowering ourselves from the pedestal. We don't yet understand how our brains work but they do "work" somehow and a lot can be understood by looking at how it "fails". Hallucinations are an interesting window in how our brains work and seeing similarities with what other processes do can be useful
- AdieuToLogic 3y agoYou make valid and interesting points. I guess, for me, it is a matter of minimizing bias and implicit personification when considering the value AI has, as well as how to better understand it for what it is. For example, I can better contemplate the innate intelligence of a whale if I do not attempt to quantify same based on what tools they create. Perhaps that is just me though.
- AvAn12 3y agoAgreed. it's just the probabilistic language model running away with generating next (token | word | phrase | sentence | paragraph ) or however it really works under the hood. Rather different than what is going on within the noggin of a human who is hallucinating.
- Obscurity4340 3y agoIsn't that sort of how actual confabulation works? Between various inputs and certain fractured or corrupted "memories" a new input is generated that would attempt to satisfy the initial prompt?
- Obscurity4340 3y ago*new output is generated
- williamcotton 3y agoI like the term “synthesize”. Sometimes what the user wants is non-factual. The other terms have a negative connotation when it could very well be an acceptable response for a given request.
- Obscurity4340 3y agoThat doesn't really convey the non-factuality or "correctness" of the answer relative to the context and expected output. Every answer is in some sense "synthesized" so its not really a meaningful way to distinguish output. Confabulation seems to me to be the least-problematic way of describing it.
- fnordpiglet 3y agoI prefer it’s displaying abductive reasoning. It’s able to produce likely answers with limited information or confounding information. It’s a type of reasoning that falls between inductive and deductive reasoning, and is a type of reasoning AI has been before now poor at. Note that reasoning isn’t itself accurate in that LLMs have no agency. But they display characteristics of abductive reasoning. By integrating LLMs into goal based agents and tying them together with inductive and deductive solvers, optimizers, and IR systems you can probably get very close to a system that can truly reason.
- sigg3 3y agoI agree with your overall description, but I still think reasoning is a term requiring autonomy. A calculator doesn't reason. 'Abductive reasoning' in this special ML sense is a matter of statistical calculation. My point being that we should stop implying that we're reconstructing human behavior, when what we are doing is _mimicking it_ through advanced recursion and statistics over huge amounts of inputs (big data). Abductive reasoning OTOH relies on huge imperfections: human bias, flawed logic, misremembered facts, invented facts, all of whom are culturally situated, a sense of abstract self, a biological self and always an overarching, immanent telos. Speculatively we might be inventing an entirely new category of existence whose logical categories (such as a special case 'abductive reasoning') cannot be understood in anthropomorphic terminology without distorting it. To me that's still science fiction, so far I've only seen automatons; shallow mimicking robots.
- fnordpiglet 3y agoI agree that we won’t create a human intelligence. If we create an intelligence it’ll at best be a warped “something else.” However I’ll make the trite point that I don’t think we understand what makes human intelligence manifest. There’s no reason to believe its not at its basis a statistical optimization based on reinforced gradient descent. In fact it’s fairly likely to be the case. My assertion is as we mix other AI techniques in a feedback cycle we will see something more than this simple single model exploration can do and it’ll converge to what we would consider intelligence. But I agree fundamentally it won’t map to a human intelligence modality.
- cyberbanjo 3y agoMaybe it's anthropomorphic but I like hallucinations because it reminds me of the DeepDream psychedelic generator
- AdieuToLogic 3y ago> Hallucinates is a far more accurate description. This is still a form of anthropomorphism. Every non-deterministic search algorithm (a.k.a. "AI") relies on pseudo-random number generators (PRNG's) and usually employ scoring/weighting functions to determine what the algorithm will search. Sometimes PRNG values will result in an algorithm searching less likely solution-spaces despite what a scoring function may rate. This is needed in order to avoid over-specialization and allow for edge-case discovery (amongst other benefits). Depending on the algorithm, this may be the only solution proffered.
- mountaintimefrm 3y agoBS'ing
- bmc7505 3y agoI have been making a similar argument for some time. The choice to anthropomorphize dialogue systems and large language models is a clichéd and misleading UI design choice. By presenting chatbots as disembodied persons, we are implicitly suggesting agency to an inanimate object, and subtly exploiting humans' social instincts into engaging with a system that is incapable of empathy. AI is best thought of as a programmable tool, not a conversational agent.
- fnordpiglet 3y agoExcept the illusion is strong enough to work in many cases. And as with much of computing, faking it is enough. 3d graphics has been built on that since day one and only recently have we had ray tracing and other more “realistic” to nature techniques generally available.
- IIAOPSW 3y agoWhat if rational "inanimate" thought is jerryrigged out of social reasoning to begin with. Simply take your system of social reasoning, attribute a fictional quiet / predictable persona to the inanimate, and reuse the social reasoning circuitry for other things. Linguistic expressions become so much more fluid when you don't have to maintain rules about one set of verbs for inanimate nouns and another set for the animate nouns. You can use "tells" for both the literal act of speech (Eg "she told me you plan to run away") and the logical act of inference (Eg "this situation tells me you plan to run away"). As the old cliche goes, "the facts speak for themselves". Some cultures go so far as to treat personification as more literal than just a linguistically expressive tool, believing every object to actually posses a "spirit". So why the steadfast resilience to such a natural interface as anthropomorphizing the machine?
- bmc7505 3y agoThere is a qualitative difference in both perception and intent between a figure of speech, and a machine specifically designed to impersonate a human. Although we articulate familiar concepts in anthropomorphic terms, no reasonable listener would interpret these phrases literally. The former is simply a play on words, while the latter is a subliminal attempt to mislead users into humanizing a machine.