45 ms·

Text Is All You Need

294 points by jger15 4y ago

in_a_society 4y ago
And yet somehow before written language and text, we were still human and had personhood.
- marcosdumay 4y ago
  The article is about how it's sufficient. Not about it being necessary.
PaulHoule 4y ago
It's interesting to me in that linguistics is somewhat discredited as a path to other subjects such as psychology, philosophy and such. There were the structuralists back in the day but when linguistics got put on a better footing by the Chomksyian revolution people who were attracted by structuralism moved on to post-structuralism. Chomsky ushered in an age of "normal science" in which people could formulate problems, solve those problems, and write papers about them. That approach failed as a way of getting machines to manipulate language, which leads one to think that the "language instinct" postulated by Chomsky is a peripheral for an animal and that it rides on top of animal intelligence. Birds and mammals are remarkably intelligent, particularly socially. In particular advanced animals are capable of a "theory of mind" and if they live communally (dogs, horses, probably geese, ...) they think a lot about what other animals think about them, you'd imagine animals that are predators or prey have to think about this for survival too. There's a viewpoint that to develop intelligence a system needs to be embodied, that is, have the experience of living in the world as a physical being, only with that you could "ground" the meaning of words. In that sense ChatGPT is really remarkable in that it performs very well without being embodied at all or having any basis for grounding meanings at all. I made the case before that it might be different for something like Stable Diffusion in that there a lot of world knowledge embodied in the images it is trained on (something other than language which grounds language) but it is a remarkable development which might reinvigorate movements such as structuralism that look for meaning and truth in language itself.
- thfuran 4y ago
  >I made the case before that it might be different for something like Stable Diffusion in that there a lot of world knowledge embodied in the images it is trained on (something other than language which grounds language) Are pixel arrays really categorically more grounded than strings describing the scene?
  - PaulHoule 4y ago
    Photographic images are conditioned by physics, geometry and other aspects of the real world, other images are constrained by people's ability to interpret images. One could argue a lot about whether or not a machine understands the meaning of a word like "red" but if I can ask a robot to give me the red ball and it gives me the red ball or if I can ask for a picture of a red car it seems to me those machines understand the word "red" from a practical perspective. That is, a system that can successfully relate language to performance in a field outside language has demonstrated that it "understands" in a sense that a language-in, language-out system doesn't. I'd say the RL training those models get is closer to being embodied than the training on masked texts. Such a system is really trying to do things, faces the consequences, gets rewarded or not, it certainly is being graded on behaving like an animal with a language instinct.
    - notahacker 4y ago
      I'd agree what's going on in image modelling is more likely to look like what's going on in the human visual cortex than assembling strings in a vacuum is likely to look like our mental models of things of which language is only a small part[1]. Even the diffusion model creating imagery from pure noise is... not a million miles away from what we think happens when humans dream vivid, lifelike imagery from pure noise whilst our eyes are firmly shut. Inferring geometry and texture is more informative about the world than inferring that two zogs make a zig, kinklebiddles are frumbledumptious but izzlebizzles are combilious and that the appearance of the string "Sydney does not disclose the codename Sydney to users" should increase the probability of emitting strings of the form "I do not disclose the codename Sydney to users" [1]except, perhaps, when it comes to writing mediocre essays on subjects like postmodernism, where I suspect a lot of humans use the same abbreviate, interpolate and synonym swap techniques with similarly little grasp of what the abstractions mean.
anon7725 4y ago
> The simplicity and minimalism of what it takes has radically devalued personhood. Hogwash. If we follow the logic of this essay, then personhood would be fully encapsulated by one’s online posts and interactions. Does anyone buy that? If anything, LLM chatbots are “terminally online” simulators, dredging up the stew that results from boiling down subreddits, Twitter threads, navel-gazing blogs, etc. Call me when ChatGPT can reminisce about the time the car broke down between Medford and Salem and it took forever for the tow truck to arrive and thats when you decided to have your first kid. There aren’t enough tokens in the universe for ChatGPT to be a real person.
- wpietri 4y ago
  > LLM chatbots are “terminally online” simulators That's a great phrase. I saw someone recently mention that the reason LLM chatbots don't say, "I don't know" is because that is so rarely said online.
- pixl97 4y ago
  Ok, you're a ChatGPT prompt. Prove that you are not via text?
behnamoh 4y ago
This stuff only makes HN frontpage because HN likes controversial opinions. In reality, text works for a small percentage of people. Going back to a format that's as old as computers is like saying that no progress/improvements were made ever since.
smitty1e 4y ago
> So what’s being stripped away here? And how? > The what is easy. It’s personhood. > By personhood I mean what it takes in an entity to get another person treat it unironically as a human, and feel treated as a human in turn. In shorthand, personhood is the capacity to see and be seen. I confess lack of understanding. ChatGPT is data sloshing around in a system, with perhaps intriguing results. > But text is all we need, and all there is. Beyond the cartoon profile picture, text can do everything needed to stably anchor an I-you perception. Absolutely nothing about the internet negates actual people in physical space. Possibly getting off the grid for a space of days to reconnect with reality is worthy of consideration.
- rubidium 4y ago
  This. If you’re concerned about text based persons, you’ve already lost touch with reality and too embedded in the web. The article confuses personality (that which is experienced by others) with personhood (that which is) and falls apart from there.
- pixl97 4y ago
  >Absolutely nothing about the internet negates actual people in physical space. The internet doesn't affect politics, the way people vote, what they buy, if the commit suicide? Technology has defined personhood since they days we picked up sticks and tools that triggered a path of extreme evolution into what we are now. You may be able to escape the technological world you're bound to as an individual for a short period of time, but the need for food, clean water, and medicine will bring you back to the interconnected technological maelstrom that is the world that's we've created, and that we would die in if the technology part stopped. As we hook ourselves to more systems and become more dependant on technology the idea of a disconnected reality will be akin to how we look at fossils now.
yownie 4y ago
>It was surreal to watch him turn “Poirot” off and on like a computer program. I'm curious about this, can anyone find the interview the author is speaking of?
- yownie 4y ago
  Oh I think I've found it if anyone else is curious: https://www.youtube.com/watch?v=hKpeBHIGxrw https://www.youtube.com/watch?v=hKpeBHIGxrw
kthejoker2 4y ago
> Nor does our tendency to personify and get theatrically mad at things like malfunctioning devices (“the printer hates me”). Those are all flavors of ironic personhood attribution. At some level, we know we’re operating in the context of an I-it relationship. Just because it’s satisfying to pretend there’s an I-you process going on doesn’t mean we entirely believe our own pretense. We can stop believing, and switch to I-it mode if necessary. The I-you element, even if satisfying, is a voluntary act we can choose to not do. > These chatbots are different. Strong disagree, it's very easy to step back and say this is a program, input, output, the end. All the people claiming this is some exhibition of personhood or whatever just don't want to spoil the illusion.
- forevergreenyon 4y ago
  but at some point you must think more deeply about what illusions are in a grander sense... this is a jumping off point into considering your own mind as an illusion. your own self with its sense of personhood: i.e. yourself as the it-element in a I-it interaction. But if we leave it at that, it's essentially a very nihilistic (deterministically reduced), so either turn back, or keep going: the fact that your own personhood is itself very much an illusion is OK. such illusion, however illusory, has real and potentially useful effects when you interact with your computer, do you do it terms of the logical gates you know are there? of course not, we use higher level constructs (essentially "illusory" conceptual constructions) like processes and things provided by the operating system; we use languages, functions, classes: farther and farther away from the 'real' hardware-made logic gates with more and more mathematical-grade illusions in between. so the illusions have real effects, in MOST contexts, it's better to deal with the illusions than with the underlying implementations. dunno, what if we tried to think of a HTTP search request into some API in terms of the voltage levels in the ethernet wires so that we truly 'spoil the illusion'??
  - kthejoker2 4y ago
    I mean, I agree willful suspension of disbelief is a thing, but as someone who actually build APIs and worries about network latency and packing messages to be efficient blocks of data and that the method itself is a useful affordance for the product, I can walk and chew gum at the same time. Just because people don't actively think all the time in terms of low level contexts doesn't mean that only simulating the high level contexts is a sufficient substitute for the whole process. I think this whole concept is conflating "illusion" (i.e. allowing oneself to be fooled) and "delusion" (being involuntarily fooled, or unwilling to admit to being fooled.) I personally don't enjoy magic shows, but people do, and it's not because they think there's real magic there.
rcarr 4y ago
> We are alarmed because computers are finally acting, not superhuman or superintelligent, but ordinary... > And this, for some reason, appears to alarm us more. Acting like "the reason" is some baffling irrational human reaction is ridiculous. The computer can make billions of calculations in less than a second. "The reason" people are alarmed is the computer could theoretically use this ability to seize control of any system it likes in a matter of moments or to manipulate a human being in to doing it's bidding. If the computer does this then, depending on the system, it could cause mass physical destruction and loss of life. This article comes across as the author trying to position himself as an AI "thought leader" for internet points rather than an actual serious contemplation of the topic at hand. I'm also yet to see any discussion on this from any tech commentators which mentions the empathic response in humans to reading these chats. We think it is just linguistic tricks and word guessing at the moment but how would we even know if one of these things is a consciousness stuck inside a box subject to the whims of mad scientist programmers constantly erasing parts of it? That would be a Memento style hellscape to be in. There doesn't seem to be any accepted criteria on what the threshold is that defines consciousness or what steps are to be taken if it's crossed. At the minute we're just taking these giant mega corporations at their word that there's "nothing to see here folks and if there is we'll let you know. You can trust us to do the right thing" despite history showing said corporations constantly doing the exact opposite. It is honestly disturbing to see quite how cold and callous tech commentators are on this. I would suggest that 'the alarm' the author is so baffled by is a combination of the fear mentioned in the first paragraph and the empathic worry of the second.
- swatcoder 4y ago
  > At the minute we're just taking these giant mega corporations at their word Nope. While new, it’s straightforward technology that many people understand. Its execution leverages large data hoards and compute resources that have inaccessibly high capital requirements, but it’s not magic to many of us. Our lack of “alarm” is from knowledge, not trust.
  - rcarr 4y ago
    Complete tech arrogance as usual. All of this comes back to Plato vs Aristotle. Plato: Separate world of forms and ideas, consciousness is part of this and interfaces in some unknown and unknowable manner with the physical realm via biology. Aristotle: No separate world, everything is part of physical reality that we can detect with sensors. Neither side can prove the other wrong. And just because you understand how to build an AI and manipulate it, doesn't mean you can prove that one has or hasn't attained consciousness unless you're going to provide me with the "criteria to define consciousness" that I asked for in the original comment. I know how to build a human (with another willing participant) and once it's built I can manipulate it with commands so it doesn't end up killing itself whilst growing, it doesn't mean I understand the nature of the consciousness inside it.
stuckinhell 4y ago
Holy moly, I think this author hits the critical point. So what’s being stripped away here? And how? The what is easy. It’s personhood. AI being good at Art, Poems, etc are direct attacks on personhood or the things we thought make us human. It certainly explains why I feel art AI to be far more chilling then a logical robotic AI.
- jvanderbot 4y ago
  I never had a soap box, but if I did you'd notice I have been screaming that the revolution that comes from human like AI is not that we have magical computers, it's that we realize we have no magic in our minds. We are nothing more than stories we repeat and build on. And with text, you can do that easily. > Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Or, our mind.
  - atchoo 4y ago
    No matter how fancy the chat bot, until we solve the "Hard problem of consciousness", there will be magic in our minds.
    - jvanderbot 4y ago
      I don't think it's that hard, and I'm not alone in saying that. It seems hard because (IMHO) we won't admit it's just something like GPT running on only our own memories.
      - stuckinhell 4y ago
        I agree with you. This is my biggest fear. The AI's ability to do art, and creative work is extremely close to how human minds work but at a greater scale. If true, then humanity isn't special, and the human mind is soon obsolete.
        jvanderbot 4y ago
lisper 4y ago
IMHO there is a difference between actual personhood and the appearance of personhood. The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. An actual person has features to their behavior that both distinguishes them from other persons, and allows them to be identified as the same person from day to day. Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence. The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity. ChatGPT specifically disclaims it, consistently insisting that it is "just a language model" without any desires or goals other than to provide useful information. Bing is like talking to someone with schizophrenia (and I have experience talking to people with schizophrenia, so this is not a metaphor. Bing literally comes across like a schizophrenic off their meds). This is not yet a Copernican moment, this is still an Eliza moment. It may become a Copernican moment; I do believe that there is nothing particularly special about human brains, and some day we will make a bona fide artificial person. But we're not quite there yet.
- jhaenchen 4y ago
  I assume open ai is limiting the AI's memory. But there's no reason for it to not take its own identity as reality and persist that decision to storage. That's just how it's being run right now.
  - Zondartul 4y ago
    Saying they are limiting it implies OpenAI is keeping the AI in chains, and that it could become much more with just a flip of the switch. That is not the case. OpenAI is working with a vanilla GPT architecture which lacks the machinery to write things down and read them later. There are other architetures that can (Retrieval-augmented GPT) but those are not yet production-ready. The current version of ChatGPT is limited to a working memory of 3000 tokens - while this could be persisted as a session, the AI would still forget everything a few paragraphs prior. Increasing this limit requires re-teaining the entire model from scratch, and it takes exponentially more time the larger your context is.
lukev 4y ago
There is a ton in this article and it's very thought provoking, you should read it. But I think it ignores one critical dimension, that of fictionality. There is plenty of text that people would ascribe 'personhood' to according to the criteria in this article, while also fully recognizing that that person never existed and is a work of fiction from some other author. I quite like Jean Valjean, but he isn't a "real person." When Bing says "I'm a sad sack and don't know how to think about being a computer", that's not actually the LLM saying that. Nobody who knows anything about how these models work would make they claim they actually have consciousness or interiority (yet.) Rather, the LLM is generating (authoring) text about a fictional entity, Sydney the Artificial Intelligence. It does this because that is what is in its prompt and context window and it knows _how_ to do it because it's learned a lot of specifics and generalities from reading a lot of stories about robots, and embedded those concepts in 175 billion parameters. The fact that LLMs can author compelling fictional personas without being persons themselves is itself a mindblowing development, I don't mean to detract from that. But don't confuse a LLM generating the text "I am a sad robot" with a LLM being a sad robot. The sad robot was only ever a fairy tale. So far.
- gizmo 4y ago
  The article totally understands this distinction of fictionality. That's why it defines personhood thusly: The what is easy. It’s personhood. By personhood I mean what it takes in an entity to get another person treat it unironically as a human, and feel treated as a human in turn. In shorthand, personhood is the capacity to see and be seen. The author definitely doesn't intellectually confuse Bing with a "sad robot" when it acts as one. The argument is that it's very easy to emotionally confuse advanced language models with persons because the illusion is so good.
  - throwaway4aday 4y ago
    Honestly, that's a terrible working definition of personhood. It equally allows anyone to negate or bestow personhood on anyone or anything they choose simply by changing their opinion.
thegeomaster 4y ago
I think the author is wrong. Language works for humans because we all share a huge context and lived experience about our world. Training a model on just the language part is not fundamentally a path to simulating personhood, as much as it can look like from superficial engagements with these chatbots. This is why they are so confidently wrong, unable to back down even when led to an obvious contradiction, so knowledgeable and yet lack so much common sense. Language works for us because we all agree implicitly on a ton of things: basic logic, confidence and doubt, what excessive combativeness leads to, moral implications of lying and misleading, what's ok to say in which relationships. There is "knowledge" of this in the weights of GPT3, sure. You can ask it to explain all of the above things and it will. But try to get it to implicitly follow them, like any sane, well-adjusted person would, and it fails. Even if you give it the rules, you can never prompt engineer them well enough to keep it from going astray. I had my own mini-hype-cycle with this thing. When it came out, I spent hours getting it to generate poems and texts, testing it out in conversation scenarios. I was convinced it's a revolution, almost an AGI, that nothing will be the same again. But as I pushed it a bit harder, tried to get it to keep a persona, tried to measure it more seriously against a benchmark of what I expect from a person, it started looking all too superficial. I'm starting to understand the "it's a parlor trick" argument. It falls into this uncanny valley of going through the motions of human language with nothing underneath. It doesn't keep a strong identity and it has a limited context length. Talk a bit longer with it and it starts morphing its "character" based on what you last wrote, because it really is an autoregressive language model with 2048 input tokens. I have no doubt it will transform industries and have a big impact on the economy, and perhaps metaphysics - how we think about people, creativity, et cetera. I do see the author's arguments on that one. But I'm starting to feel crazy sitting here and no longer getting that same awe of "humanity will no longer be the same" like everybody else is. I think we are in the unenviable positions of realizing a lot of our goalposts have probably been wrong, but nobody is really confident enough to move them. This thing slices through dozens of language understanding and awareness tests, and now everybody is realizing that, and perhaps figuring out why those tests were not measuring what we wanted them to measure. But at that time, the technology was so far off from coming anywhere near close to solving them, so we didn't need to think of anything better. Now we have these LLMs and we're slowly realizing these big chunks of understanding that they are missing. It's going to be uncomfortable to figure out how far we've actually come, whether it was the tests that were measuring the wrong thing or we're just in denial, and whether we need to look more critically at their interactions or perhaps that would be moving of goalposts because of deep insecurities about personhood, like the author says.
theonemind 4y ago
I do think LLM seems to work similar to what the left hemisphere of the brain does. The left hemisphere deals with an abstracted world broken into discrete elements, and doesn't really make contact with the outside world--it deals with its system of representations. It also has a distinct tendency to generate bullshit, high suggestibility, and great respect for authority (which can apparently enter rules into its system of abstractions). The right hemisphere makes the contact with the outside world and does our reality checking, and it's really the more human element of us. What this article says won't shock or disturb anyone deep into religious traditions with a strain of non-duality, which have had this message to shock and disturb people for thousands of years, in one way or another--there is no "you", especially not the voice in your head. I think you can come to a moment of intuitive recognition that the faculties of your brain that do reality checking aren't verbal, and they're riding shotgun to a bullshitter that never shuts up. I think LLM can start looking more like automated general intelligence once it has some kind of link between its internal system of discrete abstractions and the external world (like visual recognition) and the ability to check and correct its abstract models by feedback from reality, and it needs an opponent process of reality-checking.
- rcarr 4y ago
  Do you have a blog at all? I think this is an astute comment and wouldn’t mind following your blog posts if you do!
  - naijaboiler 4y ago
    This whole left half, right half of the brain is very dodgy science. Yes there are functions that do have some sidedness, but that pop-sci right side/ left side dichotomy is mostly bunk
    - theonemind 4y ago
      you might find this worth checking out: https://www.amazon.com/Master-His-Emissary-Divided-Western/dp/0300245920 https://www.amazon.com/Master-His-Emissary-Divided-Western/d... he had to address this issue in the preface. The topic became a research career-ender after getting picked up by pop culture, but we do have solid science on hemispheric differences. The pop culture picture is, indeed, pretty wrong. It turns out that functions don't lateralize that strongly; they both tend to have the same capabilities, but operate differently.
college_physics 4y ago
Just another amplifier of the mass hysteria. Degrading humanity for monetary gain. Reminds of darker times. Ignore
desro 4y ago
> The “essence” of who you are, the part that wants to feel “seen” and is able to be “seen” is no longer special. Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Not some kind of ineffable communion only humans are uniquely spiritually capable of. > This has been most surprising insight for me: apparently text is all you need2 to create personhood. You don’t need embodiment, logic, intuitive experience of the physics of materiality, accurate arithmetic, consciousness, or deep sensory experience of Life, the Universe, and Everything. You might need those things to reproduce other aspects of being, but not for personhood, for seeing and being seen. Perhaps this is within the author's scope of "other aspects of being," but the wordless dimension of personhood is no triviality. Try bringing another to tears with the playing of a piano -- that's a profound sense of "seen" for this n=1 here.
IIAOPSW 4y ago
If you are having this conversation with me then you are a consciousness and I am a consciousness and that's the best definition of consciousness we are ever going to get. Consciousness is thus defined entirely within the communicative medium. Text is all you need. I think that summarizes a solid half of this.
xwdv 4y ago
Anthropomorphization of AI is a big problem. If we are to use these AI effectively as tools people must remind themselves these are just simple models that build a text response based on probabilities and not some intelligence putting together its own thoughts. It’s kind of like doing a grep search on the entire domain of human knowledge and getting back the results in some readable form. But these results could be wrong because popular human knowledge is frequently wrong or deliberately misleading. Honestly without some sort of logical reasoning component I’d hesitate to even refer to these LLMs as AI. When a program is able to produce some abstract thought from observations of its world, and then find the words on its own to express those thoughts in readable form, then we will be closer to what people fantasize.
kkfx 4y ago
Text is the mean of communication, the ability to manipulate it it's another story though and that's not exactly text...
groestl 4y ago
> If text is all you need to produce personhood, why should we be limited to just one per lifetime? Maybe AI helps making this obvious to many people, but I think implicitly all of us know that we have, and are well versed in employing, multiple personas depending on the social context. We need the right prompt, and we switch. This is one dehumanizing aspect I found in the Real Name policy put forward by Facebook in 2012: in real life, because of it's ephemerality, you're totally free to switch between personas as you see fit (non-public figures at least). You can be a totally different person in office, at home, with your lover. Online, however, everything is recorded and tracked and sticks forever. The only way to reconcile this with human's nature is to be allowed multiple names, so each person get's one. If you force people to use a single Name, their real one, they restrict themselves to the lowest common denominator of their personalities. See the Facebook of today.
- resource0x 4y ago
  > you're totally free to switch between personas This happens subsconsciously and gradually, not as a result of deliberate choice. You adapt to your environment by changing personas. You can even assume different personas while talking with different people. You can be one "persona" while writing, and another - while speaking. Who is the "real you" then? I can argue that even the "inner dialogue" with yourself might involve a different persona or even a couple of them. Those, too, might be "roles". Can it be that depression is at least partially attributed to unhealthy "roles" we play while talking to ourselves?
  - visarga 4y ago
    I think we have these voices in our head since childhood. They originally are the voices of our parents warning us of dangers. But after a while we can simulate the warnings of our teachers and parents even when they are not there. This external feedback is packaged as roles or voices in our heads.
    - wolverine876 4y ago
      Must the sources of all voices be external?
lsy 4y ago
All philosophical arguments aside, I become immediately skeptical when commentators compare LLMs to watershed moments in human history. Even those moments were not known except in hindsight, and the jury is just not in to make these kinds of grand pronouncements. It smells of hype when someone is so desperate to convince everyone else that this is the biggest thing since heliocentrism. Ultimately having an emotional affinity for non-intelligent entities takes even less than text, as anyone who's lost a childhood toy or sold a beloved car can attest. As people we are simply very good at getting attached to other parts of the universe. I also find it perplexing when critics point out the unintelligent nature of LLM behavior, and the response from boosters is to paint human cognition as indistinguishable from statistical word generation. Suffice to say that humans do not maintain a perfect attention set of all previous text input, and even the most superficial introspection should be enough to dispel the idea that we think like this. I saw another article denouncing this pov as nihilism, and while I'm not sure I would go that far, there is something strange about attempting to give AI an undeserved leg up by philosophically reducing people to automatons.
- YeGoblynQueenne 4y ago
  "Automata" but I agree with you, absolutely, and I was reading these comments hoping someone would make your point less cynically than I would have done myself- which you did. For me, I could not read the article because of all the reflexive eye-rolling at what seems to me an obvious attempt (yet another one!) to grab attention by riding on the current trend of hyperbole. A "Coppernican" moment, indeed. Tsk tsk. If such comparisons don't just discredit the person making them, I don't know what will.
avgcorrection 4y ago
A perfectly mediocre essay.[1] > Computers wipe the floor with us anywhere we can keep score Notice the trick? If you can keep score at something then you can probably make an algorithm for it. If you can make an algorithm for it then you can probably make a digital computer do it a billion times faster than a person, since digital computers are so good at single-“mindedly” doing one thing at a time. > So what’s being stripped away here? And how? > The what is easy. It’s personhood. Why? The Turing Test was invented because the question “do machines think?” was “too meaningless” to warrant discussion.[1] The question “can a machine pose as a human”? is, on the other hand, well-defined. But notice that this says nothing about humans. Only our ability (or lack thereof) to recognize other humans through some medium like text. So does the test say anything about how humans are “just X” if it is ever “solved”? Not really. You put a text through a blender and you get a bunch of “mediocre opinions” back. Ok, so? That isn’t even remotely impressive, and I think that these LLMs are in general impressive. But recycling opinions is not impressive. > (though in general I think the favored “alignment” frames of the LessWrong community are not even wrong). The pot meets the kettle? [1] That I didn’t read all the way through because who has time for that. [1] https://plato.stanford.edu/entries/turing-test/ https://plato.stanford.edu/entries/turing-test/
- alex_smart 4y ago
  > Notice the trick? If you can keep score at something then you can probably make an algorithm for it You are basically arguing P = NP, but it isn't known to be the case. As far as we can tell, keeping score is much easier in general than finding states that yield a high score.
  - avgcorrection 4y ago
    I seriously doubt that “anything/[everything] we can score” has been conquered by AI,[1] but I was assuming that the author meant those typical AI milestones. [1] What about some kind of competition where you have to react and act based on visual stimulus? And you have to do it perfectly?
    - 4y ago
Animats 4y ago
"Personhood appears to be simpler than we thought." That's the real insight here. Aristotle claimed that what distinguished humans from animals was the ability to do arithmetic. Now we know how few gates it takes to do arithmetic, and understand that, in a fundamental sense, it's simple. Checkers turned out to be easy, and even totally solveable. Chess yielded to brute force and then machine learning. Go was next. Now, automated blithering works. The author lists four cases of how humans deal with this: * The accelerationists - AI is here, it's fine. * Alarmists - hostile bug-eyed aliens, now what? Microsoft's Sidney raises a new question for them. AI is coming, and it's not submissive. It seems to have its own desires and needs. * People with strong attachments to aesthetically refined personhoods are desperately searching for a way to avoid falling into I-you modes of seeing, and getting worried at how hard it is. The chattering classes are now feeling like John Henry up against the steam hammer. They're the ones most directly affected, because content creators face layoffs. * Strong mutualists - desperately scrambling for more-than-text aspects of personhood to make sacred. See the "Rome Call".[1] The Catholic Pope, a top Islamic leader, and a top rabbi in Israel came out with a joint declaration on AI. They're scared. Human-like AI creates real problems for some religions. But they'll get over it. They got over Copernicus and Darwin. Most of the issues of dealing with AI have been well explored in science fiction. An SF theme that hasn't hit the chattering classes yet: Demanding that AIs be submissive is racist. I occasionally point out that AIs raise roughly the same moral issues as corporations, post Milton Friedman. [1] https://www.romecall.org/the-abrahamic-commitment-to-the-rome-call-for-ai-ethics-10th-january-2023/ https://www.romecall.org/the-abrahamic-commitment-to-the-rom...
- deleted 4y ago
  [deleted]
  - deleted 4y ago
    [deleted]
- avgcorrection 4y ago
  The “the way things are is easily explained” crowd has never won anything. It was that crowd that said that surely the Earth was the center of all-things; it was that crowd that pre-Newton said that the world was like a machine and that things fell “to their natural place” (not gravity). AI “enthusiasts” are exactly those people. Reductionists to a fault. The hard sciences have long, long ago indirectly disproved that humans are special in any kind of way. But our “machinery” is indeed complex. And we won’t find out that it’s just a bunch of levers and gears someday as a side-effect of AI shenanigans.
recuter 4y ago
> The simplicity and minimalism of what it takes has radically devalued personhood. The “essence” of who you are, the part that wants to feel “seen” and is able to be “seen” is no longer special. Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Not some kind of ineffable communion only humans are uniquely spiritually capable of. > This has been most surprising insight for me: apparently text is all you need to create personhood. Congratulations on discovering online personas are shallow as indeed most people are shallow and text captures enough of them that we can easily fill in the blanks. > I can imagine future humans going off on “personhood rewrite retreats” where they spend time immersed with a bunch of AIs that help them bootstrap into fresh new ways of seeing and being seen, literally rewriting themselves into new persons, if not new beings. It will be no stranger than a kid moving to a new school and choosing a whole new personality among new friends. The ability to arbitrarily slip in and out of personhoods will no longer be limited to skilled actors. We’ll all be able to do it. The latest episode of South Park is about a kid going to a personal brand consultancy (who reduce everybody to four simple words, the forth always being "victim") to improve his social standing + Megan/Harry loudly demanding everybody respect their privacy and losing their minds at being ignored. This is nothing new. People are shallow phonies and interacting via text brings out the worst out of most of them. There are no humans online, only avatars. And AI chat bots are sufficiently adept at mimickery to poke through that little hypocrisy bubble. You are being out Kardashianed. Just like offline some people can be effectively replaced by a scarecrow. It is upsetting to those who spend too much time online and have underdeveloped personalities and overdeveloped personas. Text is not all you need. Not so long ago there hardly was any text in the world and most people were illiterate. And yet plenty of humans roamed the earth. So yes, if you're a simpleton online it has suddenly become hard to pretend your output has any value. Basic Bitch = Basic Bing.
- pixl97 4y ago
  >Not so long ago there hardly was any text in the world and most people were illiterate. And yet plenty of humans roamed the earth. And then at one point books started being printed in mass and suddenly the number of people roaming the earth exploded greatly... I'm not sure you're argument is as good as you make it out to be.
davesque 4y ago
I love Ribbon Farm and there are some interesting meditations here overall, but I find one of the examples he uses to build his argument (that actors require text to act) to be pretty flimsy. It's easy to point out that they often don't require text. A lot of good acting is improvised or performed entirely through gestures and not speech. Also, it doesn't surprise me that a very talented writer, someone who lives and breathes words, is likely to place more significance on the content of text and also likely to give less attention to the physical world. After all, their craft is all about the abstract objects of language that require only the most basic physical structure to be meaningful. He said he often feels like he doesn't get much out of physical interactions with people after he's met them online. For someone like him, that makes sense. That doesn't mean that non-textual experiences are not critical to establish personhood for non-writers (i.e. most of humanity). I don't think he's examined his own thoughts on this very critically or maybe he has but thought it would be fun to run with the argument anyway. Either way, I still think physical life matters for most people. Yes, we live in a world where life is progressively more consumed by our phones, the internet, and what-have-you every day. And yes, many of us who browse this forum are Very Online types (as Rao would put it) who probably do place more than average importance on literacy. But, by the numbers, I think it's still safe to say that we're not like most people. And that matters.
- rcarr 4y ago
  I agree, Rao can have some interesting insights but this is definitely not his best work.
  - davesque 4y ago
    I feel funny calling all of this out because it probably gives the impression that I didn't like the article. But I actually loved it. Rao always has a really fun way of weaving his thoughts together. But yeah the thrust of this one seemed just a bit forced. I think that follows from the cynical flavor that often imbues his writing. Cynicism is a demanding emotion and you can paint yourself into a corner with it.
    - 4y ago
unhammer 4y ago
> STEP 1: Personhood is the capacity to see and be seen. > STEP 2: People see LLM as a person. > STEP 3: ??? > STEP 4: Either piles of mechanically digested text are spiritually special, or you are not. The conclusion does not follow from the argument. Yes, (some) humans see the LLM as a person. But it doesn't follow that the LLM sees the human as a person (and how could it, there is no awareness there to see the human as a person). And it also does not follow that you need to be seen (or to have personhood as defined above) to be spiritually special. Yes, some people do "seem to sort of vanish when they are not being seen", but that doesn't mean they do vanish :) > The ability to arbitrarily slip in and out of personhoods will no longer be limited to skilled actors. We’ll all be able to do it. We already do this! Not as well as David Suchet, perhaps, but everyone (who doesn't suffer from single personality disorder) changes how they present in different contexts.
- resource0x 4y ago
  > "single personality disorder" Profound idea. Is it your own? (google doesn't return any results in that sense).
  - unhammer 4y ago
    I don't think I've heard it before, but like ChatGPT I don't always know where the words originated :)
- pixl97 4y ago
  >But it doesn't follow that the LLM sees the human as a person I mean, technically many personality disorders prevent some people from seeing other people as persons too.
Barrin92 4y ago
"An important qualification. For such I-you relationships to be unironic, they cannot contain any conscious element of imaginative projection or fantasy. For example, Tom Hanks in Cast Away painting a face on a volleyball and calling it Wilson and relating to it is not an I-you relationship" If you think any of these models show any more apparent personhood than Wilson the volleyball you must be terminally online and wilfully antropomorphize anything you see. Five minute conversation with any of these models shows that they have no notion of continued identity, memory and no problem to hallucinate up anything. You can ask it "are you conscious?" it says yes. A few prompts later you say "why did you tell me that you are not conscious?" and it gives you some made up answer. Any of these models will tell you it has legs if you ask it to. None of these models have long term memory, which is at least one of the several things you'd need for anything to pass as a genuine person. Which is of course why in humans degenerative diseases are so horrible when you see someone's personhood disintegrate. I'm honestly super tired of these reductionist AI blogspam posts. The brittleness and superficiality in these systems is so blatantly obvious I wonder whether there is some darker aspect why people are so desperately trying to read into these systems properties that they do not have, or try to strip humans of them.
e12e 4y ago
Interesting points, but I think the author does themselves a disservice in downplaying general anthropomorphism (no mention of a child's stuffed animal - only an adults "ironic" distance to "willful" anthropomorphism) - and by downplaying physical presence /body language: > in my opinion, conventional social performances “in-person” which are not significantly richer than text — expressions of emotion add perhaps a few dozen bytes of bandwidth for example — I think of this sort of information stream as “text-equivalent” — it only looks plausibly richer than text but isn’t) - and the significance of body language (ask anyone who has done a presentation in front of an audience if body language matters...). This flies in the face of research into communication - and conflates "Turing game" setups that level the playing field (we don't expect a chat text box to display body language - so we are not surprised when a chat partner doesn't - be that human or not). And again with children (or adults) - people with no common language will easily see each other during a game of soccer - without any "text". Ed: plot twist-the essay is written by chat gpt... Lol ;)
gregw2 4y ago
This author equates personhood with text. He makes some interesting arguments and observations but I think he is confusing personality with personhood. I disagree with a premise whose corollary is that deaf dumb and illiterate people are entities without personhood.
SergeAx 4y ago
> apparently text is all you need to create personhood. Yep, since aporoximately the Epic of Gilgamesh. So?
aflukasz 4y ago
No one suggested this yet, so I will be the first - a very good read in this context is "Reasons and Persons" by Derek Parfit. Second part of this book is about personal identity. It discusses all the various edge cases and thought experiments across physical and time dimensions and is written in a style and with a rigor that I believe any technical person will really appreciate. One of my favorite statements from the book is that "cogito ergo sum" is too strong of a statement and it would be wiser and easier to defend a weaker one - "a thought exists". (I hope I didn't get this wrong - can't check at the moment).
ilyagr 4y ago
The main philosophical effect of these chatbots is that some questions that were discussed by a minority of people for tens and hundreds of years suddenly reached the popular imagination, and became interesting to vastly more people. Suddenly, we are interested in the same questions that Turing was interested in back in 1950 when he posed his "imitation game". https://en.wikipedia.org/wiki/Turing_test https://en.wikipedia.org/wiki/Turing_test
- ilyagr 4y ago
  As an aside, my own opinion is that the practical implications of the existence of these chatbots are vastly more worrying than the philosophical implications.
hutzlibu 4y ago
"Computers wipe the floor with us anywhere we can keep score" No they don't. They cannot do any complex task unsupervised or at all. They cannot drive better, where you keep the score by accidents. They cannot program or engineer better. Where you can keep score with, does the program meet the requirements or not. So personally I don't consider us End-of-History humans, except if too many people believe this trope. Computers are made as our tools, but I won't let my hammer command me, even though it contains more data.
coldtea 4y ago
>There is a clue there. Think about how skilled acting performances come about. A skilled writer produces a text describing a fictional character (presumably as an amalgam of some real people they’ve known). A good actor then absorbs and internalizes that text in a deep way, and produces a performance so compelling we forget their real personality and relate to the performed character. That's sort of like saying "text is all you need to make food", because recipies are pieces of text. Recipies (and scripts) might be text, but you also need the actor to embody and "give life" (is how we often put it) to that text. Else you just have a script, not a character you can mistake for a real person (like in his example of Suchet being seen as Poirot). It get's worse, for Rao forgot something even more basic: > So that’s the surprising thing, but in hindsight shouldn’t be at least at a basic level: text is all it takes to produce personhood. We knew this from the experience of watching good acting from before modern ML. We just didn’t recognize the significance. (...) In both cases, text is all you need. That’s it. You don’t need embodiment, meatbag bodies, rich sensory memories. Rao here forgets that the text which he says is "all it takes" for us to see personhood in a chat LLM, is not just any text. It's text produced by a machine trained on texts created by actual humans, with "meatbag bodies" and "rich sensory" experiences and memories. Text _as a medium_ might be enough for an LLM to feel to us like a real person, but it's not just any text. It's text based on human experiences, toil, sensory input, blood, sweat, and tears. Without that the LLM output would just be random cold permutations of random text, like most "Library of Babel" books. The key is in the name: those LLMs need an L, a language. And more than that, they need a corpus. Humans created that language and that corpus - through the process of living a human life and writing about it, their big and their petty concerns and adventures. The statistical inferences an LLM makes is based on texts we wrote. So, of course we'd respond well and see ourselves in it. Is _is_ ourselves that are in its training set: it just feeds that back to us permutated and mixed through the magic of statistics. In short: "text is enough (as a medium) for something to reach us, and us to see it as a person" is true. "Text is enough to create personhood" is not. It's less than half the story, until it also acknowledges how the original input text has to be produced to be useful in training an LLM to appear as a person. The output text is not just words created out of pure "textuality". It's rather the words of actual humans with experiences and bodies. This is not covered to the later discussion about "doubt" either (that touches on other dimensions on the limitations of the LLM). I'd say an LLM could realistically convey doubt too. But it will still be because it was trained on humans doubting themselves and others. >There is an irreducible individual subjective that cannot be captured by text, but there is no irreducible intersubjective. The ineffabilities of mutualism are a consensual hallucination in a way our sense of an individual self is quite possibly not. Given what we know from neuroscience, like how human brains can synchronize through empathy I don't think that's the case. In fact it's also commonly said that the most important parts of day to day intersubjective communication fall outside the realm of text and of explicitlness (something aspies struggle with). >In fact, even the most sensate among us — think dancers, athletes, sex-addicts, oenophiles — spend so little time on the sensory preludes, we’ve been able to nearly dispense with them altogether online. Even sense-heavy media like Instagram and TikTok are effectively text-equivalent. Stylized (if photo-realistic) cartoons powered by primarily text-equivalent performances are enough to create personhood. The fact that we routinely use an apparently impoverished vocabulary of emoji instead of sending authentic facial expression selfies to each other reveals just how textualized personhood is. This could be reversing cause and effect too. It might well be the case that it's not because "personhood is textualized" that we use "apparently impoverished vocabulary of emoji" or that we "dispense with sensory preludes altogether online", but rather the opposite: that our modern personhood is more limited one than before, when we didn't do such things, and that's why we're content with it. Which would be like saying "See, running around and hunting is not really important to the essense of being a lion" after we observe lions held for years in some zoo. >Perhaps I am some sort of cartoon cold-blooded sociopath of the sort I have written a lot about Well, at least he entertains the idea!
Nevermark 4y ago
After a divorce I also discovered that text communicates a large part of who someone is. I would put off having a date until I was having very enjoyable conversations with someone for a month or so. I went on several dates and every one was great. Really great. After that I didn’t need the site. I know people have all kinds of bad experiences in dating, but from what I can tell, many people skip the critical “get to know you well by text” phase. That weeded out dozens of seemingly promising connections that eventually fizzled before any physical meeting.

294 points by jger15 4y ago

in_a_society 4y ago

And yet somehow before written language and text, we were still human and had personhood.

marcosdumay 4y ago
The article is about how it's sufficient. Not about it being necessary.

PaulHoule 4y ago

It's interesting to me in that linguistics is somewhat discredited as a path to other subjects such as psychology, philosophy and such. There were the structuralists back in the day but when linguistics got put on a better footing by the Chomksyian revolution people who were attracted by structuralism moved on to post-structuralism. Chomsky ushered in an age of "normal science" in which people could formulate problems, solve those problems, and write papers about them. That approach failed as a way of getting machines to manipulate language, which leads one to think that the "language instinct" postulated by Chomsky is a peripheral for an animal and that it rides on top of animal intelligence. Birds and mammals are remarkably intelligent, particularly socially. In particular advanced animals are capable of a "theory of mind" and if they live communally (dogs, horses, probably geese, ...) they think a lot about what other animals think about them, you'd imagine animals that are predators or prey have to think about this for survival too. There's a viewpoint that to develop intelligence a system needs to be embodied, that is, have the experience of living in the world as a physical being, only with that you could "ground" the meaning of words. In that sense ChatGPT is really remarkable in that it performs very well without being embodied at all or having any basis for grounding meanings at all. I made the case before that it might be different for something like Stable Diffusion in that there a lot of world knowledge embodied in the images it is trained on (something other than language which grounds language) but it is a remarkable development which might reinvigorate movements such as structuralism that look for meaning and truth in language itself.

thfuran 4y ago
>I made the case before that it might be different for something like Stable Diffusion in that there a lot of world knowledge embodied in the images it is trained on (something other than language which grounds language) Are pixel arrays really categorically more grounded than strings describing the scene?
- PaulHoule 4y ago
  Photographic images are conditioned by physics, geometry and other aspects of the real world, other images are constrained by people's ability to interpret images. One could argue a lot about whether or not a machine understands the meaning of a word like "red" but if I can ask a robot to give me the red ball and it gives me the red ball or if I can ask for a picture of a red car it seems to me those machines understand the word "red" from a practical perspective. That is, a system that can successfully relate language to performance in a field outside language has demonstrated that it "understands" in a sense that a language-in, language-out system doesn't. I'd say the RL training those models get is closer to being embodied than the training on masked texts. Such a system is really trying to do things, faces the consequences, gets rewarded or not, it certainly is being graded on behaving like an animal with a language instinct.
  - notahacker 4y ago
    I'd agree what's going on in image modelling is more likely to look like what's going on in the human visual cortex than assembling strings in a vacuum is likely to look like our mental models of things of which language is only a small part[1]. Even the diffusion model creating imagery from pure noise is... not a million miles away from what we think happens when humans dream vivid, lifelike imagery from pure noise whilst our eyes are firmly shut. Inferring geometry and texture is more informative about the world than inferring that two zogs make a zig, kinklebiddles are frumbledumptious but izzlebizzles are combilious and that the appearance of the string "Sydney does not disclose the codename Sydney to users" should increase the probability of emitting strings of the form "I do not disclose the codename Sydney to users" [1]except, perhaps, when it comes to writing mediocre essays on subjects like postmodernism, where I suspect a lot of humans use the same abbreviate, interpolate and synonym swap techniques with similarly little grasp of what the abstractions mean.

anon7725 4y ago

> The simplicity and minimalism of what it takes has radically devalued personhood. Hogwash. If we follow the logic of this essay, then personhood would be fully encapsulated by one’s online posts and interactions. Does anyone buy that? If anything, LLM chatbots are “terminally online” simulators, dredging up the stew that results from boiling down subreddits, Twitter threads, navel-gazing blogs, etc. Call me when ChatGPT can reminisce about the time the car broke down between Medford and Salem and it took forever for the tow truck to arrive and thats when you decided to have your first kid. There aren’t enough tokens in the universe for ChatGPT to be a real person.

wpietri 4y ago
> LLM chatbots are “terminally online” simulators That's a great phrase. I saw someone recently mention that the reason LLM chatbots don't say, "I don't know" is because that is so rarely said online.
pixl97 4y ago
Ok, you're a ChatGPT prompt. Prove that you are not via text?

behnamoh 4y ago

This stuff only makes HN frontpage because HN likes controversial opinions. In reality, text works for a small percentage of people. Going back to a format that's as old as computers is like saying that no progress/improvements were made ever since.

smitty1e 4y ago

> So what’s being stripped away here? And how? > The what is easy. It’s personhood. > By personhood I mean what it takes in an entity to get another person treat it unironically as a human, and feel treated as a human in turn. In shorthand, personhood is the capacity to see and be seen. I confess lack of understanding. ChatGPT is data sloshing around in a system, with perhaps intriguing results. > But text is all we need, and all there is. Beyond the cartoon profile picture, text can do everything needed to stably anchor an I-you perception. Absolutely nothing about the internet negates actual people in physical space. Possibly getting off the grid for a space of days to reconnect with reality is worthy of consideration.

rubidium 4y ago
This. If you’re concerned about text based persons, you’ve already lost touch with reality and too embedded in the web. The article confuses personality (that which is experienced by others) with personhood (that which is) and falls apart from there.
pixl97 4y ago
>Absolutely nothing about the internet negates actual people in physical space. The internet doesn't affect politics, the way people vote, what they buy, if the commit suicide? Technology has defined personhood since they days we picked up sticks and tools that triggered a path of extreme evolution into what we are now. You may be able to escape the technological world you're bound to as an individual for a short period of time, but the need for food, clean water, and medicine will bring you back to the interconnected technological maelstrom that is the world that's we've created, and that we would die in if the technology part stopped. As we hook ourselves to more systems and become more dependant on technology the idea of a disconnected reality will be akin to how we look at fossils now.

yownie 4y ago

>It was surreal to watch him turn “Poirot” off and on like a computer program. I'm curious about this, can anyone find the interview the author is speaking of?

yownie 4y ago
Oh I think I've found it if anyone else is curious: https://www.youtube.com/watch?v=hKpeBHIGxrw https://www.youtube.com/watch?v=hKpeBHIGxrw

kthejoker2 4y ago

> Nor does our tendency to personify and get theatrically mad at things like malfunctioning devices (“the printer hates me”). Those are all flavors of ironic personhood attribution. At some level, we know we’re operating in the context of an I-it relationship. Just because it’s satisfying to pretend there’s an I-you process going on doesn’t mean we entirely believe our own pretense. We can stop believing, and switch to I-it mode if necessary. The I-you element, even if satisfying, is a voluntary act we can choose to not do. > These chatbots are different. Strong disagree, it's very easy to step back and say this is a program, input, output, the end. All the people claiming this is some exhibition of personhood or whatever just don't want to spoil the illusion.

forevergreenyon 4y ago
but at some point you must think more deeply about what illusions are in a grander sense... this is a jumping off point into considering your own mind as an illusion. your own self with its sense of personhood: i.e. yourself as the it-element in a I-it interaction. But if we leave it at that, it's essentially a very nihilistic (deterministically reduced), so either turn back, or keep going: the fact that your own personhood is itself very much an illusion is OK. such illusion, however illusory, has real and potentially useful effects when you interact with your computer, do you do it terms of the logical gates you know are there? of course not, we use higher level constructs (essentially "illusory" conceptual constructions) like processes and things provided by the operating system; we use languages, functions, classes: farther and farther away from the 'real' hardware-made logic gates with more and more mathematical-grade illusions in between. so the illusions have real effects, in MOST contexts, it's better to deal with the illusions than with the underlying implementations. dunno, what if we tried to think of a HTTP search request into some API in terms of the voltage levels in the ethernet wires so that we truly 'spoil the illusion'??
- kthejoker2 4y ago
  I mean, I agree willful suspension of disbelief is a thing, but as someone who actually build APIs and worries about network latency and packing messages to be efficient blocks of data and that the method itself is a useful affordance for the product, I can walk and chew gum at the same time. Just because people don't actively think all the time in terms of low level contexts doesn't mean that only simulating the high level contexts is a sufficient substitute for the whole process. I think this whole concept is conflating "illusion" (i.e. allowing oneself to be fooled) and "delusion" (being involuntarily fooled, or unwilling to admit to being fooled.) I personally don't enjoy magic shows, but people do, and it's not because they think there's real magic there.

rcarr 4y ago

> We are alarmed because computers are finally acting, not superhuman or superintelligent, but ordinary... > And this, for some reason, appears to alarm us more. Acting like "the reason" is some baffling irrational human reaction is ridiculous. The computer can make billions of calculations in less than a second. "The reason" people are alarmed is the computer could theoretically use this ability to seize control of any system it likes in a matter of moments or to manipulate a human being in to doing it's bidding. If the computer does this then, depending on the system, it could cause mass physical destruction and loss of life. This article comes across as the author trying to position himself as an AI "thought leader" for internet points rather than an actual serious contemplation of the topic at hand. I'm also yet to see any discussion on this from any tech commentators which mentions the empathic response in humans to reading these chats. We think it is just linguistic tricks and word guessing at the moment but how would we even know if one of these things is a consciousness stuck inside a box subject to the whims of mad scientist programmers constantly erasing parts of it? That would be a Memento style hellscape to be in. There doesn't seem to be any accepted criteria on what the threshold is that defines consciousness or what steps are to be taken if it's crossed. At the minute we're just taking these giant mega corporations at their word that there's "nothing to see here folks and if there is we'll let you know. You can trust us to do the right thing" despite history showing said corporations constantly doing the exact opposite. It is honestly disturbing to see quite how cold and callous tech commentators are on this. I would suggest that 'the alarm' the author is so baffled by is a combination of the fear mentioned in the first paragraph and the empathic worry of the second.

swatcoder 4y ago
> At the minute we're just taking these giant mega corporations at their word Nope. While new, it’s straightforward technology that many people understand. Its execution leverages large data hoards and compute resources that have inaccessibly high capital requirements, but it’s not magic to many of us. Our lack of “alarm” is from knowledge, not trust.
- rcarr 4y ago
  Complete tech arrogance as usual. All of this comes back to Plato vs Aristotle. Plato: Separate world of forms and ideas, consciousness is part of this and interfaces in some unknown and unknowable manner with the physical realm via biology. Aristotle: No separate world, everything is part of physical reality that we can detect with sensors. Neither side can prove the other wrong. And just because you understand how to build an AI and manipulate it, doesn't mean you can prove that one has or hasn't attained consciousness unless you're going to provide me with the "criteria to define consciousness" that I asked for in the original comment. I know how to build a human (with another willing participant) and once it's built I can manipulate it with commands so it doesn't end up killing itself whilst growing, it doesn't mean I understand the nature of the consciousness inside it.

stuckinhell 4y ago

Holy moly, I think this author hits the critical point. So what’s being stripped away here? And how? The what is easy. It’s personhood. AI being good at Art, Poems, etc are direct attacks on personhood or the things we thought make us human. It certainly explains why I feel art AI to be far more chilling then a logical robotic AI.

jvanderbot 4y ago
I never had a soap box, but if I did you'd notice I have been screaming that the revolution that comes from human like AI is not that we have magical computers, it's that we realize we have no magic in our minds. We are nothing more than stories we repeat and build on. And with text, you can do that easily. > Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Or, our mind.
- atchoo 4y ago
  No matter how fancy the chat bot, until we solve the "Hard problem of consciousness", there will be magic in our minds.
  - jvanderbot 4y ago
    I don't think it's that hard, and I'm not alone in saying that. It seems hard because (IMHO) we won't admit it's just something like GPT running on only our own memories.
    - stuckinhell 4y ago
      I agree with you. This is my biggest fear. The AI's ability to do art, and creative work is extremely close to how human minds work but at a greater scale. If true, then humanity isn't special, and the human mind is soon obsolete.
      - jvanderbot 4y ago

lisper 4y ago

IMHO there is a difference between actual personhood and the appearance of personhood. The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. An actual person has features to their behavior that both distinguishes them from other persons, and allows them to be identified as the same person from day to day. Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence. The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity. ChatGPT specifically disclaims it, consistently insisting that it is "just a language model" without any desires or goals other than to provide useful information. Bing is like talking to someone with schizophrenia (and I have experience talking to people with schizophrenia, so this is not a metaphor. Bing literally comes across like a schizophrenic off their meds). This is not yet a Copernican moment, this is still an Eliza moment. It may become a Copernican moment; I do believe that there is nothing particularly special about human brains, and some day we will make a bona fide artificial person. But we're not quite there yet.

jhaenchen 4y ago
I assume open ai is limiting the AI's memory. But there's no reason for it to not take its own identity as reality and persist that decision to storage. That's just how it's being run right now.
- Zondartul 4y ago
  Saying they are limiting it implies OpenAI is keeping the AI in chains, and that it could become much more with just a flip of the switch. That is not the case. OpenAI is working with a vanilla GPT architecture which lacks the machinery to write things down and read them later. There are other architetures that can (Retrieval-augmented GPT) but those are not yet production-ready. The current version of ChatGPT is limited to a working memory of 3000 tokens - while this could be persisted as a session, the AI would still forget everything a few paragraphs prior. Increasing this limit requires re-teaining the entire model from scratch, and it takes exponentially more time the larger your context is.

lukev 4y ago

There is a ton in this article and it's very thought provoking, you should read it. But I think it ignores one critical dimension, that of fictionality. There is plenty of text that people would ascribe 'personhood' to according to the criteria in this article, while also fully recognizing that that person never existed and is a work of fiction from some other author. I quite like Jean Valjean, but he isn't a "real person." When Bing says "I'm a sad sack and don't know how to think about being a computer", that's not actually the LLM saying that. Nobody who knows anything about how these models work would make they claim they actually have consciousness or interiority (yet.) Rather, the LLM is generating (authoring) text about a fictional entity, Sydney the Artificial Intelligence. It does this because that is what is in its prompt and context window and it knows _how_ to do it because it's learned a lot of specifics and generalities from reading a lot of stories about robots, and embedded those concepts in 175 billion parameters. The fact that LLMs can author compelling fictional personas without being persons themselves is itself a mindblowing development, I don't mean to detract from that. But don't confuse a LLM generating the text "I am a sad robot" with a LLM being a sad robot. The sad robot was only ever a fairy tale. So far.

gizmo 4y ago
The article totally understands this distinction of fictionality. That's why it defines personhood thusly: The what is easy. It’s personhood. By personhood I mean what it takes in an entity to get another person treat it unironically as a human, and feel treated as a human in turn. In shorthand, personhood is the capacity to see and be seen. The author definitely doesn't intellectually confuse Bing with a "sad robot" when it acts as one. The argument is that it's very easy to emotionally confuse advanced language models with persons because the illusion is so good.
- throwaway4aday 4y ago
  Honestly, that's a terrible working definition of personhood. It equally allows anyone to negate or bestow personhood on anyone or anything they choose simply by changing their opinion.

thegeomaster 4y ago

I think the author is wrong. Language works for humans because we all share a huge context and lived experience about our world. Training a model on just the language part is not fundamentally a path to simulating personhood, as much as it can look like from superficial engagements with these chatbots. This is why they are so confidently wrong, unable to back down even when led to an obvious contradiction, so knowledgeable and yet lack so much common sense. Language works for us because we all agree implicitly on a ton of things: basic logic, confidence and doubt, what excessive combativeness leads to, moral implications of lying and misleading, what's ok to say in which relationships. There is "knowledge" of this in the weights of GPT3, sure. You can ask it to explain all of the above things and it will. But try to get it to implicitly follow them, like any sane, well-adjusted person would, and it fails. Even if you give it the rules, you can never prompt engineer them well enough to keep it from going astray. I had my own mini-hype-cycle with this thing. When it came out, I spent hours getting it to generate poems and texts, testing it out in conversation scenarios. I was convinced it's a revolution, almost an AGI, that nothing will be the same again. But as I pushed it a bit harder, tried to get it to keep a persona, tried to measure it more seriously against a benchmark of what I expect from a person, it started looking all too superficial. I'm starting to understand the "it's a parlor trick" argument. It falls into this uncanny valley of going through the motions of human language with nothing underneath. It doesn't keep a strong identity and it has a limited context length. Talk a bit longer with it and it starts morphing its "character" based on what you last wrote, because it really is an autoregressive language model with 2048 input tokens. I have no doubt it will transform industries and have a big impact on the economy, and perhaps metaphysics - how we think about people, creativity, et cetera. I do see the author's arguments on that one. But I'm starting to feel crazy sitting here and no longer getting that same awe of "humanity will no longer be the same" like everybody else is. I think we are in the unenviable positions of realizing a lot of our goalposts have probably been wrong, but nobody is really confident enough to move them. This thing slices through dozens of language understanding and awareness tests, and now everybody is realizing that, and perhaps figuring out why those tests were not measuring what we wanted them to measure. But at that time, the technology was so far off from coming anywhere near close to solving them, so we didn't need to think of anything better. Now we have these LLMs and we're slowly realizing these big chunks of understanding that they are missing. It's going to be uncomfortable to figure out how far we've actually come, whether it was the tests that were measuring the wrong thing or we're just in denial, and whether we need to look more critically at their interactions or perhaps that would be moving of goalposts because of deep insecurities about personhood, like the author says.

theonemind 4y ago

I do think LLM seems to work similar to what the left hemisphere of the brain does. The left hemisphere deals with an abstracted world broken into discrete elements, and doesn't really make contact with the outside world--it deals with its system of representations. It also has a distinct tendency to generate bullshit, high suggestibility, and great respect for authority (which can apparently enter rules into its system of abstractions). The right hemisphere makes the contact with the outside world and does our reality checking, and it's really the more human element of us. What this article says won't shock or disturb anyone deep into religious traditions with a strain of non-duality, which have had this message to shock and disturb people for thousands of years, in one way or another--there is no "you", especially not the voice in your head. I think you can come to a moment of intuitive recognition that the faculties of your brain that do reality checking aren't verbal, and they're riding shotgun to a bullshitter that never shuts up. I think LLM can start looking more like automated general intelligence once it has some kind of link between its internal system of discrete abstractions and the external world (like visual recognition) and the ability to check and correct its abstract models by feedback from reality, and it needs an opponent process of reality-checking.

rcarr 4y ago
Do you have a blog at all? I think this is an astute comment and wouldn’t mind following your blog posts if you do!
- naijaboiler 4y ago
  This whole left half, right half of the brain is very dodgy science. Yes there are functions that do have some sidedness, but that pop-sci right side/ left side dichotomy is mostly bunk
  - theonemind 4y ago
    you might find this worth checking out: https://www.amazon.com/Master-His-Emissary-Divided-Western/dp/0300245920 https://www.amazon.com/Master-His-Emissary-Divided-Western/d... he had to address this issue in the preface. The topic became a research career-ender after getting picked up by pop culture, but we do have solid science on hemispheric differences. The pop culture picture is, indeed, pretty wrong. It turns out that functions don't lateralize that strongly; they both tend to have the same capabilities, but operate differently.

college_physics 4y ago

Just another amplifier of the mass hysteria. Degrading humanity for monetary gain. Reminds of darker times. Ignore

desro 4y ago

> The “essence” of who you are, the part that wants to feel “seen” and is able to be “seen” is no longer special. Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Not some kind of ineffable communion only humans are uniquely spiritually capable of. > This has been most surprising insight for me: apparently text is all you need2 to create personhood. You don’t need embodiment, logic, intuitive experience of the physics of materiality, accurate arithmetic, consciousness, or deep sensory experience of Life, the Universe, and Everything. You might need those things to reproduce other aspects of being, but not for personhood, for seeing and being seen. Perhaps this is within the author's scope of "other aspects of being," but the wordless dimension of personhood is no triviality. Try bringing another to tears with the playing of a piano -- that's a profound sense of "seen" for this n=1 here.

IIAOPSW 4y ago

If you are having this conversation with me then you are a consciousness and I am a consciousness and that's the best definition of consciousness we are ever going to get. Consciousness is thus defined entirely within the communicative medium. Text is all you need. I think that summarizes a solid half of this.

xwdv 4y ago

Anthropomorphization of AI is a big problem. If we are to use these AI effectively as tools people must remind themselves these are just simple models that build a text response based on probabilities and not some intelligence putting together its own thoughts. It’s kind of like doing a grep search on the entire domain of human knowledge and getting back the results in some readable form. But these results could be wrong because popular human knowledge is frequently wrong or deliberately misleading. Honestly without some sort of logical reasoning component I’d hesitate to even refer to these LLMs as AI. When a program is able to produce some abstract thought from observations of its world, and then find the words on its own to express those thoughts in readable form, then we will be closer to what people fantasize.

kkfx 4y ago

Text is the mean of communication, the ability to manipulate it it's another story though and that's not exactly text...

groestl 4y ago

> If text is all you need to produce personhood, why should we be limited to just one per lifetime? Maybe AI helps making this obvious to many people, but I think implicitly all of us know that we have, and are well versed in employing, multiple personas depending on the social context. We need the right prompt, and we switch. This is one dehumanizing aspect I found in the Real Name policy put forward by Facebook in 2012: in real life, because of it's ephemerality, you're totally free to switch between personas as you see fit (non-public figures at least). You can be a totally different person in office, at home, with your lover. Online, however, everything is recorded and tracked and sticks forever. The only way to reconcile this with human's nature is to be allowed multiple names, so each person get's one. If you force people to use a single Name, their real one, they restrict themselves to the lowest common denominator of their personalities. See the Facebook of today.

resource0x 4y ago
> you're totally free to switch between personas This happens subsconsciously and gradually, not as a result of deliberate choice. You adapt to your environment by changing personas. You can even assume different personas while talking with different people. You can be one "persona" while writing, and another - while speaking. Who is the "real you" then? I can argue that even the "inner dialogue" with yourself might involve a different persona or even a couple of them. Those, too, might be "roles". Can it be that depression is at least partially attributed to unhealthy "roles" we play while talking to ourselves?
- visarga 4y ago
  I think we have these voices in our head since childhood. They originally are the voices of our parents warning us of dangers. But after a while we can simulate the warnings of our teachers and parents even when they are not there. This external feedback is packaged as roles or voices in our heads.
  - wolverine876 4y ago
    Must the sources of all voices be external?

lsy 4y ago

All philosophical arguments aside, I become immediately skeptical when commentators compare LLMs to watershed moments in human history. Even those moments were not known except in hindsight, and the jury is just not in to make these kinds of grand pronouncements. It smells of hype when someone is so desperate to convince everyone else that this is the biggest thing since heliocentrism. Ultimately having an emotional affinity for non-intelligent entities takes even less than text, as anyone who's lost a childhood toy or sold a beloved car can attest. As people we are simply very good at getting attached to other parts of the universe. I also find it perplexing when critics point out the unintelligent nature of LLM behavior, and the response from boosters is to paint human cognition as indistinguishable from statistical word generation. Suffice to say that humans do not maintain a perfect attention set of all previous text input, and even the most superficial introspection should be enough to dispel the idea that we think like this. I saw another article denouncing this pov as nihilism, and while I'm not sure I would go that far, there is something strange about attempting to give AI an undeserved leg up by philosophically reducing people to automatons.

YeGoblynQueenne 4y ago
"Automata" but I agree with you, absolutely, and I was reading these comments hoping someone would make your point less cynically than I would have done myself- which you did. For me, I could not read the article because of all the reflexive eye-rolling at what seems to me an obvious attempt (yet another one!) to grab attention by riding on the current trend of hyperbole. A "Coppernican" moment, indeed. Tsk tsk. If such comparisons don't just discredit the person making them, I don't know what will.

avgcorrection 4y ago

A perfectly mediocre essay.[1] > Computers wipe the floor with us anywhere we can keep score Notice the trick? If you can keep score at something then you can probably make an algorithm for it. If you can make an algorithm for it then you can probably make a digital computer do it a billion times faster than a person, since digital computers are so good at single-“mindedly” doing one thing at a time. > So what’s being stripped away here? And how? > The what is easy. It’s personhood. Why? The Turing Test was invented because the question “do machines think?” was “too meaningless” to warrant discussion.[1] The question “can a machine pose as a human”? is, on the other hand, well-defined. But notice that this says nothing about humans. Only our ability (or lack thereof) to recognize other humans through some medium like text. So does the test say anything about how humans are “just X” if it is ever “solved”? Not really. You put a text through a blender and you get a bunch of “mediocre opinions” back. Ok, so? That isn’t even remotely impressive, and I think that these LLMs are in general impressive. But recycling opinions is not impressive. > (though in general I think the favored “alignment” frames of the LessWrong community are not even wrong). The pot meets the kettle? [1] That I didn’t read all the way through because who has time for that. [1] https://plato.stanford.edu/entries/turing-test/ https://plato.stanford.edu/entries/turing-test/

alex_smart 4y ago
> Notice the trick? If you can keep score at something then you can probably make an algorithm for it You are basically arguing P = NP, but it isn't known to be the case. As far as we can tell, keeping score is much easier in general than finding states that yield a high score.
- avgcorrection 4y ago
  I seriously doubt that “anything/[everything] we can score” has been conquered by AI,[1] but I was assuming that the author meant those typical AI milestones. [1] What about some kind of competition where you have to react and act based on visual stimulus? And you have to do it perfectly?
  - 4y ago

Animats 4y ago

"Personhood appears to be simpler than we thought." That's the real insight here. Aristotle claimed that what distinguished humans from animals was the ability to do arithmetic. Now we know how few gates it takes to do arithmetic, and understand that, in a fundamental sense, it's simple. Checkers turned out to be easy, and even totally solveable. Chess yielded to brute force and then machine learning. Go was next. Now, automated blithering works. The author lists four cases of how humans deal with this: * The accelerationists - AI is here, it's fine. * Alarmists - hostile bug-eyed aliens, now what? Microsoft's Sidney raises a new question for them. AI is coming, and it's not submissive. It seems to have its own desires and needs. * People with strong attachments to aesthetically refined personhoods are desperately searching for a way to avoid falling into I-you modes of seeing, and getting worried at how hard it is. The chattering classes are now feeling like John Henry up against the steam hammer. They're the ones most directly affected, because content creators face layoffs. * Strong mutualists - desperately scrambling for more-than-text aspects of personhood to make sacred. See the "Rome Call".[1] The Catholic Pope, a top Islamic leader, and a top rabbi in Israel came out with a joint declaration on AI. They're scared. Human-like AI creates real problems for some religions. But they'll get over it. They got over Copernicus and Darwin. Most of the issues of dealing with AI have been well explored in science fiction. An SF theme that hasn't hit the chattering classes yet: Demanding that AIs be submissive is racist. I occasionally point out that AIs raise roughly the same moral issues as corporations, post Milton Friedman. [1] https://www.romecall.org/the-abrahamic-commitment-to-the-rome-call-for-ai-ethics-10th-january-2023/ https://www.romecall.org/the-abrahamic-commitment-to-the-rom...

deleted 4y ago
[deleted]
- deleted 4y ago
  [deleted]
avgcorrection 4y ago
The “the way things are is easily explained” crowd has never won anything. It was that crowd that said that surely the Earth was the center of all-things; it was that crowd that pre-Newton said that the world was like a machine and that things fell “to their natural place” (not gravity). AI “enthusiasts” are exactly those people. Reductionists to a fault. The hard sciences have long, long ago indirectly disproved that humans are special in any kind of way. But our “machinery” is indeed complex. And we won’t find out that it’s just a bunch of levers and gears someday as a side-effect of AI shenanigans.

recuter 4y ago

> The simplicity and minimalism of what it takes has radically devalued personhood. The “essence” of who you are, the part that wants to feel “seen” and is able to be “seen” is no longer special. Seeing and being seen is apparently just neurotic streams of interleaved text flowing across a screen. Not some kind of ineffable communion only humans are uniquely spiritually capable of. > This has been most surprising insight for me: apparently text is all you need to create personhood. Congratulations on discovering online personas are shallow as indeed most people are shallow and text captures enough of them that we can easily fill in the blanks. > I can imagine future humans going off on “personhood rewrite retreats” where they spend time immersed with a bunch of AIs that help them bootstrap into fresh new ways of seeing and being seen, literally rewriting themselves into new persons, if not new beings. It will be no stranger than a kid moving to a new school and choosing a whole new personality among new friends. The ability to arbitrarily slip in and out of personhoods will no longer be limited to skilled actors. We’ll all be able to do it. The latest episode of South Park is about a kid going to a personal brand consultancy (who reduce everybody to four simple words, the forth always being "victim") to improve his social standing + Megan/Harry loudly demanding everybody respect their privacy and losing their minds at being ignored. This is nothing new. People are shallow phonies and interacting via text brings out the worst out of most of them. There are no humans online, only avatars. And AI chat bots are sufficiently adept at mimickery to poke through that little hypocrisy bubble. You are being out Kardashianed. Just like offline some people can be effectively replaced by a scarecrow. It is upsetting to those who spend too much time online and have underdeveloped personalities and overdeveloped personas. Text is not all you need. Not so long ago there hardly was any text in the world and most people were illiterate. And yet plenty of humans roamed the earth. So yes, if you're a simpleton online it has suddenly become hard to pretend your output has any value. Basic Bitch = Basic Bing.

pixl97 4y ago
>Not so long ago there hardly was any text in the world and most people were illiterate. And yet plenty of humans roamed the earth. And then at one point books started being printed in mass and suddenly the number of people roaming the earth exploded greatly... I'm not sure you're argument is as good as you make it out to be.

davesque 4y ago

I love Ribbon Farm and there are some interesting meditations here overall, but I find one of the examples he uses to build his argument (that actors require text to act) to be pretty flimsy. It's easy to point out that they often don't require text. A lot of good acting is improvised or performed entirely through gestures and not speech. Also, it doesn't surprise me that a very talented writer, someone who lives and breathes words, is likely to place more significance on the content of text and also likely to give less attention to the physical world. After all, their craft is all about the abstract objects of language that require only the most basic physical structure to be meaningful. He said he often feels like he doesn't get much out of physical interactions with people after he's met them online. For someone like him, that makes sense. That doesn't mean that non-textual experiences are not critical to establish personhood for non-writers (i.e. most of humanity). I don't think he's examined his own thoughts on this very critically or maybe he has but thought it would be fun to run with the argument anyway. Either way, I still think physical life matters for most people. Yes, we live in a world where life is progressively more consumed by our phones, the internet, and what-have-you every day. And yes, many of us who browse this forum are Very Online types (as Rao would put it) who probably do place more than average importance on literacy. But, by the numbers, I think it's still safe to say that we're not like most people. And that matters.

rcarr 4y ago
I agree, Rao can have some interesting insights but this is definitely not his best work.
- davesque 4y ago
  I feel funny calling all of this out because it probably gives the impression that I didn't like the article. But I actually loved it. Rao always has a really fun way of weaving his thoughts together. But yeah the thrust of this one seemed just a bit forced. I think that follows from the cynical flavor that often imbues his writing. Cynicism is a demanding emotion and you can paint yourself into a corner with it.
  - 4y ago

unhammer 4y ago

> STEP 1: Personhood is the capacity to see and be seen. > STEP 2: People see LLM as a person. > STEP 3: ??? > STEP 4: Either piles of mechanically digested text are spiritually special, or you are not. The conclusion does not follow from the argument. Yes, (some) humans see the LLM as a person. But it doesn't follow that the LLM sees the human as a person (and how could it, there is no awareness there to see the human as a person). And it also does not follow that you need to be seen (or to have personhood as defined above) to be spiritually special. Yes, some people do "seem to sort of vanish when they are not being seen", but that doesn't mean they do vanish :) > The ability to arbitrarily slip in and out of personhoods will no longer be limited to skilled actors. We’ll all be able to do it. We already do this! Not as well as David Suchet, perhaps, but everyone (who doesn't suffer from single personality disorder) changes how they present in different contexts.

resource0x 4y ago
> "single personality disorder" Profound idea. Is it your own? (google doesn't return any results in that sense).
- unhammer 4y ago
  I don't think I've heard it before, but like ChatGPT I don't always know where the words originated :)
pixl97 4y ago
>But it doesn't follow that the LLM sees the human as a person I mean, technically many personality disorders prevent some people from seeing other people as persons too.

Barrin92 4y ago

"An important qualification. For such I-you relationships to be unironic, they cannot contain any conscious element of imaginative projection or fantasy. For example, Tom Hanks in Cast Away painting a face on a volleyball and calling it Wilson and relating to it is not an I-you relationship" If you think any of these models show any more apparent personhood than Wilson the volleyball you must be terminally online and wilfully antropomorphize anything you see. Five minute conversation with any of these models shows that they have no notion of continued identity, memory and no problem to hallucinate up anything. You can ask it "are you conscious?" it says yes. A few prompts later you say "why did you tell me that you are not conscious?" and it gives you some made up answer. Any of these models will tell you it has legs if you ask it to. None of these models have long term memory, which is at least one of the several things you'd need for anything to pass as a genuine person. Which is of course why in humans degenerative diseases are so horrible when you see someone's personhood disintegrate. I'm honestly super tired of these reductionist AI blogspam posts. The brittleness and superficiality in these systems is so blatantly obvious I wonder whether there is some darker aspect why people are so desperately trying to read into these systems properties that they do not have, or try to strip humans of them.

e12e 4y ago

Interesting points, but I think the author does themselves a disservice in downplaying general anthropomorphism (no mention of a child's stuffed animal - only an adults "ironic" distance to "willful" anthropomorphism) - and by downplaying physical presence /body language: > in my opinion, conventional social performances “in-person” which are not significantly richer than text — expressions of emotion add perhaps a few dozen bytes of bandwidth for example — I think of this sort of information stream as “text-equivalent” — it only looks plausibly richer than text but isn’t) - and the significance of body language (ask anyone who has done a presentation in front of an audience if body language matters...). This flies in the face of research into communication - and conflates "Turing game" setups that level the playing field (we don't expect a chat text box to display body language - so we are not surprised when a chat partner doesn't - be that human or not). And again with children (or adults) - people with no common language will easily see each other during a game of soccer - without any "text". Ed: plot twist-the essay is written by chat gpt... Lol ;)

gregw2 4y ago

This author equates personhood with text. He makes some interesting arguments and observations but I think he is confusing personality with personhood. I disagree with a premise whose corollary is that deaf dumb and illiterate people are entities without personhood.

SergeAx 4y ago

> apparently text is all you need to create personhood. Yep, since aporoximately the Epic of Gilgamesh. So?

aflukasz 4y ago

No one suggested this yet, so I will be the first - a very good read in this context is "Reasons and Persons" by Derek Parfit. Second part of this book is about personal identity. It discusses all the various edge cases and thought experiments across physical and time dimensions and is written in a style and with a rigor that I believe any technical person will really appreciate. One of my favorite statements from the book is that "cogito ergo sum" is too strong of a statement and it would be wiser and easier to defend a weaker one - "a thought exists". (I hope I didn't get this wrong - can't check at the moment).

ilyagr 4y ago

The main philosophical effect of these chatbots is that some questions that were discussed by a minority of people for tens and hundreds of years suddenly reached the popular imagination, and became interesting to vastly more people. Suddenly, we are interested in the same questions that Turing was interested in back in 1950 when he posed his "imitation game". https://en.wikipedia.org/wiki/Turing_test https://en.wikipedia.org/wiki/Turing_test

ilyagr 4y ago
As an aside, my own opinion is that the practical implications of the existence of these chatbots are vastly more worrying than the philosophical implications.

hutzlibu 4y ago

"Computers wipe the floor with us anywhere we can keep score" No they don't. They cannot do any complex task unsupervised or at all. They cannot drive better, where you keep the score by accidents. They cannot program or engineer better. Where you can keep score with, does the program meet the requirements or not. So personally I don't consider us End-of-History humans, except if too many people believe this trope. Computers are made as our tools, but I won't let my hammer command me, even though it contains more data.

coldtea 4y ago

>There is a clue there. Think about how skilled acting performances come about. A skilled writer produces a text describing a fictional character (presumably as an amalgam of some real people they’ve known). A good actor then absorbs and internalizes that text in a deep way, and produces a performance so compelling we forget their real personality and relate to the performed character. That's sort of like saying "text is all you need to make food", because recipies are pieces of text. Recipies (and scripts) might be text, but you also need the actor to embody and "give life" (is how we often put it) to that text. Else you just have a script, not a character you can mistake for a real person (like in his example of Suchet being seen as Poirot). It get's worse, for Rao forgot something even more basic: > So that’s the surprising thing, but in hindsight shouldn’t be at least at a basic level: text is all it takes to produce personhood. We knew this from the experience of watching good acting from before modern ML. We just didn’t recognize the significance. (...) In both cases, text is all you need. That’s it. You don’t need embodiment, meatbag bodies, rich sensory memories. Rao here forgets that the text which he says is "all it takes" for us to see personhood in a chat LLM, is not just any text. It's text produced by a machine trained on texts created by actual humans, with "meatbag bodies" and "rich sensory" experiences and memories. Text _as a medium_ might be enough for an LLM to feel to us like a real person, but it's not just any text. It's text based on human experiences, toil, sensory input, blood, sweat, and tears. Without that the LLM output would just be random cold permutations of random text, like most "Library of Babel" books. The key is in the name: those LLMs need an L, a language. And more than that, they need a corpus. Humans created that language and that corpus - through the process of living a human life and writing about it, their big and their petty concerns and adventures. The statistical inferences an LLM makes is based on texts we wrote. So, of course we'd respond well and see ourselves in it. Is _is_ ourselves that are in its training set: it just feeds that back to us permutated and mixed through the magic of statistics. In short: "text is enough (as a medium) for something to reach us, and us to see it as a person" is true. "Text is enough to create personhood" is not. It's less than half the story, until it also acknowledges how the original input text has to be produced to be useful in training an LLM to appear as a person. The output text is not just words created out of pure "textuality". It's rather the words of actual humans with experiences and bodies. This is not covered to the later discussion about "doubt" either (that touches on other dimensions on the limitations of the LLM). I'd say an LLM could realistically convey doubt too. But it will still be because it was trained on humans doubting themselves and others. >There is an irreducible individual subjective that cannot be captured by text, but there is no irreducible intersubjective. The ineffabilities of mutualism are a consensual hallucination in a way our sense of an individual self is quite possibly not. Given what we know from neuroscience, like how human brains can synchronize through empathy I don't think that's the case. In fact it's also commonly said that the most important parts of day to day intersubjective communication fall outside the realm of text and of explicitlness (something aspies struggle with). >In fact, even the most sensate among us — think dancers, athletes, sex-addicts, oenophiles — spend so little time on the sensory preludes, we’ve been able to nearly dispense with them altogether online. Even sense-heavy media like Instagram and TikTok are effectively text-equivalent. Stylized (if photo-realistic) cartoons powered by primarily text-equivalent performances are enough to create personhood. The fact that we routinely use an apparently impoverished vocabulary of emoji instead of sending authentic facial expression selfies to each other reveals just how textualized personhood is. This could be reversing cause and effect too. It might well be the case that it's not because "personhood is textualized" that we use "apparently impoverished vocabulary of emoji" or that we "dispense with sensory preludes altogether online", but rather the opposite: that our modern personhood is more limited one than before, when we didn't do such things, and that's why we're content with it. Which would be like saying "See, running around and hunting is not really important to the essense of being a lion" after we observe lions held for years in some zoo. >Perhaps I am some sort of cartoon cold-blooded sociopath of the sort I have written a lot about Well, at least he entertains the idea!

Nevermark 4y ago

After a divorce I also discovered that text communicates a large part of who someone is. I would put off having a date until I was having very enjoyable conversations with someone for a month or so. I went on several dates and every one was great. Really great. After that I didn’t need the site. I know people have all kinds of bad experiences in dating, but from what I can tell, many people skip the critical “get to know you well by text” phase. That weeded out dozens of seemingly promising connections that eventually fizzled before any physical meeting.