4 ms·
The only sane way is to treat LLMs like a computer in Star Trek. Give it precise orders and clarify along the way, and treat it with respect but also know its a
by jagermo 1y ago
The only sane way is to treat LLMs like a computer in Star Trek. Give it precise orders and clarify along the way, and treat it with respect but also know its a machine with limits. Its not Data, its the ships voice
- falcor84 1y agoWait - why is it not Data? Where is the line?
- nkohari 1y agoThat topic (ship's computer vs. Data) is actually discussed at length in-universe during The Measure of a Man. [0] The court posits that the three requirements for sentient life are intelligence, self-awareness, and consciousness. Data is intelligent and self-aware, but there is no good measure for consciousness. [0] https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Trek:_The_Next_Generation) https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Tre...
- ImHereToVote 1y agoDoesn't ChatGPT fulfill these criteria too?
- nkohari 1y agoAgain, there's no real measure for consciousness, so it's difficult to say. If you ask me, frontier models meet the definition of intelligence, but not the definition of self-awareness, so they aren't sentient regardless of whether they are conscious. This is a pretty fundamental philosophical question that's been considered for centuries, outside of the context of AI.
- ImHereToVote 1y agoChatGPT knows about the ChatGPT persona. Much like I know the persona I play in society and at home. I don't know what the "core" me is like at all. I don't have access to it. It seems like a void. A weird eye. No character, no opinions. The persona; I know very well.
- nkohari 1y agoTo the extent it "knows" (using that word loosely) about the persona, it's deriving that information from its system prompt. The model itself has no awareness. The sooner we stop anthropomorphizing AI models, the better. It's like talking about how a database is sentient because it has extremely precise memory and recall. I understand the appeal, but LLMs are incredibly interesting and useful tech and I think that treating them as sentient beings interferes with our ability to recognize their limits and thereby fully harness their capability.
- falcor84 1y agoNot the parent, but I understood it as them saying that the model has as part of its training data many conversations that older versions of itself had with people, and many opinion pieces about it. In that sense, ChatGPT learns about itself by analyzing how its "younger self" behaved and was received, not entirely unlike how a human persona/ego is (at least in part) dependent on such historical data.
- ImHereToVote 1y agoI mean it in the way an Arduino knows a gas leak is happening. I similarly like the Arduino, I know about my persona that I perform. I'm not anthropomorphizing the Arduino. If anything, I'm mechamorphizing me.
- jagermo 1y agoits not self-aware, regardless what it tells you (see the original link)
- falcor84 1y agoI'm not sure what you're referring to in the original link, can you please paste an excerpt? But thinking about it - how about this, what if you have a fully embodied LLM-based robot, using something like Figure's Helix architecture [0], with a Vision-Language-Action model, and then have it look at the mirror and see itself - is that on its own not sufficient for self-awareness? [0] https://www.figure.ai/news/helix https://www.figure.ai/news/helix
- ImHereToVote 1y agoI'm not claiming it is.
- dclowd9901 1y agoIn a Chinese room sort of way, sure. The problem is we understand too well how it works, so any semblance of consciousness or self awareness we know to be simple text generation.
- falcor84 1y agoThe word "simple" is doing quite a massive amount of work there.
- dclowd9901 1y agoHm that's fair. Retroactively I'd say "obscenely resource intensive text generation." Meanwhile for me to generate text all I need is water and a banana.
- falcor84 1y agoThat's not so clear cut. I have a half decent local LLM running on my phone for a significantly lower power consumption than a human body.
- krackers 1y agoUsing science fiction as a basis for philosophy isn't wise, especially TNG which has a very obvious flavor of "optimistic human exceptionalism" (contrast with DS9, where I think Eddington even makes this point).
- Cthulhu_ 1y agoI'd argue the ship's computer is not an LLM, but a voice assistant.
- jagermo 1y agoI don't know, it has been shown multiple times that it can understand voice requests, find and organize relevant files. For example in The measure of a Man (TNG).
- codedokode 1y agoThere is no need for respect, do you respect your phone? Do you respect the ls utility? Treat it as as search engine, and I wish it used the tone for the replies that makes it clear that it is just a tool and not a conversation partner, and not use misleading phrases like "I am excited/I am happy to hear" etc. How can a bunch of numbers be happy. Maybe we need to make a blacklist of misleading expressions for AI developers.
- AlecSchueler 1y agoThe tone you take with it can demonstrably affect the results you get, which isn't true of the other tools you listed.
- codedokode 1y agoThis is a bug that needs to be fixed.
- falcor84 1y agoWhy? I for one prefer to live in a world where people are nicer by default, without first engaging in mental calculus about whether the other side is sufficiently like them to merit niceties.
- AlecSchueler 1y agoWhether or not that's true doesn't the efficacy of the advice given above.
- jagermo 1y agoDo what you want, i am not risking offending the machine spirit.