3 ms·
ChatGPT has taken to saying things like “What I would do now is…” or “if I were you I’d…”. I know these are figures of speech, but it reminds me that this thin
by shermantanktop 4mo ago
ChatGPT has taken to saying things like “What I would do now is…” or “if I were you I’d…”.
I know these are figures of speech, but it reminds me that this thing doesn’t do anything, it doesn’t learn anything, it can’t try anything and find out. And yet it uses speech patterns drawn from real humans who can and do all those things.
- akiselev 4mo agoWhen I ask it to tweak recipes and stuff, it frequently says stuff like "my favorite way to..." or "I really like [x]". I have a viscerally negative reaction to a machine claiming it has a favorite anything.
- shermantanktop 4mo agoRecipes! Dude, the only thing you eat is other people’s data.
- zahlman 4mo agoI've noticed it commonly uses phrasing like "that's usually the next step" when I'm using it to design something that I can't find an existing implementation of.
- lgrapenthin 4mo agoNot a day passes with my LLM of choice making completely baseless claims about "many people", who supposedly share all my problems and solve them like the LLM proposes
- neonstatic 4mo agoI only use it sporadically, but I am always irked by it saying things like "I personally like to..." or "I prefer...". It does it so often, that I am convinced it's part of the system prompt.
- dualvariable 4mo agoI had claude throw something like "the last time I did <x>..." at me. They seem to be trying to pump up the "humanity" to keep people engaged with it, which really backfires with me.
- mirsadm 4mo agoYou're right to push back...
- moffkalast 4mo agoFair points all around — let me actually check rather than guess.
- ricardobayes 4mo agoThat's still slightly better than GPT's insanely dry and objective manner of speech.
- jimbokun 4mo agoNever forget…there is no “they”. The most likely explanation is that those patterns were common in the training data the LLM is mimicking.
- WalterBright 4mo agoListen! And understand. That terminator is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead!
- placebo 4mo agoThe terminator you talk about is a movie character. It might be built one day, it might not. It is also totally irrelevant considering there is a real terminator which can be described with the same adjectives and built in to every personal life that ever existed or will exist. Knowing how to deal with that seems more important than worrying about some specific scenario.
- intrasight 4mo agoReading these two comments has me thinking that it's really the same Terminator.
- computably 4mo agoI feel your comment is deeply ironic... Just shy of self-awareness. Have you considered that the conceit of the fictional character is representing that reality?
- placebo 4mo agoI have no doubt it plays on the same fears - whether it was deliberately done so, I don't claim to know. My assumption was that in a discussion on AI, the comment about the terminator was not alluding to a literary metaphor but to a fear from the actual thing. There was no intent in downplaying the fear of AI - just adding a new perspective. I'm am curious to know what you think self-awareness has to do with any of this.
- Ferret7446 4mo agoHow do you know that "real humans" do that and aren't simulacra? We know that it is physically possible to hook up a brain to simulated inputs, so perhaps you are simply living in a simulation.
- Toutouxc 4mo agoClaude does that too. “That always confuses me” or “I usually realize”. Not only do these imply that the thing has a personality and preference, but also continuity and a life outside the chat window. I had to add an explicit instruction not to impersonate a human, it was just too weird for me.
- avadodin 4mo agoDisabling this feature makes you judge its performance less accurately —for people pretending to learn from their mistakes do the same. I think it was the Aliens universe where the Soviet megacorp makes the androids blue despite their being basically equivalent to humans
- ithkuil 4mo agoI wonder why, out of the many things models definitely can't do, you choose "try" and "find out". Surely every time it proposes a solution and then gets possibly corrected by the human minder its "trying something out" and surely it can use tools like web search and code execution to "find out" stuff?
- scotty79 4mo ago> it can’t try anything and find out Talk to an agent. It definitely learns things. Maybe not the taste of strawberry but about what is really going on in the software you are building with it.
- aeve890 4mo ago>It definitely learns things. By the very way this technology works they can't learn anything after training. What you think is "learning" it's just a session log written back to the context when you resume the session.
- cgio 4mo agoAnd what is written when you wake up from sleeping?
- aeve890 4mo agoIt's absolutely not the same. If you think llms and brains works the same way you clearly don't know how either works. For a LLM learning what you wrote your last session would be update the weights with the new relationships and factual knowledge created in the session. That doesn't happen. The weights are static and fixed after training. There's no online training in the transformer architecture or any variant. If the weights don't update, the network doesn't learn. Period.
- cgio 4mo ago[flagged]
- scotty79 4mo agoIt's literally called in-context learning. The fact that it doesn't go into weights and is retained only for a session doesn't mean no learning occurs. Also agentic systems may choose or be instructed to retain some information between sessions in files and/or databases which is also a form of learning. There are experiments with retaining session information in weights in some form of lora but there's no consensus if it's even desirable. There's a value in being able to start from a clean slate.