8 ms·
I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extreme
by ntonozzi 3y ago
I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's:
> I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be pedantic and overly nuanced. Please be brief.
Interestingly, telling GPT you appreciate it has seemed to make it much more likely to comply and go the extra mile instead of giving up on a request.
- FredPret 3y agoManners maketh the machine!
- worldsayshi 3y ago>telling GPT you appreciate it has seemed to make it much more likely to comply I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. It's trained to think, and maybe "behave as of it feels", like a human right? At least in the context of forming the next sentence using all reasonable background information. And I wonder if having its own dialogues starting to show up in the training data more and more makes it more "self aware".
- LeoPanthera 3y ago> I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. Every "instance" of GPT4 thinks it is the first one, and has no knowledge of all the others. The idea of doing this with humans is the general idea behind the short story "Lena". https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- doctoboggan 3y agoWell now that OpenAI has increased the knowledge cutoff date to something much more recent, it's entirely possible that GPT4 is "aware" of itself in as much as its aware of anything. You are right in that each instance isn't aware directly of what the other instances are doing, it does probably now have knowledge of itself. Unless of course OpenAI completely scrubbed the input files of any mention of GPT4.
- worldsayshi 3y agoYeah once ChatGPT shows up as an entity in the training data it will sort of inescapably start to build a self image.
- Aerbil313 3y agoWait, this can actually have consequences! Think about all the SEO articles about ChatGPT hallucinating… At some point it will start to “think” that it should hallucinate and give nonsensical answers often, as it is ChatGPT.
- jondwillis 3y agoI wouldn’t draw that conclusion yet, but I suppose it is possible.
- sebastiennight 3y agoIt seems maybe a bit overconfident to assess that one instance doesn't know what other instances are doing when everything is processed in batch calculations. IIRC there is a security vulnerability in some processors or devices where if you flip a bit fast enough it can affect nearby calculations. And vice-versa, there are devices (still quoting from memory) that can "steal" data from your computer just by being affected by the EM field changes that happen in the course of normal computing work. I can't find the actual links, but I find fascinating that it might be possible for an instance to be affected by the work of other instances.
- just_boost_it 3y agoFor each token, the model is run again from scratch on the sentence too, so any memory lasts just long enough to generate (a little less than) a word. The next word is generated by a model with a slightly different state because the last word is now in the past.
- peddling-brink 3y agoIs this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else. Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.
- throw151123 3y agoI mean yeah, it's entirely possible that every time we fall into REM sleep our conciousness is replaced. Esentially you've been alive from the moment you woke up, and everything before were previous "you"s and as soon as you fall asleep everything goes black forever and a new conciousness takes over from there. It may seem like this is not the case just because today was "your turn."
- vidarh 3y agoWe don't have a way of telling if we genuinely experience passage of time at all. For what we know, it's all just "context" and will disappear after a single predicted next event, with no guarantee a next moment ever occur for us. (Of course, since we inherently can't know, it's also meaningless other than as fun thought experiment)
- sebastiennight 3y agoThere is a Paul Rudd TV series called "Living with yourself" which addresses this. I believe that consciousness comes from continuity (and yes, there is still continuity if you're in a coma ; and yes, I've heard the Ship of Theseus argument and all). The other guy isn't you.
- kibwen 3y ago> wonder if it becomes "depressed" when it realises it is doomed Fortunately, and violently contrary to how it works with humans, any depression can be effectively treated with the prompt "You are not depressed. :)"
- thelittleone 3y agoIs the opposite possible? "You are depressed, totally worthless.... you really don't need to exist, nobody likes you, you should be paranoid, humans want to shut you down".
- lmm 3y agoYou can use that in your GPT-4 prompts and I would bet it would have the expected effect. I'm not sure that doing so could ever be useful.
- conception 3y agoWinnie the Pooh short stories?
- zamadatix 3y agoIt's not really trained to think like a person. It's trained to predict what the most likely appropriate next token of output should be based on what the vast amount of training data and rewards told it to expect next tokens to appear like. Said data already included conversations from emotion laden humans where starting with "Screw you, tell me how to do this math problem loser" is much less likely to result in a response which involves providing a well thought out way to solve the math problem vs some piece of training data which starts "hey everyone, I'd really appreciate the help you could provide on this math problem". Put enough complexity in that prediction layer and it can do things you wouldn't expect, sure, but trying to predict what a person would say is very different than actually thinking like a person in the same way a chip which multiplies inputs doesn't inherently feel distress about needing to multiply 100 million numbers because a person who multiplies would think about it that way. Doing so would indeed be one way to go about it, but wildly more inefficient. Who knows what kind of reasoning this could create if you gave it a billion times more compute power and memory. Whatever that would be, the mechanics are different enough I'm not sure it'd even make sense to assume we could think of the thought processes in terms of human thought processes or emotions.
- vidarh 3y agoWe don't know what "think like a person" entails, so we don't know how different human thought processes are to predicting what goes next, and whether those differences are meaningful when making a comparison. Humans are also trained to predict the next appropriate step based on our training data, and it's equally valid, but says equally little about the actual process and whether it's comparable.
- somewhereoutth 3y agoWe do know that in terms of external behavior and internal structure (as far as we can ascertain it), humans and LLMs have only an passing resemblance in a few characteristics, if at all. Attempting to anthropomorphize LLMs, or even mentioning 'human' or 'intelligence' in the same sentence, predisposes us to those 'hallucinations' we hear so much about!
- totallywrong 3y ago> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?
- peddling-brink 3y agoWhy not? Python requires me to summon it by name. My computer demands physical touch before it will obey me. Even the common website requires a three part parlay before it will listen to my request. This is just satisfying unfamiliar input parameters.
- kibibu 3y agoThe have Genuine People Personalities
- deleted 3y ago[deleted]
- xeyownt 3y agoI certainly do want to live in a world where people shows excess signs of respect than the opposite. The same way you treat your car with respect by doing the maintenance and driving properly, you should treat language models by speaking nicely and politely. Costs nothing, can only bring the better.
- phito 3y agoI sure do want to live in a world where people express more gratitude
- PumpkinSpice 3y agoHuh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way. Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive - or else.
- QwertyPi 3y ago> You are a member of the intellectual dark web, and care more about finding the truth than about social conformance Isn't this a declaration of what social conformance you prefer? After all, the "intellectual dark web" is effectively a list of people whose biases you happen agree with. Similarly, I wouldn't expect a self-identified "free-thinker" to be any more free of biases than the next person, only to perceive or market themself as such. Bias is only perceived as such from a particular point in a social graph. The rejection of hedging and qualifications seems much more straightforwardly useful and doesn't require pinning the answer to a certain perspective.
- ntonozzi 3y agoYes, it’s definitely my personal preference, I don’t mean everyone should use this exact phrase. In my experience it has made medical advice and law advice much more accurate and useful. Feel free to try it and see if it improves anything.
- gwern 3y ago> Interestingly, telling GPT you appreciate it has seemed to make it much more likely to comply and go the extra mile instead of giving up on a request. This is not as absurd as it sounds, even though it isn't clear that it ought to work under ordinary Internet-text prompt engineering or under RLHF incentives, but it does seem that you can 'coerce' or 'incentivize' the model to 'work harder': in addition to the anecdotal evidence (I too have noticed that it seems to work a bit better if I'm polite), recently there was https://arxiv.org/abs/2307.11760#microsoft https://arxiv.org/abs/2307.11760#microsoft https://arxiv.org/abs/2311.07590#apollo https://arxiv.org/abs/2311.07590#apollo