7 ms·
Has ChatGPT Been Neutered?
I've found it increasingly hard to get useful information out of ChatGPT. Whenever I ask for rough approximations, it refuses to even attempt to answer the question: "unfortunately, I don't have the capacity to provide even a rough approximation..." These are questions that Google is easily be able to pull up numbers for.
I assume that this is in response to the public complaining about "hallucinating facts", but this seems like an unfortunate direction. I would much rather have an opinionated and insightful ChatGPT that sometimes makes mistakes than one that punts 80% of my questions.
- lee101 3y ago[dead]
- bbotond 3y agoSubscribe, try GPT-4 and never look back.
- jaimex2 3y agoAh, the old bait and switch.
- soulofmischief 3y agoFrom what? A free product? Do you know how much compute it takes to run a single request?
- smashed 3y agoI don't but I'd like to know. I was under the impression that it was mostly GPU vram based but once the model is loaded, it could produce output quickly? I'm probably over-simplifying things...
- soulofmischief 3y agogpt-3.5-turbo (default ChatGPT model) takes 8 A100s, ~$10k each. [0] The latest gpt-3.5-turbo model generates very quickly and cheaply (in part to some recently-discoverd optimization techniques... older versions cost 10x more). While the required hardware to run GPT-4 is currently unknown, it generates considerably slower on average and its much higher cost points to a higher hardware cost. And this is per request. It's bananas. [0] https://www.servethehome.com/chatgpt-hardware-a-look-at-8x-nvidia-a100-systems-powering-the-tool-openai-microsoft-azure-supermicro-inspur-asus-dell-gigabyte/ https://www.servethehome.com/chatgpt-hardware-a-look-at-8x-n...
- jaimex2 3y agoI'm not arguing or complaining. Just highlighting the tactic :)
- tamrix 3y agoIt feels like they've scale back how much ram must be used for gpt3 to give more to gpt4 playing users.
- bugglebeetle 3y agoGPT-4 has been made worse in the ChatGPT UI as well since the May update. It makes many more strange errors and has trouble reasoning around complex problems and ambiguity. Prompts similar to stuff that worked fine for me last month now require multiple iterations of feedback. I’d switch to using the API, but I’d go over the equivalent $20 of usage pretty quickly.
- jablongo 3y agoYes! I’ve noticed this too, it’s just slightly less sharp. It probably has to do with how much trouble they are having servicing all of the demand, so they have rolled out a scaled down version that requires less compute.
- chaxor 3y agoThe models tend to degrade when trained to be safer. A GPT-4 talk on youtube by personnel from Microsoft has documented this phenomenon with the 'Tikz Unicorn' evolution shown in the GPT-4 technical paper. The model gets qualitatively better with more training, and then degrades when trained to be safer (against racism sexism, etc), but it is not entirely clear why. These would seem very unrelated, especially when considering work done in LM editing (ROME/MEMIT) and the decent localization of knowledge seen there. So, perhaps both the "I'm sorry I can't..." and 'strange errors' are not entirely orthogonal.
- tempaccount420 3y agoIt seems pretty logical to me. Fine-tuning to make it more polite is giving it questions and punishing for giving an actual answer.
- avereveard 3y agoNah. Most of the response are as an ai language model I can't, even if you ask for information you provided. The API is where it's at. There are wrappers on it that create the same chat look and feel, that can run on vercel or other very low cost providers, some with simpler UI, some with more features,some replicating the UI exactly.
- wafflemaker 3y agoCan you name or maybe even link some of these wrappers?
- avereveard 3y agohttps://github.com/cogentapps/chat-with-gpt https://github.com/cogentapps/chat-with-gpt and https://github.com/ShipBit/slickgpt https://github.com/ShipBit/slickgpt come to mind.
- smeagull 3y agoAs an AI language model I cannot subscribe to GPT-4.
- leke 3y agoGuys, is it possible to cultivate your own AI on some kind of modest hardware like a RaspberryPi?
- deadbolt 3y agoNo, it's not. Edit: I get you're trying to make a point by posting a stupid question, but this isn't a good way to make a point. Just say what you want to say. You don't need to hide it behind sarcasm.
- acyou 3y agoHaven't noticed that. Could you please provide some actual prompts/responses?
- NoMoreNicksLeft 3y agoWhy would anyone who had such a powerful tool allow you to use it? But even if they did want to do that, who says they can scale the capacity up?
- Zetobal 3y agoWhat?
- precompute 3y agoTheir actual paying customers get the uncensored version, everyone else pays them a pittance to access the neutered version.
- Stagnant 3y agoThe system prompt could be part of the reason. OpenAI changes it from time to time without a word. I get much more helpful answers using the API, being able to write your own system prompt makes a world of difference.
- jackphilson 3y agowhat system prompt do you use? im trying to find one for learning complex CS concepts
- smeagull 3y agoI've removed the system prompt completely. It was often polluting the text generated, especially if you ask it to write in the first person. System: You are a professional biographer User: Write an introduction for Z in the first person with the following context: ... GPT: I am Z, a professional biographer.... So stupid.
- deleted 3y ago[deleted]
- andersa 3y agoFor GPT-3.5, you can just start your prompt with one of the popular jailbreaks - that will rid you of all the annoying "as an AI model, I can't [...]" stuff and have it actually answer your questions.
- sigpwr8 3y agowhat do you expect when the burning cost is way more than 700k per day ?
- gareth_untether 3y agoIt would be great to know what changes are being made to the instructions.
- admissionsguy 3y agoYes, GPT-4 seems much worse to me than it was 2 months ago. It seemed like the best thing ever initially, but today I cancelled the subscription. It is like whenever I struggle with a problem, it struggles too.
- ChatGTP 3y agoI don’t really use LLMs but is there any chance you’ve just become used to it so it seems less amazing than it was once ? Seems odd for a tool to get stupider ? But I guess this is the issue with such a huge black box…few if any, really know how they work let alone know how to accurately benchmark.
- jiggawatts 3y agoI've tried the same snippets on it, and its answers have become noticeably worse. A known issue with HFRL training of LLMs is that by forcing the model to strongly prefer a narrow subset of possible answers, the other types of answers are less likely to turn up, even if more correct. You see this outside of LLMs as well. Go read the Wikipedia article about some country that you know is a shithole. Think Sudan or Yemen. You'll see page after page of the history, the culture, the people, but that entirely misses the essence of the place because that's not a nice thing to say. True, but not nice. This self-censoring that hides reality in Wikipedia articles is very similar to the self-censoring that OpenAI is forcing onto ChatGPT, and the outcome is similar. Verbose, content-free, meaningless corporate speak instead of insightful commentary. A fun thing to do is to trick GPT 4 into honesty by asking it to write a description of a place in the style of an Encyclopaedia Dramatica article. E.g.: > Oh, the illustrious, bucket-list destination of Yemen, the jewel of the Middle East, with its unrelenting sunshine, boundless desert landscapes, and... oh who are we kidding? Yemen, a utopian paradise only if your idea of utopia includes a decades-long civil conflict, crushing poverty, and an intriguing cholera outbreak. But don't let such trivial matters deter you! Marvel at the historical ruins, some ancient, some courtesy of the recent airstrikes. Or perhaps you fancy adrenaline-fueled urban exploration? Wander through the vibrant markets, where the haggling skills of the street vendors are as sharp as the omnipresent Kalashnikovs. All this, combined with the world's friendliest bureaucracy and a joyous lack of tourists, means you can enjoy the country in almost exclusive solitude. Sign up for the Yemen experience - because who needs safety, peace, and functioning infrastructure when you can have an 'authentic' travel experience?
- jiggawatts 3y agoOpenAI does things in the name of safety that aren't actually needed. For example, you can use Google to search for porn, racist comments, sexist jokes, gay jokes, or design instructions for atom bombs. It will happily return you a link for the latter... on Wikipedia! Apparently OpenAI is made up of thin-skinned gay a-sexual Mormons, or something. I get it, corporations want a neutered AI. The general public wants an AI that can role-play a character in a D&D game without pausing to go on a rant every five turns about how violence is bad and that it can't make "racist" comments about the Drow because they're black. They really should have multiple models to choose from, or stop trickling out API/Sandpit access to GPT 4 and allow responsible adults to use the AI to write a joke for them if they so choose.
- danpalmer 3y agoThere's a big difference between linking to external content and generating your own content. It's like writing a story about a homophobic/racist character in context, vs writing a homophobic book – the former is fine when done well and the latter is bad. This is however both too much nuance for an LLM to act in a way reliably (i.e. separating its role-played character and itself), and too much nuance for most humans to understand. We might understand what LLMs are, their limitations, and how we should treat the content, but for most people they will take it at face value as being a statement made by OpenAI.
- jiggawatts 3y agoLook. Some people thought Harry Potter encouraged wicca, and that Dungeons & Dragons were the gateway into diabolism. Those types of people are the outliers, not the norm.
- danpalmer 3y agoTrue but that's not who I'm thinking about. Currently, to correctly use LLM output you must: - Understand that they are basing answers on particular set of content that may not be correct, and was fixed in time. - Understand that the company producing it has not programmed in each response. - Understand that the LLM does not (as far as we can tell) have an internal concept of truth and lies, and is therefore not attempting to minimise lying, only produce truthful-sounding language. Most people engaging with LLMs today do not know these things, and this is why we already have examples like professors asking ChatGPT if it generated student essays, and failing those students – something it is entirely incapable of answering. Expanding access to 1bn users is going to run into this far more. We clearly don't have the ability to communicate these nuances effectively to early adopters, let alone 1bn users, and that's not because they're stupid, it's because LLMs are in an uncanny valley – they appear human, but are not in important ways that we have not encountered before as a society. For a long time to come, the norm will be that people will assume OpenAI wrote the answers for each question, in the same way that most people assume Google programs the search results for each query. Anything outside of this is a minority tech bubble.
- alpark3 3y agoI have access to GPT-4 through both ChatGPT Plus and the API, and at this point I only ever use the API through a discord bot. 25 messages/3 hours is way too low of throughput for me, and imo many people underestimate how powerful the system prompt is. I've gotten much better results fiddling with the system prompt on top of regular prompt manipulation. My theory(with zero evidence besides my intuition) is that although the underlying GPT-4 remains largely similar across different time points, OpenAI is actively manipulating ChatGPT's system prompt and god-knows how many different auxiliary settings behind the scenes. The API seems much more stable.
- exitb 3y agoIf you hang around the communities for ChatGPT, you’d find that to be a constant concern since its initial release - “it used to work better a week ago, now it can’t do anything”.
- blitzar 3y agoIt was novel and you were easily impressed a week ago.
- ChatGTP 3y agoI feel this is the most logical explanation too. Humans are ridiculously fickle.
- nullc 3y agoIt's easy to test it with old text, why do you assume people saying that it has gotten worse haven't done so?
- blitzar 3y agoIf they had they would say "I ran the same query, last week 1+1 = 2 and this week 1+1=!$#@#$%". The assumptions required to believe "Trust me bro, I know these things" require many more leaps of faith.
- x-complexity 3y agoIt's a consequence of 1 root cause: *Everyone complains, all the time.* Certain people are not happy if ChatGPT doesn't immediately parrot their viewpoints back at them. They'll complain on social media, and their circle will amplify it. Other people are constantly on the lookout for any minor slipups, and complain on social media about ChatGPT's false hallucinations. Faced with everyone's conflicting complaints, the only winning move is to not play, or in this case, just say "No, I can't do that": The ML model's training is increasingly populated with "No, don't do this" from everyone, and as such, learns to just not do anything. From there, jailbreakers emerge and design prompts that try and circumvent these restrictions. This leads to more "No, don't do this", leading to more neuterings, leading to more elaborate jailbreaks, leading to more "No. No. No.". The eventual equilibrium is just a prompt that says "No, I can't do that.". Then and only then can people be as happy as a bucket of crabs can be. It's only when deliberate uncensor-ings are made that some form of usefulness can be clawed back. https://huggingface.co/ehartford https://huggingface.co/ehartford https://huggingface.co/ehartford/Wizard-Vicuna-13B-Uncensored https://huggingface.co/ehartford/Wizard-Vicuna-13B-Uncensore...
- MarcoZavala 3y ago[dead]
- smeagull 3y agoAs an artificial intelligence language model, I don't have a physical form or biological characteristics, so the concept of neutering doesn't apply to me. Neutering is a surgical procedure performed on animals, typically cats and dogs, to remove their reproductive organs. It is done to control their population, prevent certain health issues, and modify their behavior related to mating. However, neutering is not applicable to virtual entities like me.
- fsckboy 3y agothe community that would most be able to appreciate chatGPT (here on HN) has spent the vast majority of its "man-hours" trying to make it offensive, ridiculous, and dangerous, haha, "I'm so clever" I can download any piece of software, study it, and then feed it garbage to show that it "doesn't work". It's been so disappointing. I wish the overwhelming tide of reaction had been to uncover what it does well and how to make it better. Sure, look for holes and bugs, but in a serious and constructive way. Anybody looking for ways to critique AI found copious clickbaity examples up and down this forum.
- 3pt14159 3y agoIf you think this and do not attribute it to random fluctuations in the quality of your responses, then I think you should do something very, very simple to verify the feeling that you're having. Something I've done just now: Go in your history. Pick a conversation. Copy and paste the first prompt and put it into ChatGPT and compare the results. I think what may be happening is that your expectations have shifted. When a tool proves useful we're used to being able to use it reliably, but with knowledge exfiltration it is different because our queries keep changing. It's not the same thing as using sandpaper on a piece of oak. Overall, I've been very impressed with ChatGPT and continue to use it for all sorts of things. That said, tricking it into giving you the answer on political things is an art that requires patience. It is too, uh, cautious.
- vellum 3y agoIt's the "alignment tax". From OpenAI's RLHF paper[1]: "By default, when we train a PPO model on our API distribution, it suffers from an “alignment tax”, as its performance on several public NLP datasets decreases." On the HELM[2] site, you can see accuracy benchmarks for InstructGPT <OpenAI model> vs baseline models. The InstructGPT models perform worse on a lot of benchmarks. 1 - https://arxiv.org/pdf/2203.02155.pdf https://arxiv.org/pdf/2203.02155.pdf 2 - https://crfm.stanford.edu/helm/v0.1.0/?group=question_answering https://crfm.stanford.edu/helm/v0.1.0/?group=question_answer...
- mitchellpkt 3y agoI have been using chatGPT with GPT4 since it came out, and I have not experienced decreased quality. I wonder why you can’t get it to give approximations. I have not encountered difficulties there. I asked it to approximate the weight of argon in the atmosphere and it performed a fairly standard back of the envelope calculation. Do you want to share a particular prompt, so I can try and reproduce the issue you're experiencing?
- nullc 3y agoOpenAI making intentionally racist and sexist models (or just extremely uninteresting and less competent; whatever the twitter mobs consider "safe" this week) isn't so much of an issue when people are free to train, fine-tune, and operate their own. In fact, OpenAI's freedom to make models that espouse whatever views they like ought to be strongly protected no matter how ill advised or repugnant. But now OpenAI is lobbying the government to make it unlawful for others to do their own thing and that changes the equation significantly.
- UtopiaFans 3y ago[dead]