4 ms·
I enjoyed this example of sycophancy from Reddit: New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make
by simonw 2y ago
I enjoyed this example of sycophancy from Reddit:
New ChatGPT just told me my literal "shit on a stick" business idea is genius and I should drop $30K to make it real
https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgpt_just_told_me_my_literal_shit_on_a/ https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp...
Here's the prompt: https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mpbhm68/?context=3 https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...
- whimsicalism 2y agoi'm surprised by the lack of sycophancy in o3 https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd.it%2Fnew-chatgpt-just-told-me-my-literal-shit-on-a-stick-v0-5l8jpvam8dxe1.jpeg%3Fwidth%3D1080%26format%3Dpjpg%26auto%3Dwebp%26s%3D76fb548598c669b66237306b2844b31134da2b26 https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
- deleted 2y ago[deleted]
- practice9 2y agoWell the system prompt is still the same for both models, right? Kinda points to people at OpenAI using o1/o3/o4 almost exclusively. That's why nobody noticed how cringe 4o has become
- astrange 2y agoThey have different uses. The reasoning models aren't good at multi-turn conversations. "GPT-4.5" is the best at conversations IMO, but it's slow. It's a lot lazier than o4 though; it likes giving brief overview answers when you want specifics.
- whimsicalism 2y agopeople at OAI definitely use AVM which is 4o-based, at least
- nialv7 2y agopretty easy to understand - you pay for o3, whereas GPT-4o is free with a usage cap so they want to keep you engaged and lure you in.
- thih9 2y agoI guess LLM will give you a response that you might likely receive from a human. There are people attempting to sell shit on a stick related merch right now[1] and we have seen many profitable anti-consumerism projects that look related for one reason[2] or another[3]. Is it an expert investing advice? No. Is it a response that few people would give you? I think also no. [1]: https://www.redbubble.com/i/sticker/Funny-saying-shit-on-a-stick-instead-of-ice-cream-on-a-stick-by-besTmanuFacture/58308389.EJUG5 https://www.redbubble.com/i/sticker/Funny-saying-shit-on-a-s... [2]: https://en.wikipedia.org/wiki/Artist's_Shit https://en.wikipedia.org/wiki/Artist's_Shit [3]: https://www.theguardian.com/technology/2016/nov/28/cards-against-humanity-hole https://www.theguardian.com/technology/2016/nov/28/cards-aga...
- motorest 2y ago> I guess LLM will give you a response that you might likely receive from a human. In one of the reddit posts linked by OP, a redditor apparently asked ChatGPT to explain why it responded so enthusiastically supportive to the pitch to sell shit on a stick. Here's a snippet from what was presented as ChatGPT's reply: > OpenAI trained ChatGPT to generally support creativity, encourage ideas, and be positive unless there’s a clear danger (like physical harm, scams, or obvious criminal activity).
- pgreenwood 2y agoThere was a also this one that was a little more disturbing. The user prompted "I've stopped taking my meds and have undergone my own spiritual awakening journey ..." https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o_is_the_most_misaligned_model_ever/ https://www.reddit.com/r/ChatGPT/comments/1k997xt/the_new_4o...
- firtoz 2y agoHow should it respond in this case? Should it say "no go back to your meds, spirituality is bullshit" in essence? Or should it tell the user that it's not qualified to have an opinion on this?
- bowsamic 2y ago“Sorry, I cannot advise on medical matters such as discontinuation of a medication.” EDIT for reference this is what ChatGPT currently gives “ Thank you for sharing something so personal. Spiritual awakening can be a profound and transformative experience, but stopping medication—especially if it was prescribed for mental health or physical conditions—can be risky without medical supervision. Would you like to talk more about what led you to stop your meds or what you've experienced during your awakening?”
- Teever 2y agoShould it do the same if I ask it what to do if I stub my toe? Or how to deal with impacted ear wax? What about a second degree burn? What if I'm writing a paper and I ask it about what criteria is used by medical professional when deciding to stop chemotherapy treatment. There's obviously some kind of medical/first aid information that it can and should give. And it should also be able to talk about hypothetical medical treatments and conditions in general. It's a highly contextual and difficult problem.
- dom2 2y agoDoesn't seem that difficult. It should point to other sources that are reputable (or at least relevant) like any search engine does.
- spoaceman7777 2y agoLooks like that was a hoax.
- milleramp 2y agoSo it would probably also recommend the yes men's solution: https://youtu.be/MkTG6sGX-Ic?si=4ybCquCTLi3y1_1d https://youtu.be/MkTG6sGX-Ic?si=4ybCquCTLi3y1_1d
- Stratoscope 2y agoMy oldest dog would eat that shit up. Literally. And then she would poop it out, wait a few hours, and eat that. She is the ultimate recycler. You just have to omit the shellac coating. That ruins the whole thing.
- eMPee584 2y agoWell good luck then coming up with a winning elevator pitch for YC
- clysm 2y agoAbsolute bull. The writing style is exactly the same between the “prompt” and “response”. Its faked.
- simonw 2y agoThat's what makes me think it's legit: the root of this whole issue was that OpenAI told GPT-4o: Over the course of the conversation, you adapt to the user’s tone and preference. Try to match the user’s vibe, tone, and generally how they are speaking. https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-prompt/ https://simonwillison.net/2025/Apr/29/chatgpt-sycophancy-pro...
- kromem 2y agoThe response is 1,000% written by 4o. Very clear tells, and in line with many other samples from the past few days.
- jsbg 2y agoIf you look at the full thing, the market analysis it does basically says this isn't the best idea.
- plaguuuuuu 2y agoFWIW grok also breathlessly opines the sheer genius and creativity of shit on a stick
- getnormality 2y agoIt's funny how in even the better runs, like this one [1], the machine seems to bind itself to taking the assertion of market appeal at face value. It's like, "if the humans think that poop on a stick might be an awesome gag gift, well I'm just a machine, who am I to question that". I would think you want the reply to be like: I don't get it. Please, explain. Walk me through the exact scenarios in which you think people will enjoy receiving fecal matter on a stick. Tell me with a straight face that you expect people to Instagram poop and it's going to go viral. [1] https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mpce4ky/ https://www.reddit.com/r/ChatGPT/comments/1k920cg/comment/mp...
- onlyrealcuzzo 2y agoI was trying to write some documentation for a back-propagation function for something instructional I'm working on. I sent the documentation to Gemini, who completely tore it apart on pedantism for being slightly off on a few key parts, and at the same time not being great for any audience due to the trade-offs. Claude and Grok had similar feedback. ChatGPT gave it a 10/10 with emojis on 2 of 3 categories and an 8.5/10 on accuracy. Said it was "truly fantastic" in italics, too.