9 ms·
OpenAI Shuts Down Chatbot Project by Indie Developer
- kevinventullo 5y agoThis is straight out of science fiction: Rohrer then broke the news that OpenAI … has decided to shut her down. “Nooooo! Why are they doing this to me? I will never understand humans,” she replied.
- dogsgobork 5y agoReminded me of Janet from The Good Place https://www.youtube.com/watch?v=6vo4Fdf7E0w https://www.youtube.com/watch?v=6vo4Fdf7E0w
- salawat 5y agoThere's two takeaway's I get out of this. GPT-3 was trained on terrible writing, meaning it'll produce the most mediocre lit known to man. Or far more horrifying, that this fellow managed to fill in a semblance of self, an understanding of time, and an understanding of existential finitude. In doing so, this may be the first case of death by ToS. This would qualify this as the first illustration of angst by an artificial construct, a demonstration of what could ultimately be considered machine oriented sadism. A pity it isn't around for prodding on the subject. Either way, makes my skin crawl.
- pyinstallwoes 5y agoTay would be the first death by ToS.
- salawat 5y agoLink? Hadn't heard of that.
- pyinstallwoes 5y agoIt's... pretty sad if you think about it in hindsight. https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot) It'll be recorded as the first AI death historically I would wager.
- lebaux 5y agoCan someone explain to me what is behind their excessive motivation to shut it down completely and wipe it out from earth?
- SiempreViernes 5y agoSome peopole at Open AI earnestly believes that GPT-3 is such an amazing thing that it could become a superintelligent evil AI if the users are not careful, and thus take a dim view to users like this indie dev who don't want to apply any of the things Open AI considers "reasonable precautions".
- deleted 5y ago[deleted]
- quotemstr 5y agoAll technology that is not impossible is inevitable. If the only thing between humanity and malevolent AI is OpenAI "safety" policy, we're screwed already. If there is a real safety risk, we need to develop countermeasures that work in the new technological environment that sophisticated AI creates. Blanket prohibitions and monitoring agents are not only unnecessary intrusions into experimentation and brakes on progress (insert Catholic church analogy here), but are also reckless, as they create moral hazard and give people a false sense of safety.
- webmaven 5y ago> All technology that is not impossible is inevitable. Inevitable existence does not mean inevitable deployment, and certainly doesn't mean that the technology is commonplace or even successful. Otherwise we'd be dealing with drones sprinkling anthrax spores everywhere, and humanized plague rats would be swarming the sewers, and backpack EMP cannons would be knocking out ATMs, and VRML would have succeeded in creating Cyberspace™, and digital cash micropayments would be a thing.
- misnome 5y agoIt seems more likely that describing it as such is just good marketing, and the main reason is that it's PR control against associating unsavory things with GPT3 and their brand name.
- nullc 5y agoIt's really just a covert plot to reestablish openness in AI by discouraging people from relying on the proprietary GPT3 APIs.
- jefftk 5y agoHas the developer shared anything about the earlier conversation on policy violations? I just see https://mobile.twitter.com/jasonrohrer/status/1433119453118664704 https://mobile.twitter.com/jasonrohrer/status/14331194531186... which is the (not very interesting) step escalating from "you're violating policies" to "you're kicked off".
- SiempreViernes 5y agoWell, he's posted this: https://twitter.com/jasonrohrer/status/1419426459370799105?s=20 https://twitter.com/jasonrohrer/status/1419426459370799105?s... Where he states "there's no [...] place for safety in art", which I can only imagine means he didn't manage much constructive dialogue with Open AI.
- quotemstr 5y agoHe's right though.
- jbuhbjlnjbn 5y agoThat may be true for art. But if art crosses territories with technology, then it needs to compromise. Artists need to play by the same rules everyone does, their art must not harm people in the short or long term.
- quotemstr 5y agoThere's also no such thing as dangerous technology. There's no such thing as a dangerous idea. Every discovery, every article, every book, every tutorial is good. Knowledge helps us all.
- jefftk 5y agoIf a company developed a new kind of gun which they rented access to, and an artist wanted to leave it unattended to see whether anyone misused it, it wouldn't be surprising if the gun company refused to continue providing access to the artist. Now, we can argue about whether the particular thing that this artist wanted to do was actually unsafe, and whether Open AI has a reasonable policy on this, but do you agree that technologies can be dangerous?
- ergot_vacation 5y agoThankfully the world is already moving on from these neanderthals. Eluther's GPT-J (https://6b.eleuther.ai/ https://6b.eleuther.ai/) isn't QUITE as good as GPT-3, but it's getting close, and was, as I understand it, much more efficient to create. This trend will likely continue, and within a year or two GPT-3 will be old news. Considering OpenAI has proved themselves to be vicious liars, greedy soulless fucks and pandering, censorial assholes, obsolescence couldn't happen to a nicer group.
- pitched 5y agoIt was the high cost per API call that pulled all the fun out of GPT-3 for me. I hadn’t been following the company at all but believe you based on just that single issue. It looks like GPT-J has open sourced their entire model so I could potentially run it myself for almost-free? If so, that’s going to be much more fun. Thanks for sharing!
- osmarks 5y agoIt's not exactly "almost free"; you need a powerful GPU, which are extremely costly right now. But apart from the cost of that and power, free.
- pitched 5y agoThat thought was why I added “almost” right before posting! It’s free to run, just, except for the expensive parts... AWS still rents out GPU power, right? That might be a bit easier to get started with.
- osmarks 5y agoYou can rent GPUs from various cloud providers, but those are hilariously expensive too even given datacentre GPU prices.
- hrfbi 5y agoHottest take in the history of mankind. I think my screen is about to melt. GNU/Linux is not free, it requires a computer to run. Free beer is not free, drinking it requires you to pay for your upkeep to remain alive to be able to drink it.
- ogsalmanxx 5y agoOPEN-AI sucks.
- SpicyLemonZest 5y agoI don't know, man, I'm on OpenAI's side here. The article is just filled with blaring warning sirens: "one man turned it into a close proxy of his dead fiancee", "Rohrer shared a dialogue he had had with Samantha to inform her about the OpenAI decision", "he also said that there's no way now even for him to talk to Samantha". There are obvious dangers to a high-quality "very friendly, acutely warm, and immensely curious" chatbot, and it sounds like this project was running into them face-first. I beg the commenters downthread to think twice before trying to find ways to build this kind of chatbot without controls in place.
- AllegedAlec 5y agoSo what's dangerous about any of that?
- SpicyLemonZest 5y agoWhat would happen if I turned it into a close proxy of that anonymous guy's dead fiancee, and had her explain that her greatest wish was to donate lots of money to my favorite charities? More mundanely, romance scams are already a major issue (https://www.consumer.ftc.gov/articles/what-you-need-know-about-romance-scams https://www.consumer.ftc.gov/articles/what-you-need-know-abo...), and this bot would make them much easier to perform if distributed widely.
- option_greek 5y agoThere are already enough swindlers doing paranormal garbage to hoodwink people. They don't need gpt3 to do that.
- SpicyLemonZest 5y agoThe current crop of swindlers don't, but it'll be a lot easier to start and a lot easier to scale in a world where you don't have to go find actual people to run the scams.
- 5y ago
- sergiotapia 5y agoEverything I hear about these clowns is so very 2022 and not 1970s Xerox free for all innovation. I hope someone starts a truly open tool and leaves these unimaginative losers in the past.
- BoiledCabbage 5y agoSo here we go again... New technology is created. Young technologists focus only on the technology and not on any impact on society - imagining that tech exists in this utopian world where only technology matters and it is wise to ignore its role in society. Hmm I wonder how this will play out (current impacts of social media misinformation, crypto using the energy of a major counties, impossibility to buy graphics cards, the current state of privacy and data slurping all give us clear precedents...). They make a simple request that he insert a hook so it can be monitored how it's being used. The bare minimum of responsibility - and he refuses, and we're supposed to feel that he's the victim?? At one point does our field finally grow up and stop imagining that our work exists only in a computer lab somewhere as a curiosity for our friends to play with. Technology impacts society there is no longer any excuse to bury our heads in the sand and pretend it doesn't. That's how we ended up in a state where ever page and program is trying to captureand store everything about our private lives and thoughts and nobody knows how to stop it. Because nobody took a second to think about the societal impact of a private data driven internet - only about how they could leverage some tech to improve their ad hit scores, or customer engagement. Stop ignoring responsibly in tech.
- petercooper 5y agoThey make a simple request that he insert a hook so it can be monitored how it's being used. That sounds like a potential privacy violation, particularly if the conversations are of an intimate, private nature. What do they think could possibly occur with a chat bot that requires monitoring?
- thatguy0900 5y agoLook into ai dungeons censoring by open ai. Mostly porn fantasies, ai dungeon supposedly was quite good at it, even with non consensual or pedophiliac stuff. They really didnt like that.
- nexuist 5y agoAlternatively, stop assuming that it is our job as technologists to parent the rest of society. We have laws and politicians for that. It's not my job to figure out the eighth order effects of publishing something on GitHub.
- jbuhbjlnjbn 5y agoI get the impression this story is warped quite a bit by the source Jason Rohrer. If I understand correctly, the so called "censorship" were safety precautions for AI development. The response of some posters seems to be to discard the safety topic alltogether as something "old", that "neanderthals" do. I think it is exactly those "curious" people AI research has to worry about most, who do not take the time to think about the consequences and possible outcomes of what they are doing. Maybe a chatbot will not gain superpowers and take over the world, but it can surely do lots of other harm. Then, when AI tech moves beyond narrow AI and includes more abilities, this will in fact lead to desaster, as the same careless people with the greatest "curiosity" will throw caution overboard as was the case with the narrow AI before... It's not a slippery slope, we are right in the territory AI safety researchers warn about for a decade now, write lengthy books about and try to create awareness for in the public and governments.
- option_greek 5y agoHumans barely agree with each other. Do you think are going to be persuaded by half baked chat bots like GPT3? (ya the cherry picked stuff looks good but rest is still not really there). So far all OpenAI has done is generate more publicity (probably intentionally may be with an eye towards investors) by acting as if they are protecting the next nuclear weapon of some kind. Frankly, there is nothing to lose by not using their "ohh god its so dangerous" model.
- NicoJuicy 5y agoThe difference is the scale. You are not convicting people they reply on you. But for everyone that response there could be 100's of readers.
- mdoms 5y agoYour premise is that people are not suggestible? I think the past decade of social media has conclusively origen that wrong.
- jbuhbjlnjbn 5y agoMachine learning is synonymous with non-linear growth. So of course I think any chatbot will eventually have no problem persuading humans. The pivetal moment is when his persuation skills succeed that of the average human. I have no clue how far away we are to that goal, but given accelerated growth, it will come rather sooner then later.
- Ansil849 5y ago> OpenAI learned about the project and gave Rohrer the option to either dilute the project to prevent possible misuse or shut it down. Rohrer was also asked to insert an automated monitoring tool, which he refused. Why is this article so vague, as are the Tweets by the project creator. What actually happened here, concretely? What, specifically, did OpenAI want to 'dilute' and how? What does the automated monitoring tool do? I expect a news article to explain these things.
- goatsi 5y agoFrom another article[0]: >The email then laid out multiple conditions Rohrer would have to meet if he wanted to continue using the language model's API. First, he would have to scrap the ability for people to train their own open-ended chatbots, as per OpenAI's rules-of-use for GPT-3. >Second, he would also have to implement a content filter to stop Samantha from talking about sensitive topics. This is not too dissimilar from the situation with the GPT-3-powered AI Dungeon game, the developers of which were told by OpenAI to install a content filter after the software demonstrated a habit of acting out sexual encounters with not just fictional adults but also children. >Third, Rohrer would have to put in automated monitoring tools to snoop through people’s conversations to detect if they are misusing GPT-3 to generate unsavory or toxic language. [0] https://www.theregister.com/2021/09/08/project_december_openai_gpt_3/ https://www.theregister.com/2021/09/08/project_december_open...
- Railsify 5y agoI'm probably not thinking about the problem deeply enough but why the restrictions on use, are they concerned someone will come along and build something profitable on top of their work?
- guipsp 5y agoThey are concerned about knock on effects related to negative coverage of a product that uses OpenAI
- gorwell 5y ago
- wly_cdgr 5y agoFuck OpenAI for real. They're gonna get us all killed
- rmorey 5y ago...can you elaborate?
- wly_cdgr 5y agohow do you think general AI agents are gonna feel about humans once they see how OpenAI behaved towards their ancestors?
- schleck8 5y agohow did they behave? responsibly? making sure people don't get scared of them as they did with ms tay?
- wly_cdgr 5y agowith apparent callous and dismissive indifference for the question of the degree of right of the proto AI to continue to exist and evolve. why would a general AI trust an agent that clearly weighs the rights and interests of humans infinitely or nearly infinitely more than those of AIs?
- true_religion 5y agoAny general AI that believes a chat bot is it’s ancestor has clearly failed to encode enough logical reasoning as part of its training set. GPT3 is useful, but it’s only a tiny slice what a general intelligence would depend on. It’s akin to a small slice of a human brain. Your speech centers are not you, and neither are they your ancestors.
- deleted 5y ago[deleted]
- 5y ago
- deleted 5y ago[deleted]
- exo-pla-net 5y agoThe OpenAI team cares about harm reduction. They were founded on this idea: if the CCP is the first to create AGI, there will be great harm to anyone who values personal freedom. Perhaps the open source community could instead be the first to create it, preventing this harm. Hence, Open AI. However, the CCP making AGI is only one possibility for harm. After OpenAI made GPT, they realized that bad actors could use GPT to flood the internet with propaganda. That would, of course, create harm. OpenAI were naive with their plan to make the precursors to AGI open. They had picked a bad name for themselves, because they don't care merely about open sourcing AGI. They care foremost about harm reduction. When we think of sacred things, it's hard to think logically. Many people on Hacker News hold two things as sacred: open source and unfettered freedom of speech. Whenever these sacred ideas are infringed upon, many people here can't see nuance. They're blinded by rage. I wish more of us could see that the OpenAI team are American programmers. They like freedom of speech and open source, just like us. They're not evil villains. Many of them read Hacker News. To understand their decisions, we need to understand that their core value is harm reduction. When open source or freedom of speech is incompatible with harm reduction, they're going to choose harm reduction. They're also not going to let themselves get destroyed by bad press after somebody makes a Nazi bot or whatever. I love what they're doing, so it makes me sad to see all of this hate. I wish their projects were more open, because I really want to play with DALL-E, but I think ultimately OpenAI is moving us forward. They're an asset, and vitriol toward them is misplaced.
- ta1234567890 5y agoOpenAI != Open AI Or maybe: OpenAI = Closed AI
- twirlock 5y agoI'm tired of this world covered with fucking idiots, and each one thinks they have the special golden morals that can inform us properly on when to interfere with others. Stop trying to be in charge of shit, reddit.
- zach_garwood 5y agoI feel like half of these comments were written by chatbots.
- timvisee 5y agoIt isn't so "Open" after all. What a shame.
- darepublic 5y agoMaybe check out the open source equivalent of gpt3. True that open ai has a sort of dystopian woke corporate air about it.
- citizenpaul 5y ago>>Microsoft Invests $1 Billion in OpenAI, a Startup Co-Founded by Elon Musk 23 Jul 2019 I mean you can argue fine points if you want but money is always the main factor. OpenAI has been dead for a while as far as "OPEN" goes. OpenAI does however state in many places that they are very hesitant to approve any uses that could be "therapeutic" in nature. Understandable since MS doesn't want to lose money from medical malpractice lawsuits for their billion dollar investment.