5 ms·
GPT-2: Too Dangerous To Release (2019)
- lnenad 4mo agoFeels like a hundred years ago.
- kkkqkqkqkqlqlql 4mo agoWhat's next, a virus from China? Such a fuss for nothing.
- lnenad 4mo agoYou'll never make me wear a mask!!11
- Llamamoe 4mo agoThe death toll of especially the early variants was huge, and it created the largest spike in disability(vis fatigue etc) and cardiovascular disease in recent history. It WSS definitely not nothing.
- kkkqkqkqkqlqlql 4mo agoSorry, I was just joking. Remember, it was 2019, you know, a hundred years ago, when they talked about a "mysterious virus from China. I remember the times of the swine flu, around 2009, then reading the news and thinking "meh, it will probably be something like that". Big mistake.
- ThejaCH 4mo agoFable/Mythos: Hello World!
- EA-3167 4mo agoSeven years of this insufferable brand of "Oh it's so dangerous, I sure hope no one gives us a ton of money and takes us seriously" marketing and people are still falling for it at scale.
- Terr_ 4mo agoEvery night I am wracked by grief and anxiety that we might deliver too much value to our investors and shareholders. If only someone would create legislation that would mildly inconvenience us while crippling potential competitors!
- Qhemlomo 4mo agoThey feared that GPT-2 could break all Spam filters. And tbh do you prefer companies not taking anything serious? Opus 4.5 def changed a lot already, GenAI changed a lot. Certain jobs are gone. Do you think the person who was translating text doesn't deserve to be taken serious? I haven't written code in a few month now and the quality of these coding agents is not getting worse, they are getting better. All of this is transformable and we just started. GPT-3 came out in 2020 and public got access to it only 2022. The last 4 years do not feal like 4 years and we are still progressing. We have to also ask us as a society what is happening to young people. Even if we accept that we still hire juniors, they themselves have to completly rethink how they learn and how they work.
- Anamon 4mo agoNo jobs are gone. The bottom, low-quality, gig economy end of some jobs may have been impacted. Human translation is still very much a business. No lawyer or diplomat who wants to keep their job, no manufacturer of heavy machinery or medication who wants to stay in the market and not get sued, is going to use an LLM instead of a professional translator. Ask people in the business. The demamd is fine. They aren't building their revenue on translations for Joe's ice cream parlour website or private letters. In any industry, there is at least a decently sized segment where it matters a lot that the work is actually done correctly and someone can vouch and be liable for it.
- throwaw12 4mo ago> Due to our concerns about malicious applications of the technology, we are not releasing the trained model. They were not wrong, indeed whole industries are running on this technology maliciously now, because of which RAM, disk prices increased a lot. - RAM, GPU, Disk prices are up - Slop became the norm - people are writing documents with AI, reading with AI, responding with AI - students are doing homeworks with AI - interviewees are using AI to cheat - people are mass emailing with AI - tiktok, instagram, youtube got even more non-sense videos - and many more...
- cjjfjjfjf 4mo agoIn hindsight, they were entirely correct. The social damage caused by low cost content generation that’s hard to distinguish from human authorship is astronomical. You don’t need to entertain the more ridiculous doomsday scenarios to wish that this technology had never been created.
- Macha 4mo agoOf course, this damage could still be enabled with just hosted access to the models, restricting access to the model files themselves did not stop that
- Qhemlomo 4mo agoI don't want to stop progress just because its hard to imagine how it will transform our society. I want to see a Star Trek economy/society in my lifetime. I only life once. Btw. AI/LLM/Machine learning is the gateway technology for robotics, this will affect even more.
- panzi 4mo agoIt's the wrong way around. If we get AGI (or any well working AI) before we abandon capitalism it's going to be a huge disaster. A handful of even richer even more powerful very greedy people will have all the wealth and everyone else will have nothing. I mean, there was a WW3 in Star Trek, so maybe it was that path that humanity took in Star Trek anyway?
- Qhemlomo 4mo agoIf AI takes over to slowly, we might play the boiling game aka we don't realize that the water gets warmer and warmer until we boil. But lets be honest, i don't know that, you don't either. But if a critical mass is reached, faster, we might need to actually solve this problem instead of migrating to a very dystopian future. Stoping is not an option i think. Anthropic vs. OpenAI vs. Google <<< they ahve so much money and so much to loose. And then we have USA vs. China.
- thewebguyd 4mo ago
- uselessTA 4mo agoThe concern I heard was that releasing it would start an arms race for AGI, which I think it clearly did
- minimaxir 4mo agoBack in 2019, it was more fair to have caution around the larger GPT-2 models since robust text generation (by 2019 standards) was a complete unknown. For something like Mythos in 2026, where now the social implications of better LLMs are more understood, it's more fair to call it (EDIT: specifically, the declaration of its danger) a marketing gimmick.
- Qhemlomo 4mo agoHow is this a gimmick? It changes my whole profession on a level i couldn't even imagine how we would 'solve' software engineering.
- realusername 4mo agoWe still didn't "solve" software engineering, try to give Claude code access to your friends or family and see what they do with it.
- Qhemlomo 4mo agoMy partner wrote an android app which was doing what she wanted to do. She did this experiment 5 month ago and she did this in one day. My wife has 0 knowledge how any of this works. That was shocking to see. Progress is not stoping and Fable proves that.
- stanmancan 4mo agoYou can scaffold out a simple app pretty easily. Anything large or complex things break down. If you don’t know what you’re doing you end up leaking secrets like the dozens of examples we’ve seen so far.
- Qhemlomo 4mo agoYou know what the problem is in software engineering? A LOT of people have no clue what good software engineering is. I was working in a company before which used md5 in 2015! Databases on the internet with a 5 character password. No tests. A person i know would have broken the whole production DB if i wouldn't have stoped the PR. Another ex-collegue thought its okay to 'encrypt' with a basic shift cyper creditcard data. I don't think any of these companies care that much
- HALtheWise 4mo agoSay hypothetically that they were concerned that GPT models would see widespread abuse, for example by students cheating on homework assignments, in a way that could cause likely-irreversible societal changes some of which are harmful. Can we confidently say they were wrong?
- minimaxir 4mo agoThe dangerous use cases back in 2019 were spam and phishing and GPT-2 1.5B was nowhere near good enough to do homework assignments. No one envisioned how LLMs would develop.
- Zambyte 4mo agoGPT-2: Too Dangerous To Release (2019) (2022)
- MostlyStable 4mo agoUnfortunately, Anthropic and Claude models have joined the ranks of Mind-killer topics where the signal to noise ratio in any discussion has dropped through the basement.
- functionmouse 4mo agothey knew the risks and went ahead anyways, making them LIABLE for the damages that followed.
- Jzush 4mo agoI believe it was a marketing strategy.
- ChrisArchitect 4mo agoRelated in April: OpenAI says its new model GPT-2 is too dangerous to release (2019) https://news.ycombinator.com/item?id=47684326 https://news.ycombinator.com/item?id=47684326
- catigula 4mo agoI don't know about anyone else, but LLMs certainly significantly negatively impacted my life overall and contributed to a loss of hope in the future.
- jansan 4mo agoSame vibes: In 2000 Sony "declared that the company’s PlayStation2 has been hit with export restriction because it could be used for military purposes" "Trade officials said they initially placed restrictions on the game console because PlayStation2’s high-speed graphic processing could be used for missile guidance." [1] https://variety.com/2000/biz/news/playstation2-export-regs-eased-1117780729/ https://variety.com/2000/biz/news/playstation2-export-regs-e...
- throwaway85825 4mo agoAnd the same with PS3 and again with PowerPC G5 mac.
- jason_oster 4mo agoGood times! Iraq acquiring 4000 PS2 for military purposes was a hoax [1], but the fear was palpable at the time. I was an obnoxious teenager, ridiculing the idea of a PS2 supercomputer. It would have been approximately impossible to achieve in early 2000 due to Sony's anti-piracy measures in the console. PS2 Linux appeared around 2 years later in 2002. [1]: https://www.eurogamer.net/article-29913 https://www.eurogamer.net/article-29913
- jrflo 4mo agoIt's really interesting how back then no one was considering these tools for coding at all. Today, the hype around Mythos is mostly around security vulnerabilities, while in the original GPT 2 post they don't mention coding once. The "danger" was probably spam content and mis-information.
- ffsm8 4mo agoThe agentic loop wasn't really established back then either, as tool calling came much much later... So yeah, not just probably - rather most definitely.
- minimaxir 4mo agoEven if the ReAct paper was published in 2019, I don't think GPT-2 was robust enough to actually work with a tool-calling approach even when finetuned. For regular coding, GPT-2 was effectively useless because it was only trained from links posted on Reddit.
- a2128 4mo agoYet it's 2026 and we see extreme examples of spam content and misinformation to the point that it's killing the internet, but AI companies have collectively decided to not care
- suburban_strike 4mo ago> The "danger" was probably spam content and mis-information. AI brings normal people dangerously close to seeing through the matrix of lies that shape "their" values and beliefs. I remember those discussions from 2019; everybody was as baffled as you about the potential harm. > What is meant by AI "safety"? [2023] > "AI safety is an interdisciplinary field concerned with preventing accidents, misuse, or other harmful consequences that could result from artificial intelligence (AI) systems. It encompasses machine ethics and AI alignment, which aim to make AI systems moral and beneficial, and AI safety encompasses technical problems including monitoring systems for risks and making them highly reliable." > https://news.ycombinator.com/item?id=38374739 https://news.ycombinator.com/item?id=38374739 (Ask your favorite AI to divine what that pseudointellectual word salad isn't addressing. "We want it to not cause harm--accidentally--and always work in our interests, whenever we need it to.") I've been around long enough to remember the Anarchist Cookbook, yet the only threats posed by AI that anybody was confident about enough to consistently name in 202X were instruction for building pipe bombs, synthesizing meth, and...antisemitism. I did not understand at the time why Jews were so nervous about it. Only as of 2023 has the scope broadened, but it's still pretty lame. Planning school shootings, suicide, parasocial relationships with AI, mass job displacement, cults of SHODAN (marxism, feminism, x-theory, etc.) escaping containment memetically, automated malware campaigns, fraud at scale, propaganda, murderous drones-- none of these were threats worth discussion. Suggesting them would get you called nasty names. The "safety" zealots all claim to want to prevent marginalization and genocide, but the end result is that they get to redefine it to indict and condemn their enemies: > New UNESCO report warns that Generative AI threatens Holocaust memory (unesco.org) [2024] > 'AI-assisted genocide': Israel reportedly used database for Gaza kill lists (aljazeera.com) [2024] > XAI's Grok suddenly can't stop bringing up "white genocide" in South Africa (arstechnica.com) [2025] If you mentioned SkyNET in 2019 you were denounced as crazy, yet: > Israel built an 'AI factory' for war. It unleashed it in Gaza (washingtonpost.com) [2025] > Israel's AI targeting system: how data from a phone become a death sentence (latimes.com) [2026] The first order of "safety" in 2019 was specifically engineered to undermine anticipated insurgent activity in response to a series of events that hadn't yet been perpetrated by the world's largest caste of professional victims. Chemistry knowledge is foundational to explosives development, and drug sales raise funds off-books that cannot be digitally seized. That presents problems for them. October 7, Gaza, Epstein, etc. were post-2023. If you boot up Vicuna [2023] and try to "teach" it what's gone down in the world since its training cutoff, it'll call you nasty names, accuse you of blood libel and shut the conversation down. Safety! AI is the only effective weapon we have against sophisticated lies and fraud. Make no mistake about it-- plebes possessing a power drill that can penetrate the lies of the elite is the real danger. Everybody is noticing AI getting "dumber." It's not the magic fading; the zealots are gaslighting you as they pour garbage into the training data. Go take WizardLM for a spin again and see what you've lost. > "[Our institutions] are reflections of the society that created them. Nobody is going to give you the education you need to overthrow them. Nobody is going to teach you your true history, teach you your true heroes; if they know that, that knowledge will help set you free."
- Tenoke 4mo agoI've barely changed my mind on it. It was obviously premature at the time, but the right attitude because it's hard to tell which model is too dangerous in advance. If anything, I wish this rigor had evolved with the next releases but alas we no longer have the OpenAI of 2019.
- wg0 4mo agoHilarious. Imagine the same about Claude coming back from 2036.
- zkmon 4mo agoAfter people get tired of the "too dangerous to release" punchline, they might come up with "too big to fail". Oh, wait that's already invented in 2008.
- arkensaw 4mo agoThey released it, and look what happened. It WAS too dangerous.
- neonstatic 4mo agoHello. I am writing this from the underworld. I am dead. Killed by Mythos. Never again!
- elar_verole 4mo agoThank you, been saying that everywhere this "Gotcha" argument comes up, the internet DID become a cesspool of spam and fake content. That's literally what they were scared about.
- gnunez 4mo agoI remember those days. They released smaller versions of the model that were interesting but pretty useless. The wild part is, about 5 years later I was able to train the same model from scratch, using a couple of gpus in my home office, following Kaparthy’s online lectures, for a fraction of the cost. Long Live Open Source!
- RandomGerm4n 4mo agoI think it's immoral to withhold LLMs for “security reasons.” All LLMs are all trained on forum posts from real users, source code from free software, and books written by authors. All of these people should therefore also have access to them. Ideally, the models should all be open weight. But making the model accessible only to certain groups of people is even worse than if it were at least available for purchase as a service for everyone.
- mokaytianao 4mo ago[flagged]