6 ms·
The AI That's Too Dangerous to Release
- deleted 7y ago[deleted]
- Animats 7y agoOh, it was trained on Reddit. That makes it clearer why it comes up with coherent blithering.
- parksy 7y agoIf it was trained on the entire body of scientific knowledge (or you know, as close as possible), would it be able to spit out useful correlations by interpolating? If trained on massive sets of source code and briefs (or perhaps a set of unit tests), could it spit out functioning code based on a problem statement? I'm just not well versed enough with ML to know if these things are a possibility or if it will just remix its training data in probabilistically correct, not-guaranteed-to-make sense ways?
- burtonator 7y agoI'm unclear but considering that the text output is nonsense I imagine the code would be nonsense. It MIGHT compile but might not actually do anything. I'm curious about this myself. Maybe it could transcompile or re-implement common algorithms to already solved problems.
- ppod 7y agoThis has been tried. The code looks kind of plausible, but doesn't quite compile. Some of the other examples (Shakespeare, Latex, wikipedia source code) are pretty good at a structural level. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ This had about 10 million parameters though, compared to OpenAi's 1.5B. I don't think their text is nonsense though. There's some very interesting examples. They have since released publicly a 345M parameter version, there are some nice examples from that model pre-trained with modern poetry on this twitter account: https://twitter.com/rossgoodwin https://twitter.com/rossgoodwin
- toomuchtodo 7y agoIt seems we have enough computing power to let these ideas run wild and see what weighted values come out?
- markedathome 7y ago> If trained on massive sets of source code and briefs (or perhaps a set of unit tests), could it spit out functioning code based on a problem statement? Microsoft's latest Visual Studio 2019 beta (v3) has intellicode[0] support which is supposed to ML "guess" 'code' based on equivalent github similarities. I don't think it is quite near to what you are asking, but certainly part of the way there. 0. https://visualstudio.microsoft.com/services/intellicode/ https://visualstudio.microsoft.com/services/intellicode/
- yellow_lead 7y agoCan't wait for those bugs caused by autocorrect.
- mercer 7y ago(typing) 'Robert (autocomplete) 'Robert'); DROP TABLE students;--
- LifeLiverTransp 7y agoLittle bobby tables we used to call him. I hope you learned from this to sanitize your database and your AI Input.
- Figs 7y agoAlmost certainly not. It will generate things that look like the text of scientific papers on a casual glance, but which will be dream-logic gibberish if you actually read it. The unicorn example makes no sense if you stop and think about what it's actually saying. e.g. a "unicorn" that has four horns? Solves a 200 year old mystery... about something just discovered? So close you could touch their horns... while viewing them from the air? It doesn't make sense if you actually think about it at all, but if you're not really paying attention, the style is like a news report. It's better at making up bullshit, but it's still bullshit.
- MegaButts 7y agoThis is a good point, and it's a little concerning to me that we will likely soon be inundated not just with spam from lowly-paid shills, but literal bots generating a near endless supply of confident-sounding nonsense. I am not an expert with machine learning, but it seems (at a very high level) we've done amazingly well at creating models that can recognize and sometimes recreate patterns. But they never seem to have any ability to understand the patterns. I'm sure someone much more knowledgeable could compare it to a child of whatever age (or maybe I'm just completely wrong).
- derefr 7y agoI fully believe that this is 90% of what the human brain is doing to solve the same problems, though. Brains just also have a constraint/filter layer, determining which of these bits of "made up bullshit" should be allowed to rise from the layer generating them, to the motor output layer. Or, to put that another way, flipping the perspective around: our minds could consist of an intelligent, analytical, but utterly unimaginative agent, that sits there listening to a stream of suggestions spewed out by a distinct second agent, one that is "creative" but has no idea about the constraints of things like physics. The brain's analytical agent filters this stream of suggestions, taking notice of the suggestions that seem like they'll make the world change in the ways it "wants"†; and then it does those. † Or, according to modern perceptual-control research, the agent attempts to predict the world that will occur a few seconds in the future, with a bias toward predicting world-states the reward-system has annotated as being rewarding; and then it looks at what motor commands it "would have" issued in that hypothetical world, and actually issues those. The stream of suggestions, in this model, serve as input to feed the generative model of potential world-states; the executive agent then must notice whether the potential world-state is a "possible" world or an "impossible" world (and whether the motor commands required of it are "possible" or "impossible" inputs), and filter out the "impossible" worlds. Under this hypothesis, dreaming is the state when your executive responsible for filtering out "impossible" worlds isn't online. So you just get a continuous "impossible" world generated from the streamed suggestions of the creative-but-stupid bullshit-generating agent, with nothing to tear it down—just as seen in these generative AIs. As consciousness returns, the mental predicted world-state is noticed to be impossible by the now-online analytical agent, and is torn down.
- joejerryronnie 7y agoIt can generate billions of memes per second. Who knew the first job to be automated out of existence was going to be internet troll?
- sideaccount 7y agoLex Fridman recently stated on the Rogan podcast that openai only open sources a basic version of their ai tools, is this true?
- criddell 7y agoYes, it's mostly true. Everything will eventually be released, they are just doing so in stages and are hoping to generate a broad discussion of the technology and it's implications. Some details are available here: https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/
- pigscantfly 7y agoA cynic might counter that this is not going to qualitatively change things and that there is a huge PR motive, but no one's a saint, you know?
- gwern 7y agoThere's two issues here. There's the models and there's the source code. The two smallest models have been released now, but it's unclear if/when OA will release any more publicly. They have released even less source code: all the code people have been using to train & finetune GPT-2 models has been implemented by third parties and OA has declined to release any code beyond simple sampling code, with no hint of even considering releasing training code in the future, much less releasing 'everything'. (And the sampling code isn't even that great; top-k sampling, for example, instead of standard beam search.)
- 6gvONxR4sf7o 7y agoOne correction/clarification: It wasn't trained on reddit comments, but on reddit links. So it's not learning to write like a redditor. It's learning to write like what redditors read.
- samstave 7y agoBut ive been on reddit 13 years. I extremely rarely upvote stories. I read the comments section more than i read the articles. I cant be the only one.
- deleted 7y ago[deleted]
- johnwyles 7y agoFrom the example it looks like with next to no changes at all it would be able to spit out a plethora of "fake news" articles, even if it's only source was reddit.
- yogthos 7y agoI find it bizarre that people worry about this since we clearly already produce more than enough bullshit on our own without any need for AI.
- barbecue_sauce 7y agoIt's the difference between drowning in an ocean of bullshit and living on a planet that is one giant bullshit ocean.
- holbue 7y agoAI scales much better, than human bs generators. I consider at least this aspect concerning.
- yogthos 7y agoThere are 7 billion humans on the planet now, we don't have a problem generating bullshit at scale. The same methods you'd use to discern AI generated bullshit are already needed to evaluate current bullshit. If anything I'd argue that this kind of tech becoming common place could get people to start evaluating all news more critically.
- lfnoise 7y agoBest sentence in the article: "It also happens to be trained on a large chunk of Reddit, since the author decided that this was undeniably the perfect location to obtain high quality, impeccable prose."
- czr 7y agoWhile snappy, this sentence is misleading. The model is trained on webpages whose links have received at least 3 karma on reddit, not on reddit comments themselves [0]. The motivation for this choice is not to find impeccable prose, but rather to establish a lower bound on quality for the dataset, and avoid training the model on too much of the spam/nonsense text that is frequent in e.g. CommonCrawl. Also, the "author" is not a single person, but rather 6+ people [1]. [0] https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ [1] https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf https://d4mucfpksywv.cloudfront.net/better-language-models/l...
- yogthos 7y agoObviously now that people have seen that this is possible it will be replicated in short order whether they release it or not.
- pigscantfly 7y agoFor those interested, this open-source replication of the dataset is a good place to start https://skylion007.github.io/OpenWebTextCorpus/ https://skylion007.github.io/OpenWebTextCorpus/ - obviously, you'll need access to some large GPUs as well. I'm currently training one across four cards, although I'd like to throw more power at it. Very interested in altering the architecture to reduce memory requirements (without unduly effecting the loss). Excited to see what the community comes up with.
- petters 7y agoYes, but training it is very expensive.
- yogthos 7y agoBut that only needs to be done once. Also, it turns out that the current crop of neural nets is incredibly inefficient https://www.technologyreview.com/s/613514/a-new-way-to-build-tiny-neural-networks-could-create-powerful-ai-on-your-phone/ https://www.technologyreview.com/s/613514/a-new-way-to-build...
- ghobs91 7y agoSuper clickbaity and low effort title.
- jonathankoren 7y agoI agree. The hype is strong with OpenAI. The marketing of GPT-2 seemed simultaneously braggadocio while also trying to capitalize on woke AI. “We made this super cool thing. It’s super dangerous. You can’t see it.” I mean, come on. They can earn their reputation the regular way, and it will be fine. Sure they have the click baity DOTA demos, and maybe they are doing some interesting stuff. But OpenAI’s announcement of GPT-2 was particularly egregious.
- vortico 7y agoStupid question: Can I play with GPT-2 by entering my own prompts using a website or downloadable project? Or is it not ready yet?
- Voloskaya 7y agoYou can, see: https://github.com/openai/gpt-2/blob/master/DEVELOPERS.md https://github.com/openai/gpt-2/blob/master/DEVELOPERS.md. But this is for a smaller version of GPT-2 (~400M parameters). Since the bigger model (2B parameters) was deemed too dangerous by OpenAI.
- vortico 7y agoThanks! Attempting now.
- czr 7y agoYou can try out the 345M-parameter model in your browser here https://talktotransformer.com https://talktotransformer.com. The more powerful models are available to AI/security research groups upon request.
- kleiba 7y agoGPT-2
- vortico 7y agoCorrected, thanks
- jancsika 7y ago> It also happens to be trained on a large chunk of Reddit, since the author decided that this was undeniably the perfect location to obtain high quality, impeccable prose. Hm, why in the world would they not train it on Sci-hub articles and generate output for which we have plenty of domain experts who can judge the quality?
- czr 7y agoI'm not sure if you're being tongue-in-cheek, but: * Training on Sci-hub or book torrents would probably get them sued, so that's right out. * The point of training the model on reddit-linked-articles with 3+ karma (not reddit comments, as the article suggests) is to filter out worthless content (spam or non-text pages) while still getting a large, diverse sampling of human writing. They're training huge models and they need as much data as possible to do so. * Every native english speaker is a "domain expert" for the purpose of evaluating the model's results. The point at which we need subject-matter experts to evaluate the quality of neural-net generated research papers is many decades away; the excitement around GPT-2 is that it can generate coherent English sentences at all.
- rossenberg79 7y agoFor those who do not see the danger of advanced AI generated text being rolled out everywhere, imagine a web where you have to carefully read everything you see in order to determine if it’s written by a human before you can start to take the text seriously. That means no more skimming threads to get the gist of what people are saying, no more skimming through answers on stackoverflow, no more skimming through articles. An article that looks reasonable on a cursory glance only begins to fall apart when you spend (waste) time reading it carefully. It also means anyone posting content must spend extra effort proving they are a human for their readers.
- samstave 7y agoConversely, maybe the people who should be replaced by such AI are teachers and professors. What if you could beneficially weaponize an AI to teach children/people a vast body of knowledge quickly on any given subject. Please see the movie; Lawnmower Man
- matt4077 7y agoNobody is disputing that AI can be useful, so your comment seems to be missing the point. It would be slightly more coherent to at least point to a potential use case for text generation specifically.
- andreyk 7y agoPretty decent overview of the model, but did they really have to go with that title... IMHO the more interesting story with GPT 2 is the hype around it as well as the (huge) backlash around its release. [self-plug coming] If interested, check out this summary of that whole story: https://www.skynettoday.com/briefs/gpt2 https://www.skynettoday.com/briefs/gpt2
- bArray 7y agoPeople write fake news all of the time, why is this AI so "dangerous"? At worst it'll just add to an already existing mountain of news. At best I believe it will prompt people to be more critical of what they read and where they read it from, after all, how many times do you need to be embarrassed by some auto-generated news sources?
- bjourne 7y agoSpam. How on earth would a spam filter determine if such an auto-generated text is spam or ham? Since it doesn't have a human brain it can't. So we get into an arms race again. Right now, spam filters seem to have the upper hand but that can change very quickly. You can also use this text generation for less nefarious purposes such as for submitting journal articles (https://pdos.csail.mit.edu/archive/scigen/ https://pdos.csail.mit.edu/archive/scigen/), creating fake facebook campaigns, creating fake github profiles, phishing on dating sites, fool Google's page ranking algorithms and so on.
- bArray 7y agoTo be completely honest, a lot of these services need a wake up call anyway. Fake papers, fake Facebook campaigns, fake GitHub accounts, faking dating profiles and crappy search rankings are already a problem - but these services are currently unmotivated to solve these problems as they still work 99% of the time. We are already in an arms race for people's attention and data, that horse already left the stable. Kindly asking people not to produce fake news is so far not working out for us. This AI will be replicated, likely very soon, if not already.
- bjourne 7y agoI don't think your argument is very convincing since this problem would affect almost every site on the internet and almost every text-based communication channel. If burglary increased 100-fold, I don't think many would see it as a wake-up call to install beefier home security systems.
- 7y ago
- drivingmenuts 7y agoIs it too soon for a Butlerian Jihad?