36 ms·

Better Language Models and Their Implications

426 points by yigitdemirag 8y ago

option 8y ago
This is very impressive. The decision to not release the model is questionable imho. There are labs, companies and state agents which have way more compute than OpenAI and therefore can do even better. Perhaps we need some kind of competition for detecting machine generated vs human generated content?
- nprateem 8y ago
  That's a GAN.
resters 8y ago
While censoring the full data set seems in some way to support the rationale of the OpenAI charter, it also means that only state actors and very well-funded entities will be able to use the work to create models of the size necessary to do the impressive stuff in the write up. Based on the concerns, it would seem that restricting the capabilities only to state actors would have the opposite of the intended effect. Why not let thousands of amateur researchers, undergrads, etc., use the model to detect instances where the model was used to generate text, etc.?
- skybrian 8y ago
  I would guess this reduces the risk. Why would you say it does the opposite? My argument: state actors might misuse this tech, but letting any script kiddie do whatever they want almost guarantees someone will misuse it.
  - mikeyouse 8y ago
    Which is exactly what happens whenever there's a leak of NSA or other foreign government 'hacking' tools. As soon as they're public, ransomware authors and other shitty actors all deploy them to steal as much as possible before systems are patched. Case-in-point: Wannacry: https://en.wikipedia.org/wiki/WannaCry_ransomware_attack https://en.wikipedia.org/wiki/WannaCry_ransomware_attack NotPetya: https://en.wikipedia.org/wiki/Petya_(malware)#2017_cyberattack https://en.wikipedia.org/wiki/Petya_(malware)#2017_cyberatta...
    - resters 8y ago
      I think it’s a bit different because the attacks are not entirely clear. The power of many of the attacks would be for them to fly under the radar because nobody was expecting it. So having lots of script kiddie attacks (which would be sloppier and easier to notice) would lead to a more rapid adoption of safeguards.
- jfoster 8y ago
  They can always revisit the decision once other entities seem to have made this progress. In the meantime, they don't have to be so disruptive.
- lostmsu 8y ago
  Or, in 3 years, anyone. I suggest everyone start thinking on ramifications now.
legatus 8y ago
I was honestly surprised by the quality of the generated text. While I can't say I've been following the state of the art in the last months, this seems like a pretty important step forward. Furthermore, at the end of the post they note that the samples are somewhat representative of their results. Maybe they should consider releasing a text file with some more (not hand-chosen) samples? Whatever the case, fantastic work, my congratulations to the authors.
- wuthefwasthat 8y ago
  Thank you! We've released 500 random unconditional samples from GPT-2 at https://github.com/openai/gpt-2/blob/master/gpt2-samples.txt https://github.com/openai/gpt-2/blob/master/gpt2-samples.txt
  - legatus 8y ago
    Sorry about that, I should have not missed it. Since you're here, I'll ask a quick question to satisfy my curiosity. This is a pretty big step, even just looking at the benchmarks. Where do you see text generation going in 5 years? What about 10 years? As I said, no need to give long explanations, I just want to know what do you expect, as an insider,on an intuitive level.
  - Scaevolus 8y ago
    Some of these are basically horror short stories: ======================================== SAMPLE 164 ======================================== I want to see a ton of households with kids with no food in their houses. Surviving famine requires storing up food in hopes a spoiled enclosure might provide the behaviors of food memories we crave. In near future, they will have to leave their cages. We're spending all our time theorizing about animal minds, yet we have living, reacting animals. As if we can just no be in touch with our emotions at is practical. My kale starts from an academic grown Himalayan. It hydrates in the garden, but it seems another species than we know is fermenting! My general lab is a recently renovated cave survival room with sun-filled bowls full of food for rats. The interior design is nearly utilitarian (no smart phones, blog, writing tools, or ovens there), except the cave ceiling. This is where we grow the food to feed animals. Which they will eventually eat. My girl starts out with a kind of scaffolding green and wild, but soon she can't grow more than a few leaves. Who needs her monotub artistically scissors and wire? The boxy roots are brittle right away before growing into skin weeds. So that's not going to do it. An unsteady start for another young carnivore's evening. I go in for some priming with baking soda and dry iron. A fun little experiment is once the kopi luwak seems she will grow some damp spikes. Consider this:chopsticks are soft under a microscope, webbies, and twining vines all apply more pressure from tangles of animal blood. Taking over the cultural mythos of science, I peek over her purple skeleton spikes with an endoscope. Once the higher organs take over, the kabuka unfurrows skin at a clever angle to reach past the body curse powered at this end of the lab. Curiously, without a hoe or a pot to turn into a tiled floor, she does not chew with her jaws. I have undressed her to her base to fill the bowl. She is pushing her way to get her shelter to stretch out into, or if I've planned correctly, squish into. She does not move like a biological creature, pulling up with her haunches while excellent and facing directly downward. She prefers laying there moss-covered and mother-prey encouraged paying attention to her uphill path. Lack of summer fur gives me ample room to poke through her skin, pull out black seeds, and taste the damp thing she'll roost under in future. Would the static reference very much reduce a virus being blocked within its first expenditure, nights? I'm not sure, but it's for sure a more effective inoculation. Hopefully, these experiments will be continued. Hopefully. My guru and master who made us does not focus so much on running where we want as it does on empathy for the soil,, plants, and animals with which he touched in some perfect way. Delhi is often flooded my first morning in the shafts of windy sunshine. Let's not give success an easy choice. The trees are in the underground and they change colors weekly. In simple terms, they're scattered flowers taking their cues from the transynthetic forest. While I'm pioneering, the one thing I sacrifice in the relationship is conducive to imagination. A full color-changing lets us enjoy cozy warmth while simultaneously losing whoever is formal and rational for the feel good activity. Like cosmic lap of the firmotsuites. An American outcropping of stonemines blocks the sunlight. The unseen wave depose colonies to accommodate my habit; sandblasting. There are microbes from amoebas that change the composition of the water. In the habitat where the natural and mecopate are as thoroughly ever so mindful of the fence as the meesthe gardener has been, I'll shed this duality when I sit with her. Food grows from space; human needs feed them. Sandbringation turning our microbes and animals can save thousands in pandemic deaths. I value them all in metaphorical and real (if tough) tenacity. Species survival deals good cop in these two every where storytelling machines. Door is opened. My guru goes rescuing the kupi luwak elder from drought resistant drought. I alone have learned her humdrum bookish biogeography is yielding to light and fire. Cow has problems taking roots. Bring my love to her and make a mend. She needs a clean-air daughtersnail easy experience not in a month's time, but in just a few hours of thinking about reflexes, calendar dates, and natural spatial relations for a mind that has grown certain.
winterismute 8y ago
Seems like in the near future the idea of "turing test" and the one of "not fake news" will eventually coincide...
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
yongjik 8y ago
> ... some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.” Man, the auto-generated text is hilarious. And uncannily good. Though I have to wonder if it's total random fluke or there's one among their 1.5 billion parameters that predict "likelihood of mythical bestiality in South America".
- sanxiyn 8y ago
  Considering that it memorized Gettysburg Address verbatim and knows Charles Darwin wrote Origin of Species, it probably knows about both South America and unicorns more than me... Unicorn story does demonstrate that it knows South America has to do with Argentina, Andes Mountains, and University of La Paz.
zellyn 8y ago
The generated Unicorn story has about the writing quality (in both senses: standard and feel) of fanfic
minimaxir 8y ago
The generated text sounds too good; is it possible that the model overfit the source material (especially since the n-previous-tokens value is infinite, while other approaches like char-rnns/textgenrnn use a fixed window length)? It's something I've encountered many times while working with text generation.
- nmca 8y ago
  Try googling sentences.
  - minimaxir 8y ago
    Hmm, I tested a few sentences and it didn't turn up any exact matches (aside to this article), so maybe I'm wrong. With a temperature of 0.7/1.0, that's enough for sufficiently random text I suppose. (the raw, uncurated generated text using the smaller model is a bit more random: https://raw.githubusercontent.com/openai/gpt-2/master/gpt2-samples.txt https://raw.githubusercontent.com/openai/gpt-2/master/gpt2-s...)
    - wuthefwasthat 8y ago
      Those samples are from the large model (GPT-2)! Regarding memorization vs. generalization, see our paper for more analysis.
      - yorwba 8y ago
        Have you tried to determine which parts of the training data contributed to the model generating a certain output? I wonder whether the model avoids reproducing exact matches of the training data by splicing several similar articles together. For example, the generated text about the Civil War mentions that Thomas Jefferson Randolph [0] was named after his grandfather, the president. But is the wording mostly influenced by articles talking about that specific fact, or does it draw from more general examples of someone being named after their grandfather? [0] https://en.wikipedia.org/wiki/Thomas_Jefferson_Randolph https://en.wikipedia.org/wiki/Thomas_Jefferson_Randolph
resters 8y ago
Just as many pesticides mimic the hormonal and chemical signals of pests to drive certain behaviors that lead to eradication, this work mimics the linguistic signals of humans. I think viewing it metaphorically as the most sophisticated humanicide discovered to date is probably appropriate. Consider that conventional munitions make an effective pesticide but are not used due to their side effects. Instead, chemicals are used to destroy or mimic the perception and production of various signals so that populations of unwanted critters effectively self-destruct. Imagine a war fought with a weapon like this that left entire cities perfectly intact! # end hyperbole
- honzzz 8y ago
  This is exactly the line of thinking that this article inspired in me... AI scouring the internet, figuring out what makes us tick and generating perfectly persuasive stories to convince us to to... what? I kept thinking about the story "Sort by controversial" by Scott Alexander: https://slatestarcodex.com/2018/10/30/sort-by-controversial/ https://slatestarcodex.com/2018/10/30/sort-by-controversial/
- est31 8y ago
  https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World
- deleted 8y ago
  [deleted]
- deleted 8y ago
  [deleted]
beefman 8y ago
I wonder how this would do on the Hutter Prize (I doubt it would beat the current record but I'm curious what the result would be) http://prize.hutter1.net http://prize.hutter1.net
- gwern 8y ago
  They include the enwik8 BPC estimates. It may not be strictly comparable - the most obvious issue is that since the HP takes the compression paradigm of intelligence, the compressor size is part of the total, and those 1.5b parameters certainly are not cheap. (This is one reason I think the HP is outdated. The corpus is not big enough to allow the superior asymptotics of approaches like RNNs or Transformers to compensate for their far larger binary size. HP is not measuring progress on an intelligence metric we care about, it's sort of measuring a 'demo scene' metric of intelligence.)
zawerf 8y ago
It's pretty interesting that their training set consists of "outbound links from Reddit which received at least 3 karma". There are definitely large subreddits which are flooded by highly voted fake news which you don't want to emulate (unless that's the goal). It also reminds me of a short fictional story which explores what would happen if an AI learn how to maximize reddit's sort by controversial score instead: https://slatestarcodex.com/2018/10/30/sort-by-controversial/ https://slatestarcodex.com/2018/10/30/sort-by-controversial/ Maybe that dystopian story is closer to reality than we thought?
gallerdude 8y ago
These samples are freaky good. We're approaching some threshold very, very fast. I'm not sure what that threshold is, and whether or not crossing it is a good thing, but soon we'll be there.
- Matumio 8y ago
  I like how it effortlessly switches into "git diff" mode at the end of sample 112. Sadly it doesn't do whitespace. > Showing 1 changed file with 4 additions and 19 deletions. +4 −19 png_source/colors/pointer.py Show comments View 8 png_source/colors/pointer.py @@ -35,6 +35,7 @@ def _draw_hull_class_level(self): repr(Shape[td_get_framepanel_pcs(dc) for dc in xrange(dc.cols)]), self.doublesize.values) \ } def _draw_hull_class_level [etc.] It also inserted a helpful reference link to http://wiki.openarcade.com/wiki/List_of_Programmer_Constants http://wiki.openarcade.com/wiki/List_of_Programmer_Constants (I've had to check: no, there is no openarcade wiki.) Also, sample 217 is some mediocre Java, with comments and all. Impressive how a single model can handle this all at once.
- sanxiyn 8y ago
  Sample 105 "In 2009, researchers at NASA from the Air Force Office of Scientific Research's Microwave Propulsion Laboratory held a series of contests, including one for "best brain-computer interfaces." In their prize match, teams from around the world competed to develop new brain mapping techniques for ways of collecting data from inside people and rewriting it over and over. One single original BrainBridge video colonized all channels of web-viewing in the world. I've watched many of these videos on YouTube. Advertisement - Continue Reading Below On one hand, if you're convinced that the next breakthrough in neuroprosthetics will be profoundly more powerful because better/faster or cooler/more reliable algorithms capture the wiring that connects our neurons, then it makes sense to equip humans with mind-reading implants. But when it comes to real brain-reading devices, there are a slew of caveats, arguments, and threats on the horizon that mean that these technologies cannot take off anytime soon. If you truly think that these technologies will be the basis for a global brain-reading surveillance state, it makes sense to act NOW. Since so much money is on the line (in computers and other components, research funding, patent and trademark rights, marketing, investor interest) making and maintaining a head-mounted-camera program is done with a certain level of speed by enterprises like Google and Facebook." That "Advertisement - Continue Reading Below" is very realistic indeed? These technologies, like GPT-2, will be the basis for a global brain-reading surveillance state, so indeed it makes sense to act NOW, like not releasing training data. Kudos. But this will be replicated with a certain level of speed by enterprises like Google and Facebook. Well said.
lucidrains 8y ago
Started a Google colab with the interactive text generation script. https://colab.research.google.com/drive/1da54684tFMjPbR5idbvoCyjOoEGwIVwV https://colab.research.google.com/drive/1da54684tFMjPbR5idbv...
- marviel 8y ago
  Thank you.
- KVFinn 8y ago
  Super appreciate this!
- exit 8y ago
  to be clear, this is the "politically innocuous" open sourced model. the results are not impressive.
  - marviel 8y ago
    Yeah the results aren't amazing on their own, but if you treat them similarly to how they do over at Botnik -- with some human curation involved-- you can find some interesting sentences.
  - lucidrains 8y ago
    Yea, somewhere in between the released model and the model they presented, the deep net has learned narrative structure.
  - minimaxir 8y ago
    You need to run it a few times; the results vary very wildly.
- minimaxir 8y ago
  This is the approach I recommend; Colaboratory's GPU is helpful when the sequences are really long.
dzink 8y ago
This tech can easily be used to flood humanity’s shared brain with auto-generated propaganda. Schizophrenia of the internet in a way. There is plenty of incentive with Google algorithms favoring number of words and relevant keywords in content for rankings - you could have NLP bots lifting junk sites to top results. To step ahead in that chess game, a detection tool for fake would be just training grounds for better GAN. Instead we may see a certifying authority that labels content as human generated, certified fact maybe? Wikipedia and reddit are not safe without fast automatic moderation either. Do you have a brainstorm or idea/prototype submission site where people can submit approaches to countering bad ai actors? An white/grey hat ai bounty program of sorts?
- olleromam91 8y ago
  Great Idea...but better make sure it's not hosted on the internet.
- comboy 8y ago
  Web of trust. Your results and how much you trust given text is based on who gave it to you. You assign trust to people you know, then it's a small world effect and recursive trust calculation based on who they trust. Centralization won't work. Whether something is good or bad or fake must be subjective and based on your personal network / your beliefs. Otherwise, long term, I don't see how you could avoid dystopia.
  - pmohun 8y ago
    This has a dangerous implication that we've seen recently with the newsfeed "bubble." If dissenting opinions are never allowed into your sphere of influence then we may continue to see an accelerating polarization of our society towards extremes.
    - comboy 8y ago
      Bubble that's dangerous for society is when everybody believes the same thing, not when everybody has different set of beliefs. I know the more popular bubble is information one and you make a fair point, but I believe it's powered more by recommendation systems than the things that you trust. I got distracted lately and I wanted to clean it up before putting it out there but if somebody is bored: http://comboy.pl/wot.html http://comboy.pl/wot.html
avivo 8y ago
This was only a matter of time. For the DEFCON AI Village in August I talked about the implications of this sort of tech, and how that impacts how we release "exploit" code / think about "cognitive vulnerabilities": https://medium.com/@aviv/what-does-a-world-with-automated-social-engineering-look-like-79cd09b5a7b1 https://medium.com/@aviv/what-does-a-world-with-automated-so.... If you are doing work in this space, either in ML research or related security, you need to be thinking about implications (also see e.g. https://maliciousaireport.com https://maliciousaireport.com).
- levesque 8y ago
  I mean, the ideas are there. The scope of the project is probably too big to reproduce for now, but eventually it will be accessible to your average spammer / scammer. We will get there. We won't be able to get these tools locked, make them exclusive for a certain type of responsible AI specialists. Someone will spill the beans, the models. People with bad intentions will reproduce these results. To me, the real deal is how we will manage these outbursts when they happen. I assume discriminative models will solve the problem for a while, but as with Generative adversarial networks, you will be able to train models that are harder to discriminate against. I posit we're in for a big societal change (maybe more a content crisis) sometime in the next 10 years. Pretty sure we won't be able to keep it from falling in bad hands.
  - Nydhal 8y ago
    On the flip side, this will give an edge to specific groups that can communicate in highly vernacular/non-written/non-standard languages.
  - ctoth 8y ago
    “Spam-filters, actually. Once they became selfmodifying, spam-filters and spam-bots got into a war to see which could act more human, and since their failures invoked a human judgement about whether their material were convincingly human, it was like a trillion Turing-tests from which they could learn. From there came the first machineintelligence algorithms, and then my kind.” I Row-Boat, Cory Doctorow, 2005: https://craphound.com/overclocked/Cory_Doctorow_-_Overclocked_-_I_Row-Boat-A4.pdf https://craphound.com/overclocked/Cory_Doctorow_-_Overclocke...
roywiggins 8y ago
I'd love to see what it would produce if you fed it the first sentence of your average Nigerian Prince scam. Fully-automated phishing- you could even automatically toss in a couple details about the recipient and let the AI riff on that for a bit.
songeater 8y ago
Is anyone else troubled by them not releasing the source model/dataset/parameters here? Yes, the technology can be used for malicious means - but would argue that "DeepFaking" language is FAR less of a problem than "DeepFaking" video/photo/audio... which already occurs. Seems like they went back on their charter to share AI developments broadly ("not concentrate power") under the excuse of "safety." (These results look fire btw) Note: copied my comment from dupe thread
- ipsum2 8y ago
  I agree with this. OpenAI isn't particularly 'open', compared to other AI research organizations (Notable ones that open source almost all their work are AllenAI and FAIR, but I'm sure there are others). Wonder what their excuse is for not releasing the source model or code for their DoTA bot. Surely there's no safety issues there?
- _Microft 8y ago
  Quote from their charter [0]: Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research. They referenced that from the article you commented on in the section 'release strategy' [1]: [...] we see this current work as potentially representing the early beginnings of such concerns, which we expect may grow over time. This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community. [0] https://blog.openai.com/openai-charter/ https://blog.openai.com/openai-charter/ [1] https://blog.openai.com/better-language-models/#releasestrategy https://blog.openai.com/better-language-models/#releasestrat...
fouc 8y ago
I think this would be extremely useful when we can do the inverse. Basically - can we detect if someone's writing is nonsensical or not? Can we detect if someone that is producing many well written essays is adhering to reality or not? Are they subtly re-defining terms, using flawed examples, etc? The generated example of the biologists discovering a unicorn herd is too convincing on its own. It's only because it's so outlandish that we get the sense it's fictional.
- levesque 8y ago
  Our models have reviewed your submission, and deemed it to be 87% incoherent, 52% redundant, and 24% fake. We therefore reject your submission. Sincerely, the Chief Bot Editor.
aaaaaaaaaaab 8y ago
In 10 years, content written by actual humans will be a premium niche, like tailored suits - reserved for the elites. The rest of us will be force-fed with machine-generated garbage.
- Franciscouzo 8y ago
  That doesn't makes sense, text can be distributed at marginal cost, tailored suits are expensive because there's not a lot of supply.
  - aaaaaaaaaaab 8y ago
    Tailored suits used to be cheap too, but taylors went out of business due to mass production.
- abecedarius 8y ago
  There was an old science fiction novel set in a world like that. I've forgotten the name, but they called the robot-written stuff "wordwooze". (It doesn't work as a search term because some publisher is using it now.) Added: it was The Silver Eggheads by Fritz Leiber. An odd, forgettable book itself.
- bitL 8y ago
  But will ML companies pay link tax? Assuming ML would write/fill in forms to incorporate itself, have a conversation or two with some tax office person, and then off to the wild... Where can I study computational law to stay ahead of the curve?
cs702 8y ago
This kind of "blocking-and-tackling" work is important. The authors take a well-known architecture, the Transformer[a], configure it with a progressively larger number of parameter, train it to predict the next word conditioned on previous text, using a large dataset consisting of 40GB of text scraped from the Web, and test each trained model on a range of zero-shot transfer-learning tasks. Remarkably, the performance of a Transformer in the tested tasks improves log-linearly with the number of parameters, suggesting that even the largest model tested, with 1.5B parameters, still underfits 40GB of text. This is compelling evidence that we do NOT need new architectures, NOR new kinds of training objectives, NOR new theories, for better language modeling! We can get better language modeling simply by increasing model capacity (i.e., by adding more parameters to existing models), which becomes easier and simpler to do as hardware continues to improve over time. Great work. PS. In case it's not clear: I'm not saying we should suddenly stop searching for new, better ideas and architectures. That would be silly. Please don't attack a straw-man :-) [a] https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762
- modeless 8y ago
  Agreed that "simply" scaling up with more compute will result in progress and useful systems, and work in that direction is interesting and valuable. But, while we may not need new architectures or training objectives to make progress, we do need them to approach human level sample complexity. Humans don't need to read through 40 GB of text multiple times to learn to write.
  - fartcannon 8y ago
    I guess the data requirements act as a stand-in for our genetic evolution... Our brain models are good at learning the things we learn. Our computer models aren't yet. /tautology
  - cs702 8y ago
    > Agreed that "simply" scaling up with more compute will result in progress and useful systems, and work in that direction is interesting and valuable. But, while we may not need new architectures or training objectives to make progress, we do need them to approach human level sample complexity. Yes, agreed. Nothing I said above contradicts that! :-) > Humans don't need to read through 40 GB of text multiple times to learn to write. Yes, that's true... but to keep the comparison fair, note that we do need many years of schooling to learn to read, say, at a high-school or college level. And before learning to read, we first must learn to speak, which surely helps. And we also get to inhabit bodies that see, smell, touch, and interact with the physical objects that we read and speak about during our formative years, which also helps. The more one thinks about it, 40GB of data is actually a tiny figure in comparison to the amount of training data that flows continuously to our brain from all senses. I think I read once that our brains process on the order of 10 to 100 GB of training data per second.
deleted 8y ago
[deleted]
Cybiote 8y ago
It's becoming ever more certain that the transformer architecture is one of the largest contributions to AI (not merely machine learning, but AI), often beating LSTMs despite LSTMs being expressive enough to capture Turing Equivalence (at least in theory). Its main ideas are three: shorter paths help gradient flow, the training setup and the final key aspect, unhelpfully called self-attention. Self-attention is better thought of as a form of similarity gated key-value soft memory on which learning operations allows Transformers to learn non-trivial programs with contextual weights look-ups. I also notice reported tries, suggesting some level of curation. While this level of generation is undoubtedly impressive and a sign of non-trivial levels of understanding, the ability to project along arbitrary dimensions of similarity at a fine-grained level and to learn from text instruction is more useful than text generation. Although the unicorn story was a really fun read, better than many humans already, I doubt it could have gone on for much longer. It maintains a theme but not coherently or fluently (see especially the Kennedy nanotech and recycling examples, comparing the dis-fluency there versus the excellence of the Civil War report suggest at least some over-fitting). These relatively minor caveats aside, this is unambiguously an outstanding result. Winograd Schemas are the single metric to track if interested in understanding how language understanding is truly improving. OpenAI reports 71% and wrongly report the previous record as 63%. The current record here is at 65% https://gluebenchmark.com/leaderboard https://gluebenchmark.com/leaderboard though not fully comparable. Will OpenAI be submitting? Note that you can get to 60% using about 1-2 orders of magnitude less data and compute. It concerns me that results here are so far dependent on such large data and computation. However, based on several papers I've read, I do not believe this to be inherent even in transformers. I plan to do some experiments on this when I free up some bandwidth. If everyone is pulled in by the glamour of working for a well funded, prestigious operation then it should be no surprise that they do not consider paths which operate on several orders of magnitude less data and computational resources. We all should consider bringing about a group of researchers who swear to an austere computational life of a single GPU, no more than 4-8x average RAM and CPUs that do not cross 90 Watts. The Bicameral Order would be a good name for such a group.
- wuthefwasthat 8y ago
  Yeah, there are definitely still places the samples fall short! Keep in mind we're still using very naive sampling techniques. RE Winograd: WNLI is different, see https://arxiv.org/pdf/1804.07461.pdf https://arxiv.org/pdf/1804.07461.pdf
kaffee 8y ago
Not releasing the model? These people aren't scientists. edit: toned down a bit.
- schoen 8y ago
  Somehow, some part of me really wants to see what this model would generate with that as a prompt.
- kaffee 8y ago
  Actually, this prompted me to look up the funders of OpenAI: - Sam Altman - Greg Brockman - Reid Hoffman - Jessica Livingston - Elon Musk - Peter Thiel Useful information in interpreting the mission statement, I think.
plzHireMeImGood 8y ago
Is this comparable to Google BERT (Bidirectional Encoder Representations from Transformers) ? Benchmarks are different. Can I use any of this models for other tasks no mentioned in the papers, something more than the "fine tuning"?
teabee89 8y ago
The best part is "Scroll down for video" from https://blog.openai.com/better-language-models/#sample3 https://blog.openai.com/better-language-models/#sample3 :)
mark_l_watson 8y ago
Wonderful results. I Don’t think I will experiment with the smaller available model, at least right now. I am still happy with BERT, especially for basically solving anaphora resolution (coreference of pronouns, etc.)
dogcomplex 8y ago
Plot twist: all the comments on this thread were auto-generated.
mrfusion 8y ago
I’m most impressed by its ability to answer questions about the text. Why can’t someone built something like this on top of Wikipedia? That would be amazing to ask Wikipedia any question you can think of.
- piggyzach 8y ago
  Google has already built this, but it definitely isn't perfect. http://www.seobythesea.com/2014/10/google-fact-questions-entity-references-unstructured-data/ http://www.seobythesea.com/2014/10/google-fact-questions-ent...
mrfusion 8y ago
Can anyone do an eli5 of how this works?
rfdearborn 8y ago
Sure, not releasing the full trained model probably delays it, but sooner or later a bad actor will do their own scraping and train their own model and share it around and the genie will be out of the bottle. Then what? I think we need to be conducting AI research (and building software generally) under the assumption that all of it will eventually be repurposed by bad actors. How would our practices be different if we consistently and cautiously did this? Here's a thought experiment: how would the Manhattan project have been different if it were carried out in the open and its products were instantaneously and infinitely reproducible? What is the MAD equilibrium of AI research? I think the impact potential is similar even before AGI.
- mlb_hn 8y ago
  A lot of the advancements that made the Manhattan project were published by the Germans. On hearing about Hiroshima, Otto Hahn was "'shattered,' and went on to say that he felt 'personally responsible for the deaths of hundreds of thousands of people,' believing that his discovery had made the bomb possible." (https://www.washingtonpost.com/archive/opinions/1992/03/01/the-secret-nazi-atomic-tapes/42a8222d-5ec0-4d66-b335-a71064b4e290 https://www.washingtonpost.com/archive/opinions/1992/03/01/t...)
- a-dub 8y ago
  wasn't that the point of this whole openai thing? they didn't like the idea of there being a club with just google in it that had access to resources and funding to collect and train on massive datasets so they were going to be the "bad actors" who would do their own scraping, train their own models and share them around? isn't it supposed to be called OPENai? they don't want to share the data because they don't want to throw away the edge they've gained by collecting it. :) computer programs that generate human like text aren't dangerous, the internet is full of human like text that is mostly bullshit anyway.
  - lc5G 8y ago
    I have had the same impression regarding their work on dota. They got a lot of publicity with it but their work is not open at all. They have released neither their code which runs the bots on dota2 nor their training code nor the final model. All we have is video recordings of a few games against humans.
grok2 8y ago
Nobody has asked about the animated text on the left at the top -- how is that done? That is more interesting to me!
- marvy 8y ago
  It's an mp4 video. But then the next question is: how did they make that video? And the answer to that is... I have no idea.
paraschopra 8y ago
Anyone who’s done large scale model training like this, an you shed light on following questions: What is the process like? Do you prototype locally? How do you confidence that the only limitation to good results is more compute power and NOT the model architecture or applicability of deep learning to a particular task? At what point do you decide that shelling many tens of thousands is OK? How often do you do large scale training only to find non-impressive results and hence the money wasted?
- slashcom 8y ago
  There’s a natural way to parallelize these models so that using 128 GPUs is the same as a 128x batch size. You can similarly simulate 128x batch size by accumulated gradients before backpropping. So you can test on just one or a few GPUs before you run the full thing. By that point you know it’s going to work, it’s just a matter of how well and whether you could’ve done nominally better with different tuning. There’s been enough research leading up to this paper to suspect that just scaling larger would play out.
  - paraschopra 8y ago
    Thanks. >By that point you know it’s going to work, it’s just a matter of how well and whether you could’ve done nominally better with different tuning. This can't be true in all cases, right? I'm assuming that for many initially promising results on less-compute when they scale it, the results aren't impressive. I'm very curious to know what is the trials-to-success rate of publishable results when big-compute is thrown in the mix.
    - slashcom 8y ago
      It’s indeed a very high trials to success ratio. Again though, there’s enough papers preceding this one that you could have good confidence in the effort. Another thing that helps is orgs like OpenAI have their own servers, rather than renting ec2 instances. You also don’t just launch that many things and them ignore it. You monitor it to make sure nothing is going terribly wrong. But yeah there’s also the fact that if you’re Google, throwing $2m worth of compute at something becomes worth it for some reason (eg Starcraft)
lordnacho 8y ago
This is so crazy good, someone needs to do a Turing test by sending it to some unsuspecting publishers. I get the feeling that debatepocolypse is not far away. Every forum can now be spammed with reasonable sounding gibberish that humans will have to slog through.
- gardaani 8y ago
  Few MIT students did that with their simple text generator. It is one way to expose conferences with low submission standards. http://news.mit.edu/2015/how-three-mit-students-fooled-scientific-journals-0414 http://news.mit.edu/2015/how-three-mit-students-fooled-scien...
- sanxiyn 8y ago
  We must run a bot on HN with upvotes/downvotes for comments acting as reinforcement.
- Retra 8y ago
  That's not a Turing test. A Turing test is an ongoing dialog, not a one-shot filter.
  - Isinlor 8y ago
    Add some bias to create dialogues when sampling based on movie subtitles and I could see it passing proper Turing test.
    - Retra 8y ago
      A proper Turing test allows you to ask questions whose answers must indicate novel introspection, learning, and social awareness. It's absurd to think any computer system today would pass such a test.
      - lordnacho 8y ago
        This thing does exactly that. You can see in the text it generates, it sounds like someone who's thought about things and keeps a consistent tone about the subject.
LHxB 8y ago
This reminds me of Dürrenmatt's "Die Physiker "(https://en.wikipedia.org/wiki/The_Physicists https://en.wikipedia.org/wiki/The_Physicists). While this has indeed very scary implications, one should be aware that if it's thinkable, eventually it will be thought (I'm paraphrasing here).
braindead_in 8y ago
Seems like magic! I wish I could do something similar with our chat support questions and answers. It would be nice to have something like this built-in.
lotaezenwa 8y ago
this will definitely be reverse engineered and open-sourced.[0] [0] https://en.wikipedia.org/wiki/Streisand_effect https://en.wikipedia.org/wiki/Streisand_effect
antpls 8y ago
It's beautiful that the reduced model itself is only 175 lines of Python code, thanks to TensorFlow : https://github.com/openai/gpt-2/blob/master/src/model.py https://github.com/openai/gpt-2/blob/master/src/model.py

426 points by yigitdemirag 8y ago

This is very impressive. The decision to not release the model is questionable imho. There are labs, companies and state agents which have way more compute than OpenAI and therefore can do even better. Perhaps we need some kind of competition for detecting machine generated vs human generated content?

nprateem 8y ago
That's a GAN.

resters 8y ago

While censoring the full data set seems in some way to support the rationale of the OpenAI charter, it also means that only state actors and very well-funded entities will be able to use the work to create models of the size necessary to do the impressive stuff in the write up. Based on the concerns, it would seem that restricting the capabilities only to state actors would have the opposite of the intended effect. Why not let thousands of amateur researchers, undergrads, etc., use the model to detect instances where the model was used to generate text, etc.?

skybrian 8y ago
I would guess this reduces the risk. Why would you say it does the opposite? My argument: state actors might misuse this tech, but letting any script kiddie do whatever they want almost guarantees someone will misuse it.
- mikeyouse 8y ago
  Which is exactly what happens whenever there's a leak of NSA or other foreign government 'hacking' tools. As soon as they're public, ransomware authors and other shitty actors all deploy them to steal as much as possible before systems are patched. Case-in-point: Wannacry: https://en.wikipedia.org/wiki/WannaCry_ransomware_attack https://en.wikipedia.org/wiki/WannaCry_ransomware_attack NotPetya: https://en.wikipedia.org/wiki/Petya_(malware)#2017_cyberattack https://en.wikipedia.org/wiki/Petya_(malware)#2017_cyberatta...
  - resters 8y ago
    I think it’s a bit different because the attacks are not entirely clear. The power of many of the attacks would be for them to fly under the radar because nobody was expecting it. So having lots of script kiddie attacks (which would be sloppier and easier to notice) would lead to a more rapid adoption of safeguards.
jfoster 8y ago
They can always revisit the decision once other entities seem to have made this progress. In the meantime, they don't have to be so disruptive.
lostmsu 8y ago
Or, in 3 years, anyone. I suggest everyone start thinking on ramifications now.

legatus 8y ago

I was honestly surprised by the quality of the generated text. While I can't say I've been following the state of the art in the last months, this seems like a pretty important step forward. Furthermore, at the end of the post they note that the samples are somewhat representative of their results. Maybe they should consider releasing a text file with some more (not hand-chosen) samples? Whatever the case, fantastic work, my congratulations to the authors.

wuthefwasthat 8y ago
Thank you! We've released 500 random unconditional samples from GPT-2 at https://github.com/openai/gpt-2/blob/master/gpt2-samples.txt https://github.com/openai/gpt-2/blob/master/gpt2-samples.txt
- legatus 8y ago
  Sorry about that, I should have not missed it. Since you're here, I'll ask a quick question to satisfy my curiosity. This is a pretty big step, even just looking at the benchmarks. Where do you see text generation going in 5 years? What about 10 years? As I said, no need to give long explanations, I just want to know what do you expect, as an insider,on an intuitive level.
- Scaevolus 8y ago
  Some of these are basically horror short stories: ======================================== SAMPLE 164 ======================================== I want to see a ton of households with kids with no food in their houses. Surviving famine requires storing up food in hopes a spoiled enclosure might provide the behaviors of food memories we crave. In near future, they will have to leave their cages. We're spending all our time theorizing about animal minds, yet we have living, reacting animals. As if we can just no be in touch with our emotions at is practical. My kale starts from an academic grown Himalayan. It hydrates in the garden, but it seems another species than we know is fermenting! My general lab is a recently renovated cave survival room with sun-filled bowls full of food for rats. The interior design is nearly utilitarian (no smart phones, blog, writing tools, or ovens there), except the cave ceiling. This is where we grow the food to feed animals. Which they will eventually eat. My girl starts out with a kind of scaffolding green and wild, but soon she can't grow more than a few leaves. Who needs her monotub artistically scissors and wire? The boxy roots are brittle right away before growing into skin weeds. So that's not going to do it. An unsteady start for another young carnivore's evening. I go in for some priming with baking soda and dry iron. A fun little experiment is once the kopi luwak seems she will grow some damp spikes. Consider this:chopsticks are soft under a microscope, webbies, and twining vines all apply more pressure from tangles of animal blood. Taking over the cultural mythos of science, I peek over her purple skeleton spikes with an endoscope. Once the higher organs take over, the kabuka unfurrows skin at a clever angle to reach past the body curse powered at this end of the lab. Curiously, without a hoe or a pot to turn into a tiled floor, she does not chew with her jaws. I have undressed her to her base to fill the bowl. She is pushing her way to get her shelter to stretch out into, or if I've planned correctly, squish into. She does not move like a biological creature, pulling up with her haunches while excellent and facing directly downward. She prefers laying there moss-covered and mother-prey encouraged paying attention to her uphill path. Lack of summer fur gives me ample room to poke through her skin, pull out black seeds, and taste the damp thing she'll roost under in future. Would the static reference very much reduce a virus being blocked within its first expenditure, nights? I'm not sure, but it's for sure a more effective inoculation. Hopefully, these experiments will be continued. Hopefully. My guru and master who made us does not focus so much on running where we want as it does on empathy for the soil,, plants, and animals with which he touched in some perfect way. Delhi is often flooded my first morning in the shafts of windy sunshine. Let's not give success an easy choice. The trees are in the underground and they change colors weekly. In simple terms, they're scattered flowers taking their cues from the transynthetic forest. While I'm pioneering, the one thing I sacrifice in the relationship is conducive to imagination. A full color-changing lets us enjoy cozy warmth while simultaneously losing whoever is formal and rational for the feel good activity. Like cosmic lap of the firmotsuites. An American outcropping of stonemines blocks the sunlight. The unseen wave depose colonies to accommodate my habit; sandblasting. There are microbes from amoebas that change the composition of the water. In the habitat where the natural and mecopate are as thoroughly ever so mindful of the fence as the meesthe gardener has been, I'll shed this duality when I sit with her. Food grows from space; human needs feed them. Sandbringation turning our microbes and animals can save thousands in pandemic deaths. I value them all in metaphorical and real (if tough) tenacity. Species survival deals good cop in these two every where storytelling machines. Door is opened. My guru goes rescuing the kupi luwak elder from drought resistant drought. I alone have learned her humdrum bookish biogeography is yielding to light and fire. Cow has problems taking roots. Bring my love to her and make a mend. She needs a clean-air daughtersnail easy experience not in a month's time, but in just a few hours of thinking about reflexes, calendar dates, and natural spatial relations for a mind that has grown certain.

winterismute 8y ago

Seems like in the near future the idea of "turing test" and the one of "not fake news" will eventually coincide...

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

yongjik 8y ago

> ... some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.” Man, the auto-generated text is hilarious. And uncannily good. Though I have to wonder if it's total random fluke or there's one among their 1.5 billion parameters that predict "likelihood of mythical bestiality in South America".

sanxiyn 8y ago
Considering that it memorized Gettysburg Address verbatim and knows Charles Darwin wrote Origin of Species, it probably knows about both South America and unicorns more than me... Unicorn story does demonstrate that it knows South America has to do with Argentina, Andes Mountains, and University of La Paz.

zellyn 8y ago

The generated Unicorn story has about the writing quality (in both senses: standard and feel) of fanfic

minimaxir 8y ago

The generated text sounds too good; is it possible that the model overfit the source material (especially since the n-previous-tokens value is infinite, while other approaches like char-rnns/textgenrnn use a fixed window length)? It's something I've encountered many times while working with text generation.

nmca 8y ago
Try googling sentences.
- minimaxir 8y ago
  Hmm, I tested a few sentences and it didn't turn up any exact matches (aside to this article), so maybe I'm wrong. With a temperature of 0.7/1.0, that's enough for sufficiently random text I suppose. (the raw, uncurated generated text using the smaller model is a bit more random: https://raw.githubusercontent.com/openai/gpt-2/master/gpt2-samples.txt https://raw.githubusercontent.com/openai/gpt-2/master/gpt2-s...)
  - wuthefwasthat 8y ago
    Those samples are from the large model (GPT-2)! Regarding memorization vs. generalization, see our paper for more analysis.
    - yorwba 8y ago
      Have you tried to determine which parts of the training data contributed to the model generating a certain output? I wonder whether the model avoids reproducing exact matches of the training data by splicing several similar articles together. For example, the generated text about the Civil War mentions that Thomas Jefferson Randolph [0] was named after his grandfather, the president. But is the wording mostly influenced by articles talking about that specific fact, or does it draw from more general examples of someone being named after their grandfather? [0] https://en.wikipedia.org/wiki/Thomas_Jefferson_Randolph https://en.wikipedia.org/wiki/Thomas_Jefferson_Randolph

resters 8y ago

Just as many pesticides mimic the hormonal and chemical signals of pests to drive certain behaviors that lead to eradication, this work mimics the linguistic signals of humans. I think viewing it metaphorically as the most sophisticated humanicide discovered to date is probably appropriate. Consider that conventional munitions make an effective pesticide but are not used due to their side effects. Instead, chemicals are used to destroy or mimic the perception and production of various signals so that populations of unwanted critters effectively self-destruct. Imagine a war fought with a weapon like this that left entire cities perfectly intact! # end hyperbole

honzzz 8y ago
This is exactly the line of thinking that this article inspired in me... AI scouring the internet, figuring out what makes us tick and generating perfectly persuasive stories to convince us to to... what? I kept thinking about the story "Sort by controversial" by Scott Alexander: https://slatestarcodex.com/2018/10/30/sort-by-controversial/ https://slatestarcodex.com/2018/10/30/sort-by-controversial/
est31 8y ago
https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World https://en.wikipedia.org/wiki/The_Funniest_Joke_in_the_World
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]

beefman 8y ago

I wonder how this would do on the Hutter Prize (I doubt it would beat the current record but I'm curious what the result would be) http://prize.hutter1.net http://prize.hutter1.net

gwern 8y ago
They include the enwik8 BPC estimates. It may not be strictly comparable - the most obvious issue is that since the HP takes the compression paradigm of intelligence, the compressor size is part of the total, and those 1.5b parameters certainly are not cheap. (This is one reason I think the HP is outdated. The corpus is not big enough to allow the superior asymptotics of approaches like RNNs or Transformers to compensate for their far larger binary size. HP is not measuring progress on an intelligence metric we care about, it's sort of measuring a 'demo scene' metric of intelligence.)

zawerf 8y ago

It's pretty interesting that their training set consists of "outbound links from Reddit which received at least 3 karma". There are definitely large subreddits which are flooded by highly voted fake news which you don't want to emulate (unless that's the goal). It also reminds me of a short fictional story which explores what would happen if an AI learn how to maximize reddit's sort by controversial score instead: https://slatestarcodex.com/2018/10/30/sort-by-controversial/ https://slatestarcodex.com/2018/10/30/sort-by-controversial/ Maybe that dystopian story is closer to reality than we thought?

gallerdude 8y ago

These samples are freaky good. We're approaching some threshold very, very fast. I'm not sure what that threshold is, and whether or not crossing it is a good thing, but soon we'll be there.

Matumio 8y ago
I like how it effortlessly switches into "git diff" mode at the end of sample 112. Sadly it doesn't do whitespace. > Showing 1 changed file with 4 additions and 19 deletions. +4 −19 png_source/colors/pointer.py Show comments View 8 png_source/colors/pointer.py @@ -35,6 +35,7 @@ def _draw_hull_class_level(self): repr(Shape[td_get_framepanel_pcs(dc) for dc in xrange(dc.cols)]), self.doublesize.values) \ } def _draw_hull_class_level [etc.] It also inserted a helpful reference link to http://wiki.openarcade.com/wiki/List_of_Programmer_Constants http://wiki.openarcade.com/wiki/List_of_Programmer_Constants (I've had to check: no, there is no openarcade wiki.) Also, sample 217 is some mediocre Java, with comments and all. Impressive how a single model can handle this all at once.
sanxiyn 8y ago
Sample 105 "In 2009, researchers at NASA from the Air Force Office of Scientific Research's Microwave Propulsion Laboratory held a series of contests, including one for "best brain-computer interfaces." In their prize match, teams from around the world competed to develop new brain mapping techniques for ways of collecting data from inside people and rewriting it over and over. One single original BrainBridge video colonized all channels of web-viewing in the world. I've watched many of these videos on YouTube. Advertisement - Continue Reading Below On one hand, if you're convinced that the next breakthrough in neuroprosthetics will be profoundly more powerful because better/faster or cooler/more reliable algorithms capture the wiring that connects our neurons, then it makes sense to equip humans with mind-reading implants. But when it comes to real brain-reading devices, there are a slew of caveats, arguments, and threats on the horizon that mean that these technologies cannot take off anytime soon. If you truly think that these technologies will be the basis for a global brain-reading surveillance state, it makes sense to act NOW. Since so much money is on the line (in computers and other components, research funding, patent and trademark rights, marketing, investor interest) making and maintaining a head-mounted-camera program is done with a certain level of speed by enterprises like Google and Facebook." That "Advertisement - Continue Reading Below" is very realistic indeed? These technologies, like GPT-2, will be the basis for a global brain-reading surveillance state, so indeed it makes sense to act NOW, like not releasing training data. Kudos. But this will be replicated with a certain level of speed by enterprises like Google and Facebook. Well said.

lucidrains 8y ago

Started a Google colab with the interactive text generation script. https://colab.research.google.com/drive/1da54684tFMjPbR5idbvoCyjOoEGwIVwV https://colab.research.google.com/drive/1da54684tFMjPbR5idbv...

marviel 8y ago
Thank you.
KVFinn 8y ago
Super appreciate this!
exit 8y ago
to be clear, this is the "politically innocuous" open sourced model. the results are not impressive.
- marviel 8y ago
  Yeah the results aren't amazing on their own, but if you treat them similarly to how they do over at Botnik -- with some human curation involved-- you can find some interesting sentences.
- lucidrains 8y ago
  Yea, somewhere in between the released model and the model they presented, the deep net has learned narrative structure.
- minimaxir 8y ago
  You need to run it a few times; the results vary very wildly.
minimaxir 8y ago
This is the approach I recommend; Colaboratory's GPU is helpful when the sequences are really long.

dzink 8y ago

This tech can easily be used to flood humanity’s shared brain with auto-generated propaganda. Schizophrenia of the internet in a way. There is plenty of incentive with Google algorithms favoring number of words and relevant keywords in content for rankings - you could have NLP bots lifting junk sites to top results. To step ahead in that chess game, a detection tool for fake would be just training grounds for better GAN. Instead we may see a certifying authority that labels content as human generated, certified fact maybe? Wikipedia and reddit are not safe without fast automatic moderation either. Do you have a brainstorm or idea/prototype submission site where people can submit approaches to countering bad ai actors? An white/grey hat ai bounty program of sorts?

olleromam91 8y ago
Great Idea...but better make sure it's not hosted on the internet.
comboy 8y ago
Web of trust. Your results and how much you trust given text is based on who gave it to you. You assign trust to people you know, then it's a small world effect and recursive trust calculation based on who they trust. Centralization won't work. Whether something is good or bad or fake must be subjective and based on your personal network / your beliefs. Otherwise, long term, I don't see how you could avoid dystopia.
- pmohun 8y ago
  This has a dangerous implication that we've seen recently with the newsfeed "bubble." If dissenting opinions are never allowed into your sphere of influence then we may continue to see an accelerating polarization of our society towards extremes.
  - comboy 8y ago
    Bubble that's dangerous for society is when everybody believes the same thing, not when everybody has different set of beliefs. I know the more popular bubble is information one and you make a fair point, but I believe it's powered more by recommendation systems than the things that you trust. I got distracted lately and I wanted to clean it up before putting it out there but if somebody is bored: http://comboy.pl/wot.html http://comboy.pl/wot.html

avivo 8y ago

This was only a matter of time. For the DEFCON AI Village in August I talked about the implications of this sort of tech, and how that impacts how we release "exploit" code / think about "cognitive vulnerabilities": https://medium.com/@aviv/what-does-a-world-with-automated-social-engineering-look-like-79cd09b5a7b1 https://medium.com/@aviv/what-does-a-world-with-automated-so.... If you are doing work in this space, either in ML research or related security, you need to be thinking about implications (also see e.g. https://maliciousaireport.com https://maliciousaireport.com).

levesque 8y ago
I mean, the ideas are there. The scope of the project is probably too big to reproduce for now, but eventually it will be accessible to your average spammer / scammer. We will get there. We won't be able to get these tools locked, make them exclusive for a certain type of responsible AI specialists. Someone will spill the beans, the models. People with bad intentions will reproduce these results. To me, the real deal is how we will manage these outbursts when they happen. I assume discriminative models will solve the problem for a while, but as with Generative adversarial networks, you will be able to train models that are harder to discriminate against. I posit we're in for a big societal change (maybe more a content crisis) sometime in the next 10 years. Pretty sure we won't be able to keep it from falling in bad hands.
- Nydhal 8y ago
  On the flip side, this will give an edge to specific groups that can communicate in highly vernacular/non-written/non-standard languages.
- ctoth 8y ago
  “Spam-filters, actually. Once they became selfmodifying, spam-filters and spam-bots got into a war to see which could act more human, and since their failures invoked a human judgement about whether their material were convincingly human, it was like a trillion Turing-tests from which they could learn. From there came the first machineintelligence algorithms, and then my kind.” I Row-Boat, Cory Doctorow, 2005: https://craphound.com/overclocked/Cory_Doctorow_-_Overclocked_-_I_Row-Boat-A4.pdf https://craphound.com/overclocked/Cory_Doctorow_-_Overclocke...

roywiggins 8y ago

I'd love to see what it would produce if you fed it the first sentence of your average Nigerian Prince scam. Fully-automated phishing- you could even automatically toss in a couple details about the recipient and let the AI riff on that for a bit.

songeater 8y ago

Is anyone else troubled by them not releasing the source model/dataset/parameters here? Yes, the technology can be used for malicious means - but would argue that "DeepFaking" language is FAR less of a problem than "DeepFaking" video/photo/audio... which already occurs. Seems like they went back on their charter to share AI developments broadly ("not concentrate power") under the excuse of "safety." (These results look fire btw) Note: copied my comment from dupe thread

ipsum2 8y ago
I agree with this. OpenAI isn't particularly 'open', compared to other AI research organizations (Notable ones that open source almost all their work are AllenAI and FAIR, but I'm sure there are others). Wonder what their excuse is for not releasing the source model or code for their DoTA bot. Surely there's no safety issues there?
_Microft 8y ago
Quote from their charter [0]: Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research. They referenced that from the article you commented on in the section 'release strategy' [1]: [...] we see this current work as potentially representing the early beginnings of such concerns, which we expect may grow over time. This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community. [0] https://blog.openai.com/openai-charter/ https://blog.openai.com/openai-charter/ [1] https://blog.openai.com/better-language-models/#releasestrategy https://blog.openai.com/better-language-models/#releasestrat...

fouc 8y ago

I think this would be extremely useful when we can do the inverse. Basically - can we detect if someone's writing is nonsensical or not? Can we detect if someone that is producing many well written essays is adhering to reality or not? Are they subtly re-defining terms, using flawed examples, etc? The generated example of the biologists discovering a unicorn herd is too convincing on its own. It's only because it's so outlandish that we get the sense it's fictional.

levesque 8y ago
Our models have reviewed your submission, and deemed it to be 87% incoherent, 52% redundant, and 24% fake. We therefore reject your submission. Sincerely, the Chief Bot Editor.

aaaaaaaaaaab 8y ago

In 10 years, content written by actual humans will be a premium niche, like tailored suits - reserved for the elites. The rest of us will be force-fed with machine-generated garbage.

Franciscouzo 8y ago
That doesn't makes sense, text can be distributed at marginal cost, tailored suits are expensive because there's not a lot of supply.
- aaaaaaaaaaab 8y ago
  Tailored suits used to be cheap too, but taylors went out of business due to mass production.
abecedarius 8y ago
There was an old science fiction novel set in a world like that. I've forgotten the name, but they called the robot-written stuff "wordwooze". (It doesn't work as a search term because some publisher is using it now.) Added: it was The Silver Eggheads by Fritz Leiber. An odd, forgettable book itself.
bitL 8y ago
But will ML companies pay link tax? Assuming ML would write/fill in forms to incorporate itself, have a conversation or two with some tax office person, and then off to the wild... Where can I study computational law to stay ahead of the curve?

cs702 8y ago

This kind of "blocking-and-tackling" work is important. The authors take a well-known architecture, the Transformer[a], configure it with a progressively larger number of parameter, train it to predict the next word conditioned on previous text, using a large dataset consisting of 40GB of text scraped from the Web, and test each trained model on a range of zero-shot transfer-learning tasks. Remarkably, the performance of a Transformer in the tested tasks improves log-linearly with the number of parameters, suggesting that even the largest model tested, with 1.5B parameters, still underfits 40GB of text. This is compelling evidence that we do NOT need new architectures, NOR new kinds of training objectives, NOR new theories, for better language modeling! We can get better language modeling simply by increasing model capacity (i.e., by adding more parameters to existing models), which becomes easier and simpler to do as hardware continues to improve over time. Great work. PS. In case it's not clear: I'm not saying we should suddenly stop searching for new, better ideas and architectures. That would be silly. Please don't attack a straw-man :-) [a] https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762

modeless 8y ago
Agreed that "simply" scaling up with more compute will result in progress and useful systems, and work in that direction is interesting and valuable. But, while we may not need new architectures or training objectives to make progress, we do need them to approach human level sample complexity. Humans don't need to read through 40 GB of text multiple times to learn to write.
- fartcannon 8y ago
  I guess the data requirements act as a stand-in for our genetic evolution... Our brain models are good at learning the things we learn. Our computer models aren't yet. /tautology
- cs702 8y ago
  > Agreed that "simply" scaling up with more compute will result in progress and useful systems, and work in that direction is interesting and valuable. But, while we may not need new architectures or training objectives to make progress, we do need them to approach human level sample complexity. Yes, agreed. Nothing I said above contradicts that! :-) > Humans don't need to read through 40 GB of text multiple times to learn to write. Yes, that's true... but to keep the comparison fair, note that we do need many years of schooling to learn to read, say, at a high-school or college level. And before learning to read, we first must learn to speak, which surely helps. And we also get to inhabit bodies that see, smell, touch, and interact with the physical objects that we read and speak about during our formative years, which also helps. The more one thinks about it, 40GB of data is actually a tiny figure in comparison to the amount of training data that flows continuously to our brain from all senses. I think I read once that our brains process on the order of 10 to 100 GB of training data per second.

deleted 8y ago

[deleted]

Cybiote 8y ago

It's becoming ever more certain that the transformer architecture is one of the largest contributions to AI (not merely machine learning, but AI), often beating LSTMs despite LSTMs being expressive enough to capture Turing Equivalence (at least in theory). Its main ideas are three: shorter paths help gradient flow, the training setup and the final key aspect, unhelpfully called self-attention. Self-attention is better thought of as a form of similarity gated key-value soft memory on which learning operations allows Transformers to learn non-trivial programs with contextual weights look-ups. I also notice reported tries, suggesting some level of curation. While this level of generation is undoubtedly impressive and a sign of non-trivial levels of understanding, the ability to project along arbitrary dimensions of similarity at a fine-grained level and to learn from text instruction is more useful than text generation. Although the unicorn story was a really fun read, better than many humans already, I doubt it could have gone on for much longer. It maintains a theme but not coherently or fluently (see especially the Kennedy nanotech and recycling examples, comparing the dis-fluency there versus the excellence of the Civil War report suggest at least some over-fitting). These relatively minor caveats aside, this is unambiguously an outstanding result. Winograd Schemas are the single metric to track if interested in understanding how language understanding is truly improving. OpenAI reports 71% and wrongly report the previous record as 63%. The current record here is at 65% https://gluebenchmark.com/leaderboard https://gluebenchmark.com/leaderboard though not fully comparable. Will OpenAI be submitting? Note that you can get to 60% using about 1-2 orders of magnitude less data and compute. It concerns me that results here are so far dependent on such large data and computation. However, based on several papers I've read, I do not believe this to be inherent even in transformers. I plan to do some experiments on this when I free up some bandwidth. If everyone is pulled in by the glamour of working for a well funded, prestigious operation then it should be no surprise that they do not consider paths which operate on several orders of magnitude less data and computational resources. We all should consider bringing about a group of researchers who swear to an austere computational life of a single GPU, no more than 4-8x average RAM and CPUs that do not cross 90 Watts. The Bicameral Order would be a good name for such a group.

wuthefwasthat 8y ago
Yeah, there are definitely still places the samples fall short! Keep in mind we're still using very naive sampling techniques. RE Winograd: WNLI is different, see https://arxiv.org/pdf/1804.07461.pdf https://arxiv.org/pdf/1804.07461.pdf

kaffee 8y ago

Not releasing the model? These people aren't scientists. edit: toned down a bit.

schoen 8y ago
Somehow, some part of me really wants to see what this model would generate with that as a prompt.
kaffee 8y ago
Actually, this prompted me to look up the funders of OpenAI: - Sam Altman - Greg Brockman - Reid Hoffman - Jessica Livingston - Elon Musk - Peter Thiel Useful information in interpreting the mission statement, I think.

plzHireMeImGood 8y ago

Is this comparable to Google BERT (Bidirectional Encoder Representations from Transformers) ? Benchmarks are different. Can I use any of this models for other tasks no mentioned in the papers, something more than the "fine tuning"?

teabee89 8y ago

The best part is "Scroll down for video" from https://blog.openai.com/better-language-models/#sample3 https://blog.openai.com/better-language-models/#sample3 :)

mark_l_watson 8y ago

Wonderful results. I Don’t think I will experiment with the smaller available model, at least right now. I am still happy with BERT, especially for basically solving anaphora resolution (coreference of pronouns, etc.)

dogcomplex 8y ago

Plot twist: all the comments on this thread were auto-generated.

mrfusion 8y ago

I’m most impressed by its ability to answer questions about the text. Why can’t someone built something like this on top of Wikipedia? That would be amazing to ask Wikipedia any question you can think of.

piggyzach 8y ago
Google has already built this, but it definitely isn't perfect. http://www.seobythesea.com/2014/10/google-fact-questions-entity-references-unstructured-data/ http://www.seobythesea.com/2014/10/google-fact-questions-ent...

mrfusion 8y ago

Can anyone do an eli5 of how this works?

rfdearborn 8y ago

Sure, not releasing the full trained model probably delays it, but sooner or later a bad actor will do their own scraping and train their own model and share it around and the genie will be out of the bottle. Then what? I think we need to be conducting AI research (and building software generally) under the assumption that all of it will eventually be repurposed by bad actors. How would our practices be different if we consistently and cautiously did this? Here's a thought experiment: how would the Manhattan project have been different if it were carried out in the open and its products were instantaneously and infinitely reproducible? What is the MAD equilibrium of AI research? I think the impact potential is similar even before AGI.

mlb_hn 8y ago
A lot of the advancements that made the Manhattan project were published by the Germans. On hearing about Hiroshima, Otto Hahn was "'shattered,' and went on to say that he felt 'personally responsible for the deaths of hundreds of thousands of people,' believing that his discovery had made the bomb possible." (https://www.washingtonpost.com/archive/opinions/1992/03/01/the-secret-nazi-atomic-tapes/42a8222d-5ec0-4d66-b335-a71064b4e290 https://www.washingtonpost.com/archive/opinions/1992/03/01/t...)
a-dub 8y ago
wasn't that the point of this whole openai thing? they didn't like the idea of there being a club with just google in it that had access to resources and funding to collect and train on massive datasets so they were going to be the "bad actors" who would do their own scraping, train their own models and share them around? isn't it supposed to be called OPENai? they don't want to share the data because they don't want to throw away the edge they've gained by collecting it. :) computer programs that generate human like text aren't dangerous, the internet is full of human like text that is mostly bullshit anyway.
- lc5G 8y ago
  I have had the same impression regarding their work on dota. They got a lot of publicity with it but their work is not open at all. They have released neither their code which runs the bots on dota2 nor their training code nor the final model. All we have is video recordings of a few games against humans.

grok2 8y ago

Nobody has asked about the animated text on the left at the top -- how is that done? That is more interesting to me!

marvy 8y ago
It's an mp4 video. But then the next question is: how did they make that video? And the answer to that is... I have no idea.

paraschopra 8y ago

Anyone who’s done large scale model training like this, an you shed light on following questions: What is the process like? Do you prototype locally? How do you confidence that the only limitation to good results is more compute power and NOT the model architecture or applicability of deep learning to a particular task? At what point do you decide that shelling many tens of thousands is OK? How often do you do large scale training only to find non-impressive results and hence the money wasted?

slashcom 8y ago
There’s a natural way to parallelize these models so that using 128 GPUs is the same as a 128x batch size. You can similarly simulate 128x batch size by accumulated gradients before backpropping. So you can test on just one or a few GPUs before you run the full thing. By that point you know it’s going to work, it’s just a matter of how well and whether you could’ve done nominally better with different tuning. There’s been enough research leading up to this paper to suspect that just scaling larger would play out.
- paraschopra 8y ago
  Thanks. >By that point you know it’s going to work, it’s just a matter of how well and whether you could’ve done nominally better with different tuning. This can't be true in all cases, right? I'm assuming that for many initially promising results on less-compute when they scale it, the results aren't impressive. I'm very curious to know what is the trials-to-success rate of publishable results when big-compute is thrown in the mix.
  - slashcom 8y ago
    It’s indeed a very high trials to success ratio. Again though, there’s enough papers preceding this one that you could have good confidence in the effort. Another thing that helps is orgs like OpenAI have their own servers, rather than renting ec2 instances. You also don’t just launch that many things and them ignore it. You monitor it to make sure nothing is going terribly wrong. But yeah there’s also the fact that if you’re Google, throwing $2m worth of compute at something becomes worth it for some reason (eg Starcraft)

lordnacho 8y ago

This is so crazy good, someone needs to do a Turing test by sending it to some unsuspecting publishers. I get the feeling that debatepocolypse is not far away. Every forum can now be spammed with reasonable sounding gibberish that humans will have to slog through.

gardaani 8y ago
Few MIT students did that with their simple text generator. It is one way to expose conferences with low submission standards. http://news.mit.edu/2015/how-three-mit-students-fooled-scientific-journals-0414 http://news.mit.edu/2015/how-three-mit-students-fooled-scien...
sanxiyn 8y ago
We must run a bot on HN with upvotes/downvotes for comments acting as reinforcement.
Retra 8y ago
That's not a Turing test. A Turing test is an ongoing dialog, not a one-shot filter.
- Isinlor 8y ago
  Add some bias to create dialogues when sampling based on movie subtitles and I could see it passing proper Turing test.
  - Retra 8y ago
    A proper Turing test allows you to ask questions whose answers must indicate novel introspection, learning, and social awareness. It's absurd to think any computer system today would pass such a test.
    - lordnacho 8y ago
      This thing does exactly that. You can see in the text it generates, it sounds like someone who's thought about things and keeps a consistent tone about the subject.

LHxB 8y ago

This reminds me of Dürrenmatt's "Die Physiker "(https://en.wikipedia.org/wiki/The_Physicists https://en.wikipedia.org/wiki/The_Physicists). While this has indeed very scary implications, one should be aware that if it's thinkable, eventually it will be thought (I'm paraphrasing here).

braindead_in 8y ago

Seems like magic! I wish I could do something similar with our chat support questions and answers. It would be nice to have something like this built-in.

lotaezenwa 8y ago

this will definitely be reverse engineered and open-sourced.[0] [0] https://en.wikipedia.org/wiki/Streisand_effect https://en.wikipedia.org/wiki/Streisand_effect

antpls 8y ago

It's beautiful that the reduced model itself is only 175 lines of Python code, thanks to TensorFlow : https://github.com/openai/gpt-2/blob/master/src/model.py https://github.com/openai/gpt-2/blob/master/src/model.py