5 ms·

The Unreasonable Effectiveness of Recurrent Neural Networks (2015)

65 points by sidcool 10y ago

okket 10y ago
Previous discussion: https://news.ycombinator.com/item?id=9584325 https://news.ycombinator.com/item?id=9584325 (458 days ago, 211 comments)
gavinpc 10y ago
Good article, but ugh, please leave Shakespeare alone if (as you freely admit) you don't know anything about Shakespeare. Just because the real thing sounds strange to you doesn't mean that your "samples" are anything but nonsense. edit To (preemptively) stress that I am not trolling: I find the topic and the writeup interesting and useful, and I appreciate the work that people do to share their experiences. But this is not a good example of the RNN working; it's a good example of the RNN not working, and should either be omitted or presented at least neutrally, instead of glibly saying > I can barely recognize these samples from actual Shakespeare :)
- scott_s 10y ago
  It works about as well/poorly as the other examples (Paul Graham, Wikipedia, math in LaTex). The samples from all of the examples are nonsense. What's interesting is that they, mostly, follow the form of the original.
  - gavinpc 10y ago
    Fair enough. My objection was that the OP implies this is a passable substitute, but okay, in the larger context it's clearly a joke.
- jcranmer 10y ago
  I think it's a usual tendency of NNs to produce output that looks meaningful to non-experts, yet is complete gibberish to experts. At a PLDI2016 keynote, the presenter gave this example from running an RNN on all PLDI submissions: "We present a convenient alias analysis that attains a number of queries in a typedef style." (fuller context, with the surrounding sentences being more bizarre may be found here: http://www.cs.ucsb.edu/~ckrintz/PLDI2016-keynote-final-zorn.pdf http://www.cs.ucsb.edu/~ckrintz/PLDI2016-keynote-final-zorn....).
adamwi 10y ago
Really interesting article (missed the previous post)! How far is this from being implementable for a practical application such as e.g. a "spellchecker and improvement ideas" for coding? E.g. train with "good" code from open source projects, have it running and highlighting areas with potential errors/improvement potential while you code (not counting trivial errors caught by e.g. lint tools).

65 points by sidcool 10y ago

okket 10y ago

Previous discussion: https://news.ycombinator.com/item?id=9584325 https://news.ycombinator.com/item?id=9584325 (458 days ago, 211 comments)

gavinpc 10y ago

Good article, but ugh, please leave Shakespeare alone if (as you freely admit) you don't know anything about Shakespeare. Just because the real thing sounds strange to you doesn't mean that your "samples" are anything but nonsense. edit To (preemptively) stress that I am not trolling: I find the topic and the writeup interesting and useful, and I appreciate the work that people do to share their experiences. But this is not a good example of the RNN working; it's a good example of the RNN not working, and should either be omitted or presented at least neutrally, instead of glibly saying > I can barely recognize these samples from actual Shakespeare :)

scott_s 10y ago
It works about as well/poorly as the other examples (Paul Graham, Wikipedia, math in LaTex). The samples from all of the examples are nonsense. What's interesting is that they, mostly, follow the form of the original.
- gavinpc 10y ago
  Fair enough. My objection was that the OP implies this is a passable substitute, but okay, in the larger context it's clearly a joke.
jcranmer 10y ago
I think it's a usual tendency of NNs to produce output that looks meaningful to non-experts, yet is complete gibberish to experts. At a PLDI2016 keynote, the presenter gave this example from running an RNN on all PLDI submissions: "We present a convenient alias analysis that attains a number of queries in a typedef style." (fuller context, with the surrounding sentences being more bizarre may be found here: http://www.cs.ucsb.edu/~ckrintz/PLDI2016-keynote-final-zorn.pdf http://www.cs.ucsb.edu/~ckrintz/PLDI2016-keynote-final-zorn....).

adamwi 10y ago

Really interesting article (missed the previous post)! How far is this from being implementable for a practical application such as e.g. a "spellchecker and improvement ideas" for coding? E.g. train with "good" code from open source projects, have it running and highlighting areas with potential errors/improvement potential while you code (not counting trivial errors caught by e.g. lint tools).