3 ms·

Training Recurrent Neural Networks [pdf]

88 points by e19293001 10y ago

the8472 10y ago
> The certificate is only valid for the following names: www.cs.toronto.edu, cs.toronto.edu
- walty 10y ago
  You can just change the URL to https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_thesis.pdf https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_the... to get valid https.
  - Jimbo_029 10y ago
    Could you explain how this works?
    - nayuki 10y ago
      The University of Toronto owns both the domains toronto.edu and utoronto.ca. Some sub-sites work on both domains, and some don't. Apparently they misconfigured their servers so that they are using the toronto.edu certificate on a utoronto.ca site.
joe_the_user 10y ago
Also needs 2013 designation, Especially important as neural net knowledge seems to be evolving quickly. And perhaps someone can explain why paper matters relative to the plethora of papers and approaches "out there".
e19293001 10y ago
I found this from neural-networks course in coursera[0]. The author of this paper had been discussed as an example of what recurrent neural nets can now do. Here's the description from the slide: •  Ilya Sutskever (2011) trained a special type of recurrent neural net to predict the next character in a sequence. •  After training for a long time on a string of half a billion characters from English Wikipedia, he got it to generate new text. – It generates by predicting the probability distribution for the next character and then sampling a character from this distribution. – The next slide shows an example of the kind of text it generates. Notice how much it knows! Some text generated one character at a time by Ilya Sutskever’s recurrent neural network: In 1974 Northern Denver had been overshadowed by CNL, and several Irish intelligence agencies in the Mediterranean region. However, on the Victoria, Kings Hebrew stated that Charles decided to escape during an alliance. The mansion house was completed in 1882, the second in its bridge are omitted, while closing is the proton reticulum composed below it aims, such that it is the blurring of appearing on any well-paid type of box printer. [0] - https://www.coursera.org/learn/neural-networks/ https://www.coursera.org/learn/neural-networks/
- florianletsch 10y ago
  You probably know this already, but if you are just looking for an illustrated demonstration of what RNNs are capable of in the text domain, probably the best brief article is the post by Andrej Karpathy about [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/).
eveningcoffee 10y ago
Considering that it is from 2013 and from Hinton team then are the Restricted Boltzmann Machines actually necessary for this?

88 points by e19293001 10y ago

the8472 10y ago

> The certificate is only valid for the following names: www.cs.toronto.edu, cs.toronto.edu

walty 10y ago
You can just change the URL to https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_thesis.pdf https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_the... to get valid https.
- Jimbo_029 10y ago
  Could you explain how this works?
  - nayuki 10y ago
    The University of Toronto owns both the domains toronto.edu and utoronto.ca. Some sub-sites work on both domains, and some don't. Apparently they misconfigured their servers so that they are using the toronto.edu certificate on a utoronto.ca site.

joe_the_user 10y ago

Also needs 2013 designation, Especially important as neural net knowledge seems to be evolving quickly. And perhaps someone can explain why paper matters relative to the plethora of papers and approaches "out there".

e19293001 10y ago

I found this from neural-networks course in coursera[0]. The author of this paper had been discussed as an example of what recurrent neural nets can now do. Here's the description from the slide: •  Ilya Sutskever (2011) trained a special type of recurrent neural net to predict the next character in a sequence. •  After training for a long time on a string of half a billion characters from English Wikipedia, he got it to generate new text. – It generates by predicting the probability distribution for the next character and then sampling a character from this distribution. – The next slide shows an example of the kind of text it generates. Notice how much it knows! Some text generated one character at a time by Ilya Sutskever’s recurrent neural network: In 1974 Northern Denver had been overshadowed by CNL, and several Irish intelligence agencies in the Mediterranean region. However, on the Victoria, Kings Hebrew stated that Charles decided to escape during an alliance. The mansion house was completed in 1882, the second in its bridge are omitted, while closing is the proton reticulum composed below it aims, such that it is the blurring of appearing on any well-paid type of box printer. [0] - https://www.coursera.org/learn/neural-networks/ https://www.coursera.org/learn/neural-networks/

florianletsch 10y ago
You probably know this already, but if you are just looking for an illustrated demonstration of what RNNs are capable of in the text domain, probably the best brief article is the post by Andrej Karpathy about [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/).

eveningcoffee 10y ago

Considering that it is from 2013 and from Hinton team then are the Restricted Boltzmann Machines actually necessary for this?