3 ms·

Thanks for the explanation. Your description superficially reminds me of a Markov chain (https://en.wikipedia.org/wiki/Markov_chain https://en.wikipedia.org/wik

Thanks for the explanation. Your description superficially reminds me of a Markov chain (https://en.wikipedia.org/wiki/Markov_chain https://en.wikipedia.org/wiki/Markov_chain). Is this related or is it totally different?

LeanderK 8y ago
I haven't read the paper the work is based on, but if the RNN outputs a probability distribution for the next letter/word then they form Markov Chains (since then they only depend on the current state and not the previous state)! RNNs are just fancy parametric functions that take a (state, input)-pair and return a new (state', output)-pair.