4 ms·

Am curious about what other sense of 'equivalence' you had in mind when you stated with great authority that RNNs are not Markovian.

• raverbashing 8y ago
  In the practical sense, implementing a RNN is easier than a Markov Chain (libraries, etc) Also RNNs can "evolve" (or be adapted) more easily to other architectures like LSTMs.
  • srean 8y ago
    That line of argument does not hold as RNNs are are just special cases of hidden Markov models. If RNNs are easy to implement it means some hidden Markov models are as well. In any case by ease of implementation I think you men availability of libraries rather than something fundamental to the approach. Anyway we have strayed from your claim that RNNs are not Markovian. I was struck by the confidence in the claim especially so because it isnt true by a long shot.
  • mratsim 8y ago
    If you try implementing a RNN from scratch with the same base as the OP N-order Markov Chain and without Numpy or a deep-learning frame work, I guarantee you that you will not find it so easy. For reference his code his 400 lines and this is what he is importing: #include <stdio.h> #include <vector> #include <string> #include <unordered_map> #include <random> #include <algorithm> #include <array> We can assume that <math.h> import is reasonable for comparison To implement RNNs for text you need to implement tensors/ndarrays of 3 dimensions with slicint and all that jazz, efficient matrix multiplication, implement the RNN, implement the RNN backprop properly. RNNs stands on the shoulder of giants.