If you're using TF 1.2.1 or above, you want to replace this line: cell = tf.contrib.rnn.MultiRNNCell([drop_cell] * num_layers) with: cell = tf.contrib.rnn.Mu

If you're using TF 1.2.1 or above, you want to replace this line: cell = tf.contrib.rnn.MultiRNNCell([drop_cell] * num_layers) with: cell = tf.contrib.rnn.MultiRNNCell([create_cell() for _ in range(num_layers)]) otherwise you'll be using the same weights for each LSTM layer and your model is less expressive.