Hey Smerity - I really appreciate your and MetaMind's work, and since you linked NMT, I wonder if you could shed light on a question I've always had about Seq2s

Hey Smerity - I really appreciate your and MetaMind's work, and since you linked NMT, I wonder if you could shed light on a question I've always had about Seq2seq models. Does the decoder side/ phase backprop all the way through into the encoder states? I assume it must (otherwise it seems like it'd be more efficient to do the encoding in its own process outside of seq2seq), but doesn't that mean it creates a weird hybrid state between the two languages in the encoder hidden states, so that then when attention is applied to the encoder states during decoding, it's not even looking at the original/ source words anymore (it's looking at the weird states)... thus I feel like everyone is technically lying when they say "Let's use attention to peak at the source words when you decode." Is my line of reasoning here sensible? Also I love the Denny Britz link. Thanks!