4 ms·
Is the model actually trained (=updating the weights instead of just the state) during en/decoding?
by threatripper 5y ago
Is the model actually trained (=updating the weights instead of just the state) during en/decoding?
- GistNoesis 5y agoYes that's the trick, the model is trained both in the encoding and the decoding from the data that is encoded/decoded, continuously as data flows, from the same scratch in both cases and then trained with exactly the same data (the raw data). Because of training is deterministic and the same data is presented during encoding and decoding, the weights of encoder and decoder are always kept in sync, such that the network used when encoding the nth byte is exactly the same as the network used when decoding the nth byte. The weights of the network are the internal state.