It's not surprising that this information is somehow saved in the internal state, but it is surprising that they were able to read it to some level. I don't th

It's not surprising that this information is somehow saved in the internal state, but it is surprising that they were able to read it to some level. I don't think the model can access this information, but an external "lie detector" would be interesting. Thanks for the paper.