I’m not convinced. I tried manually predicting high score vs low score as well and couldn’t do so reliably. The language model (viewable at https://stackrobofl

I’m not convinced. I tried manually predicting high score vs low score as well and couldn’t do so reliably. The language model (viewable at https://stackroboflow.com https://stackroboflow.com ) scores much higher accuracy predicting the next word on Stack Overflow than other datasets like IMDb. But on the IMDb dataset this approach led to state of the art sentiment analysis results by pulling out the encoder and adding a custom head whereas on Stack Overflow it didn’t grok anything about the score.